KR100953861B1 - End point detection method, mouse device applying the end point detection method and operating method using the mouse device - Google Patents

Abstract

Removing the DC bias of the acceleration signal output from the accelerometer; Low pass filtering to remove noise of the acceleration signal; Obtaining σ ^p (k) for a k th sample A (k) of the acceleration signal subjected to the low pass filtering step, where σ ^p (k) is a standard deviation or variance; Obtaining a local minimum σ ^p _LV (n) from the σ ^p (k); Obtaining a local maximum σ ^p _max (n) between two adjacent σ ^p _LV (n); Detecting a starting point and an ending point by comparing the sigma ^p _max (n) with a threshold sigma ^p _th ; The end point detecting method, a mouse device and a mouse device in accordance with this including a; the σ ^p (k) a local minimum value (local minimum) σ ^p using the _LV (n) extracting the pole of A (k) in Discuss the method of operation.

Description

End point detection method, mouse device applying the end point detection method and operating method using the mouse device}

The present invention relates to an endpoint detection method, a mouse device using the same, and a method of operating the same, and more particularly, a mouse device integrating a plurality of button input functions and an in-screen pointing function of a three-dimensional mouse, such as a TV remote control, and a method of operating the same. The present invention relates to a new end point detection method that can improve button input and pointing accuracy in a mouse device.

Unlike traditional computing environments that use a TV and a computer as separate devices, the era of home automation is coming, in which a TV and a computer are integrated into a home server. In a home automation system, all the control is done through a home server, which is an effective way to monitor this and a large flat panel display is placed in the center of the home. In addition, the demand for home robots is expected to increase dramatically for the implementation of home automation, and home robots are expected to be controlled by a network controlled by a home server.

These changes in computing environment have triggered the integration between peripheral devices such as computers and TVs, and the commercialization for them is being activated. The integration of broadcasting, telecommunications and control environment assumes the basic assumption and proceeds, so the ripple effect should be applicable to all generations regardless of age.

However, this trend of change is a big obstacle for the silver generation who is passive in changing the environment. The present invention aims to provide a next-generation information input device incorporating a silver generation, which is alienated in information, as a main user, and integrating a TV remote controller and a computer mouse.

Physically hearing / visual aging has limitations on use for small and complex objects and slows down the integration of information due to degeneration of the cranial nervous system. This is in contrast to the industry trend of digital convergence, which integrates multiple functions in one device, which is unique to the Silver generation. In addition, emotionally resisting the use of new devices through learning, the emphasis is on the convenience rather than the functionality of the product. In other words, they tend to stick to the media they are already used to and do not want to accept new changes.

Reflecting this feature of the silver generation, there is a need to provide a mouse device capable of minimizing complexity by minimizing the number of buttons and diversifying information transmission methods. In addition, intuition should be maximized so that it can be used without new learning, and it should be possible to minimize the objection to the use of the mouse by using a medium familiar to the Silver generation.

In view of the above-described necessity, the present invention has to be prioritized to determine the movement of a hand holding a mouse in simulating a mouse function using only an accelerometer sensor. The present invention proposes a new method for performing an end point detection method and a feature component extraction function for pattern recognition at the same time, and provides a new mouse device having improved button input and pointing accuracy by applying the end point detection method, and a method of operating the same.

The mouse device of the present invention performs a coordinate transformation function for pointing and a gesture recognition function for input of a numeric button. The coordinate conversion function and the motion recognition function are switched by a button for function classification.

When performing the coordinate transformation function by detecting when the mouse device is moved in the off state of the function classification button, the end point detection method (EPS) can be used to perform the coordinate conversion function without removing the offset error. Get the start and end points and use them to transform the coordinates. When performing the motion recognition function by detecting the movement of the mouse device when the function classification button is pressed on, the start point and the end point of the motion are obtained by using the end point detection method (EPS), and then the feature The operation is recognized by extracting and pattern recognition through the ZVC algorithm.

Therefore, the acceleration signal input while the function classification button is turned on is pattern-recognized through the ZVC algorithm after the endpoint detection method. The acceleration signal input when the function classification button is turned off is coordinated by analyzing the relationship between the poles existing between the start point and the end point after the start point and the end point are detected.

In order to achieve the above object, the endpoint detection method of the present invention, the step of removing the DC bias of the acceleration signal output from the accelerometer; Low pass filtering to remove noise of the acceleration signal; Obtaining σ ^p (k) for a k th sample A (k) of the acceleration signal subjected to the low pass filtering step, where σ ^p (k) is a standard deviation or variance; Obtaining a local minimum σ ^p _LV (n) from the σ ^p (k); Obtaining a local maximum σ ^p _max (n) between two adjacent σ ^p _LV (n); Detecting a starting point and an ending point by comparing the sigma ^p _max (n) with a threshold sigma ^p _th ; It includes.

Meanwhile, the endpoint detection method of the present invention obtains σ ^p (k) (wherein σ ^p (k) is a standard deviation or variance) for an acceleration signal output from an accelerometer, and calculates the σ ^p (k) region. After selecting the found local minimum values as a candidate group of the starting point and the end point, selecting the starting point and the end point in the candidate group by comparing the local maximum values between the local minimum values with a threshold value, and then eliminating the offset error existing in the starting point and the end point. It is done.

The mouse device of the present invention includes an accelerometer unit including at least one accelerometer and a function distinguishing button, and converting an acceleration signal output from the accelerometer into a digital signal; Obtain A (k), which is low pass filtered data of the digital signal output from the accelerometer unit, detect a start point and an end point in a region of standard deviation or variance with respect to A (k), and recognize a change in coordinate information. Or a controller configured to recognize an operation by removing a offset error existing within a section defined by the start point and the end point, and extracting a feature component from the acceleration signal to perform pattern recognition; It includes.

In addition, the operating method of the mouse device of the present invention, removing the DC bias of the acceleration signal output from the accelerometer; Low pass filtering to remove noise of the acceleration signal; Obtaining σ ^p (k) for a k th sample A (k) of the acceleration signal passed through the low pass filtering step, where σ ^p (k) is a standard deviation or variance; Obtaining a local minimum σ ^p _LV (n) from the σ ^p (k); Obtaining a local maximum σ ^p _max (n) between two adjacent σ ^p _LV (n); Detecting a starting point and an ending point by comparing the sigma ^p _max (n) with a threshold sigma ^p _th ; Recognizing coordinate transformation information of the mouse device; A zero velocity compensation (ZVC) step of eliminating offset errors of the acceleration signals existing at the start point and the end point; Recognizing a motion of the mouse device by extracting a feature feature of the acceleration signal and pattern recognition; It includes.

On the other hand, the operation method of the mouse device of the present invention, whether to perform the coordinate conversion function of the mouse cursor according to whether the on / off (On / Off) of the function classification button attached to the pen type mouse device performs a motion recognition function Determine whether or not to calculate σ ^p (k) for the acceleration signal output from the accelerometer, where σ ^p (k) is the standard deviation or variance, and detect the local minimum value at σ ^p (k), After selecting the local maximum value in the section between the local minimum value and comparing with the threshold value, select the starting point and the end point of the local minimum value, if the button for the function classification is ON state recognizes the signal between the starting point and the end point and operation recognition When the function classification button is in the off state, the coordinate transformation information of the mouse cursor is acquired.

The mouse device of the present invention is aimed at integrating a button input function such as a general TV remote controller and a coordinate transformation function of a 3D mouse cursor, and has an advantage of minimizing complexity and maximizing intuition and particularly suitable for silver generation. .

The endpoint detection method of the present invention can be used to replace the button input function and transform the coordinate information of the mouse cursor by focusing on motion recognition using an accelerometer in order to maximize intuition.

This method of detecting the end point and the operation of the mouse device is not the estimation of the trajectory by integrating the acceleration for motion recognition, but the pattern detection with the acceleration signal itself. Extreme Points Sampling) "algorithm.

According to the operating method of the mouse device of the present invention, the change of coordinate information can be obtained simply and quickly.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this process, the size or shape of the components shown in the drawings may be exaggerated for clarity and convenience of description. In addition, terms that are specifically defined in consideration of the configuration and operation of the present invention may vary depending on the intention or custom of the user or operator. Definitions of these terms should be made based on the contents throughout the specification.

A commonly used mouse function is a coordinate conversion function for pointing the cursor when the cursor on the monitor screen is moved up, down, left, and right. A general remote controller used as an input means such as a TV has a button function such as a number key and a channel key. It is listed. The mouse device proposed in the present invention is configured to integrate a coordinate transformation function for pointing a general mouse and a motion recognition function for button input of a general remote controller.

In the present invention, in order to provide a mouse device suitable for the silver generation, attention is given to a 'pen' as an information input device that has been familiarly used before information input devices such as a keyboard and a mouse. As a result, a pen-type mouse device is provided as an accelerometer-based gesture recognition means that responds intuitively to hand movements in order to make the most of intuition. However, the shape of the mouse device of the present invention is not necessarily limited to the pen type. Various embodiments of the mouse device may be used to achieve the object of maximizing intuition. The button function and coordinate transformation are implemented by a motion recognition algorithm using an accelerometer provided in the mouse device.

First, an operation recognition function for button input will be described. Early work on estimating motion using only three-axis accelerometers has gradually evolved into the addition of three-axis gyro sensors or six additional degrees of Inertia Measurement Unit (IMU). However, since three-axis gyro sensors and IMUs are expensive and large in size, and have disadvantages in commercialization, compared to three-axis accelerometers, it is an object of the present invention to implement a trajectory estimation using only a three-axis accelerometer but increase recognition accuracy.

On the other hand, an additional computer vision system can be used to recognize the motion, but it is not preferable in terms of cost and size of the device, and additionally, a GPS system or an optical sensor can be added to recognize the motion, but it is not desirable in terms of device size. In the present invention, the trajectory is estimated using only a 3-axis accelerometer. The main tasks to be solved when estimating trajectory using only 3-axis accelerometer are as follows.

In a pen-type mouse device equipped with an accelerometer, an error that occurs when the direction of gravity changes due to a change in inclination, a measurement error of the accelerometer itself, a rotational drift error that occurs during rotation operation, and the like cause a drift error of the accelerometer. Among them, the error caused by the change of the slope is the most fatal, so it is necessary to secure the ability to cope with such an error.

When the acceleration is A (k) at a discrete time t = k in a posture in which the pen 10 is initially inclined with respect to the horizontal plane 20 as shown in FIG. 1, the acceleration is expressed as Equation 1 below. .

A (k) = A _motion (k) + {gsinφ + gsin (φ-θ (k))} + A _bias

Here, A _motion (k) is a dynamic acceleration generated by the movement of the pen 10, and A _bias is a self error of the accelerometer. {gsinφ + gsin (φ-θ (k))} is the positive acceleration generated by gravity, and φ-θ (k) is between the pen 10 and the horizontal plane 20 which changes according to the movement of the pen 10. It represents the deviation of the angle that occurs.

The first graph on the left of FIG. 2 is the actual trajectory of the pen movement, and the second graph of FIG. 2 is the acceleration signal measured by three accelerometers separately installed on the three axes of the pen while moving the actual trajectory, The third graph is the estimated trajectory of integrating the acceleration signal twice. If the inclination of the pen at the start and end points does not coincide, the acceleration signal from the start and end points of the actual trajectory will also change, so that the velocity of one integration or the estimated trajectory of two integrations is offset error (or drift) drift) error. The integral trajectory actually differs from the trajectory of the pen movement because the integral includes the offset error of the acceleration signal for the start and end points, which increases the integrated trajectory in the positive infinity direction. to be.

As a countermeasure, a method of eliminating the offset error of the acceleration signal with respect to the start point and the end point by using a high pass filter may be considered, but it may cause damage to the acceleration signal. As another countermeasure, a method of eliminating an offset error of an acceleration signal using a zero velocity compensation (ZVC) algorithm may be considered. The present invention does not propose a Zero Velocity Compensation (ZVC) algorithm itself, but provides a new endpoint detection method and an efficient and fast mouse device operation method capable of accurately performing the ZVC algorithm.

3 illustrates ZVC in graphs and equations. The pen movement distinguishes the beginning and the end by whether the momentary motion is stopped. Thus, the velocity is almost zero at the starting point k ₁ and the ending point k ₂ . It is ZVC to use this to correct the offset error of the acceleration signal.

For convenience of explanation, the acceleration signal output from the x-axis accelerometer is shown in FIG. 3. First, the top graph shows the acceleration signal A _x output from the x-axis accelerometer. The horizontal axis represents the sampling number. The second graph shows the velocity signal V _x as the integral of the acceleration signal A _x once. The velocity for the k th sample is given by Equation 2 below.

Where T _s is the sampling period.

는 다음의 수학식 3과 같다.Next, the velocity signal before the starting point (k ₁ ) is zero, and the velocity signal is obtained by removing the offset component of the starting point (k ₁ ) from the velocity appearing in the interval after the starting point.

Is as shown in Equation 3 below.

는 다음의 수학식 4와 같다.Next, the velocity signal after the end point k ₂ is set to 0, and the velocity signal obtained by removing the offset component obtained at the end point k ₂ over the interval between the start point and the end point.

Is the same as Equation 4 below.

를 구할 수 있으며, 다음의 수학식 5로 표시된다.By differentiating this, the acceleration signal eliminates the offset error by the ZVC algorithm.

It can be obtained, and is represented by the following equation (5).

를 ZVC알고리즘으로 오프셋 오류를 제거한 값인

는 다음의 수학식 6과 같이 구해진다.Therefore, the kth sampled acceleration signal for the nth accelerometer

Is a value that eliminates offset errors with the ZVC algorithm.

Is obtained as in Equation 6 below.

Looking at ZVC described above, there are some limitations in the algorithm.

First, it is necessary to assume that the change in inclination φ-θ (k) of the pen that can occur between the starting point k ₁ and the ending point k ₂ is monotonically increasing or monotonically decreasing. The physical meaning of this assumption is indicated by reference numeral 401 of FIG. 4. To satisfy this assumption well, Δk = k ₂ This entails a constraint that k ₁ must be short.

Second, ZVC cannot be applied until the endpoint k ₂ is found. Therefore, ZVC can not be applied to on-line correction but only to off-line correction processing.

Third, the performance of ZVC is unreliable if the start point (k ₁ ) and end point (k ₂ ) are not found correctly. That is, the reliability of endpoint detection performance is directly related to the reliability of ZVC. The tendency of the reliability of the ZVC processed acceleration signal to deteriorate when the start point k ₁ and the end point k ₂ is not properly found is indicated by reference numerals 501 and 502 of FIG. 5. Therefore, it can be seen that the performance of the endpoint detection algorithm is closely related to the performance of ZVC.

Endpoint detection is addressed in the problem of pattern recognition of time series data and is an important research theme in the field of Automatic Speech Recognition (ASR). In the speech recognition field, endpoint detection refers to a process of extracting only speech data to be applied to pattern recognition from the time series data obtained in real time and discarding the rest. In speech recognition, an appropriate threshold is given to the energy magnitude of the signal or the frequency magnitude of the signal. The signal between the start point signal exceeding the threshold value and the end point smaller than the threshold value is taken as a useful signal. However, the endpoint detection algorithm developed in the speech recognition field is suitable for a speech signal having a higher frequency than the accelerometer signal and cannot be applied to an accelerometer signal having a relatively low frequency.

Thus, the present invention proposes a novel endpoint detection method suitable for accelerometer signals having a relatively low frequency.

It is physically obvious that acceleration and distance are in the second derivative. However, there are several limitations in applying this to discretized time series signals. The previously proposed ZVC is to solve this problem, but it has the limitations described above. In particular, if the trajectory includes a curve like a circle, the drift error caused by the second integration of the acceleration is more severe. In order to compensate for this, a three-axis gyro sensor may be used together. However, as described above, the price is expensive and large, and it is impossible to block the drift error as well as the disadvantages of commercialization.

Therefore, rather than integrating the accelerometer signal and estimating the trajectory, a pattern recognition method is applied to the accelerometer signal. In other words, it is intended to obtain a motion recognition performance that can be commercialized only by pattern recognition of a time-series signal of a 3-axis accelerometer. Accordingly, the present invention provides a method for motion recognition by extracting feature components and pattern recognition of a time series signal without using a conventional method of trajectory estimation by second order integration of an acceleration signal. Suggest a method.

)를 표시하며, 일점 쇄선은 표준 편차의 문턱값(σ_th )을 표시한다. 도 6의 예에서, 샘플 수 S=10, 문턱값 σ_th=0.28, 동작을 만들기 위한 최소한의 샘플 수 W=50, H=20, 샘플링 주기 T_s=100 msec이다. Figure 6 is a comparative example of the endpoint detection method presented for comparison with the endpoint detection method of the present invention. In Fig. 6, the solid line represents the low pass filtered value of the absolute value (│An│) of the acceleration signal for the nth accelerometer, and the solid line partially indicated by the circle is the portion of the absolute value of the acceleration signal (│An│). Standard Deviation(

), And the dashed-dotted line indicates the threshold value of standard deviation (σ _th ). In the example of FIG. 6, the sample number S = 10, the threshold σ _th = 0.28, the minimum number of samples W = 50 for making the operation, H = 20, and the sampling period T _s = 100 msec.

We have already explained that endpoint detection performance is closely related to the performance of ZVC. Hereinafter, a problem of the endpoint detection algorithm of the comparative example shown in FIG. 6 will be described and an endpoint detection method of the present invention for solving the problem will be described.

The endpoint detection algorithm presented in FIG. 6 is summarized by the following equation.

는 선택된 어느 하나의 가속도계에서 출력되는 가속도의 절대값인│A│의 표준 편차를 S 샘플마다 구한 값이다. 따라서,

의 관계가 성립한다. 결국

의 값이 문턱값 σ_th 보다 커진 후에 H 기간 동안 문턱값보다 큰 값을 유지할 경우 시작점(k₁)으로 잡는다. 반대로

의 값이 문턱값보다 작아진 후에 H 기간 동안 문턱값보다 작은 값을 유 지할 경우 끝점(k₂)으로 추출한다. here,

Is the value obtained for each S sample of the standard deviation of | A | which is the absolute value of the acceleration output from the selected accelerometer. therefore,

The relationship is established. finally

If the value of is greater than the threshold value σ _th and then maintains a value larger than the threshold value for H period, it is set as the starting point k ₁ . Contrary

After the value of is smaller than the threshold value, if the value is smaller than the threshold value during the H period, it is extracted as the end point (k ₂ ).

In this case, the same equation and the same result can be obtained by using the variance instead of the standard deviation in the end point detection method proposed in the present invention, as well as Equation 7 and FIG. 7 for comparison with the present invention. Is the same first.

를 유지하는 순간을 시작점과 끝점으로 추출하는 도 6에 도시한 비교예는 다음과 같은 알고리즘의 문제점을 갖는다. Consistent over time

The comparative example shown in FIG. 6 extracting the moment of maintaining the starting point and the ending point has the problem of the following algorithm.

First, since there are four parameters (H, S, W, σ _th ) that need to be set, it is difficult to find the optimal combination, which in turn hinders the accurate recognition of the operation in a short time.

를 구할 때 S 샘플을 단위로 하기 때문에 대략적인 끝점을 검출하게 된다. S의 크기를 줄이면 시간상의 정확도는 증가하지만 동작에 대한 민감도 역시 증가해 무의미한 사소한 움직임에도 반응을 보이게 된다.second,

Since we are using S sample as a unit, we find the approximate end point. Reducing the size of S increases the accuracy in time but also increases the sensitivity to motion, making it responsive to nonsensical minor movements.

In order to supplement the end point detection method of the comparative example described in FIG. 6 and to present a more reliable end point detection method, the present invention proposes a new end point detection method called "Extreme Point Sampling (EPS)". We want to improve the accuracy of.

Meanwhile, the partial standard deviation or partial variance described in FIG. 6 is a subordinate concept of the standard deviation for each section or the variance for each section described below, and is a concept belonging to one embodiment. For example, if there are 25 samples in total, if you divide the interval into 5 units among them, you will get 5 total intervals, and if you calculate the standard deviation for each interval, you will get 5 standard deviations. It is also the same). The method of obtaining the partial standard deviation can be shown in the following table.

On the other hand, the standard deviation for each section (same as the variance for each section) described as an embodiment of the present invention is equally 25 samples, at which time the standard deviation is calculated by taking five intervals. After that, it is moved sideways by 1 (where 1 corresponds to the movement length, and the movement length can be arbitrarily selected). Then, the interval is divided into five units and the standard deviation is calculated. In this way, 25 piecewise standard deviations are calculated as shown in the following table. In this case, the overlap length x is equal to 4 minus 1, which is the length of the movement. If the moving length is changed to 2, the overlap length x becomes 3 minus the moving length 2 minus 3, which is the length of a predetermined section, thereby improving the processing speed in the calculation.

That is, the overlap length x of the standard deviation for each section x = the length of the section-the moving length N, the moving length N is an integer value greater than or equal to 1 and less than or equal to the length of the predetermined section. If the moving length N is equal to the length of a certain section, the overlap length x becomes 0, and the calculated standard deviation for each section or variance for each section is the partial standard deviation or partial variance described above.

다시 설명하면, 계산시 처리 속도를 더욱 빠르게 하기 위해서는 일정 구간의 길이를 줄이고 이동 길이 N 을 일정 구간의 길이와 같아질 때까지 늘림으로써 중복 길이 x를 0으로 놓는 방법도 가능하다. 이때는 일정 구간별 표준 편차는 부분 표준 편차와 동일하게 된다.
본 발명의 σ^p(k)는 상술한 바와 같이 일정 구간별 표준 편차 또는 일정 구간별 분산이 되는 것이 계산 속도의 향상을 위하여 바람직하지만, 이에 한정되지 않고 본 발명의 σ^p(k)가 상술한 부분 표준 편차 또는 부분 분산은 물론 기타 다른 방법에 의하여 구한 표준 편차 또는 분산이 되어도 무방한 것은 자명한 사실이다.

In other words, in order to further increase the processing speed in the calculation, it is also possible to set the overlap length x to 0 by reducing the length of the predetermined section and increasing the moving length N until it is equal to the length of the predetermined section. At this time, the standard deviation for each interval is equal to the partial standard deviation.
Σ ^p (k) of the invention by the predetermined length by the standard deviation or a certain to be distributed each section preferably in order to improve the calculation speed, but this is not limited σ ^p (k) of the present invention as described above described above It is obvious that the standard deviation or variance obtained by the partial standard deviation or partial variance as well as by other methods may be used.

7 is a graph for explaining an end point detection method by pole sampling according to the present invention. Referring to Figure 7 and describes the endpoint detection method of the present invention. As mentioned earlier, the same equation holds for substituting variance instead of standard deviation, and the end point detection performance is the same.

First, σ ^p (k) is obtained.
To this end, first, the DC bias of the acceleration signal output from the accelerometer is removed and low pass filtering is used to remove noise. In this case, the DC bias refers to the amount of bias in which the reference value of the acceleration signal is biased upward or downward in the vertical direction with respect to the origin, and the acceleration signal before the start point or the acceleration signal after the end point. It can be seen at a glance that the value biased from the origin is DC bias. Referring to the upper first graph of FIG. 7 as an example, a result of removing the DC bias of the acceleration signal output from the accelerometer and performing low pass filtering is shown. When the acceleration signal value before or after the end point converges to 0, It can be seen that the DC bias has been removed.
Next, a piecewise standard deviation σ for the k th sample A (k) (shown in the upper graph of FIG. 7) of the time series data of the acceleration signal passing through the low pass filter. ^{Find p} (k). The standard deviation σ ^p (k) for each interval is a value obtained by obtaining a standard deviation for each sample on the basis of S samples of time series data, and is shown in the lower graph of FIG. 7 and may be expressed by Equation 8 below. Here, Γ _lowpass means data filtered by a low pass filter.

delete

σ ^p (k) = std (Γ _lowpass [A (k-S + 1), A (k-S + 2), ..., A (k)])

Second, find the local minimum σ ^p _LV (n) corresponding to the local minimum at σ ^p (k). σ ^p _LV (n) is indicated by the hollow circle in the bottom graph of FIG. 7.

Third, sigma ^p _max (n), which is the local maximum between two adjacent sigma ^p _LV (n), is found. sigma ^p _max (n) is represented by a solid circle in the lower graph of FIG. 7, and may be represented by Equation 9 below.

σ ^p _max (n) = max [σ ^p _LV (n-1), σ ^p _LV (n)]

Fourth, when sigma ^p _max (n) is greater than the threshold sigma ^p _th , k (n-1) is set as a start point (SP). This may be represented by the following equation (10).

if σ ^p _max (n)> σ ^p _th , start point = k (n-1)

Fifth, when σ ^p _max (n) is smaller than the threshold σ ^p _th , k (n) is set as an end point (EP). This can be expressed by the following equation (11).

if σ ^p _max (n) <σ ^p _th , end point = k (n)

Here, the start point and the end point can be found by detecting an envelope including a solid circle σ ^p _max (n) and then comparing the envelope coordinates with a threshold value.

Referring to FIG. 7, the relationship between the low-pass filtered acceleration signal and the standard deviation (or variance by a certain section) for each section, and the physical meaning of σ ^p _LV and σ ^p _max can be seen. In the lower graph of FIG. 7, the portion indicated by the hollow circle represents the local lowest point σ ^p _LV . In the upper graph of FIG. 7, it can be seen that this position corresponds to the extreme value of the acceleration signal. The point indicated by the solid circle in the lower graph of FIG. 7 is σ ^p _{m ax which} is the maximum value between two adjacent σ ^p _LVs .

Sixth, whenever a local lowest point σ ^p _LV (n) occurs during the search for the end point after the start point, the coordinates (k (n), Γ _lowpass [A (k (n))) in the graph of the acceleration signal shown at the top of FIG. ]) Is an extreme point that satisfies ΔA (n) · A (n-1) <0. ΔA (n) is Γ _lowpass [A (k (n))-A (k (n-1))]. If the coordinates (k (n), Γ _lowpass [A (k (n)))) of the graph of the acceleration signal are not extremes at this stage, then σ ^p _LV (n) is determined to be an incorrect local minimum and discarded.

Examples of discarding false poles in the graph of the acceleration signal are shown in FIGS. 8 to 10. FIG. 8 shows A (k) obtained by low pass filtering the acceleration signals output from the x-axis, y-axis, and z-axis accelerometers provided in the mouse device shown in FIG. For each interval, the standard deviation σ ^p (k) and the local minimum σ ^p _LV (n) are shown in order below. FIG. 9 shows a local minimum σ ^p _LV (n), a local maximum sigma ^p _max (n), and a threshold σ ^p _th for each of the standard deviations of each interval of FIG. 8. FIG. 10 is a graph showing red poles indicating poles that are determined to be wrong poles among signals corresponding to the local minimum point σ ^p _LV (n).

Next, the structure of the mouse device to which the EPS of the present invention is applied will be described. 11 shows the structure of such a mouse device. Referring to this, the mouse device includes an accelerometer unit 100 and a controller 300. In one embodiment, the accelerometer unit 100 is an accelerometer (110) provided for the three axes, such as the x-axis, y-axis, z-axis, A / to convert the signal output from the accelerometer 110 to a digital signal / D converter 120, a function classification button 105 used to switch the motion recognition function and the coordinate conversion function, and the interface unit 130 for transmitting the input and output signals between the control unit 300 and the accelerometer unit 100 do. The interface unit 130 transmits the digital signal output from the A / D converter 120 to the controller 300. The feedback unit 301 transmits the signal sent by the controller to the user as information such as visual, auditory, and tactile. The feedback unit 301 may be installed together with the control unit 300 as shown or may be installed inside the accelerometer unit 100 although not shown.

12 is a block diagram illustrating a motion recognition function that replaces a button input function during operation of the mouse device of the present invention. Referring to this, the accelerometer unit 100 exchanges an acceleration signal with the control unit 300 through the interface unit 130. The acceleration signal data obtained by the controller 300 is passed through the low pass filtering step 310, the end point detection step 320, the ZVC step 330, and the feature component extraction and motion recognition step 340. You can implement the conversion function.

According to the extreme point sampling of the present invention, the start point and the end point are found in the? ^P _max region (or? ^P (k) region) while finding the extreme position of the acceleration signal. The "extreme value found through Extreme Points Sampling" of the present invention is used as a feature for pattern recognition. Therefore, as shown in FIG. 12, the extreme value of the acceleration signal is used as it is in step 340 after the ZVC step 330 as shown in FIG. 12 without having to go through separate processes 350 and 360 such as the integration or the trajectory estimation. By feature extraction, motion recognition of the pen-type mouse device is possible.

Next, a method for implementing a button function by the motion recognition of the mouse device of the present invention will be described. In order to verify the end-point detection method and motion recognition performance of the present invention through experiments, 14 motions (see the figure below) were obtained 11 times (12 × 14 × 11) from 6 persons in 20 to 40 years. = 1848).

Of these, 168 data were used to create a template for each of the 14 operations, and the remaining data (1680) were used to confirm / compare the recognition rate using Dynamic Time Warp (DTW).

In case of using HMM for speech recognition for pattern recognition of acceleration signal, state should be increased for modeling non-stationary signal, and thus it is similar to DTW. In addition, DTW is sensitive to the performance of endpoint detection, so it is suitable for performance comparison of endpoint detection algorithm.

The recognition results are summarized in Table 1 below. The case 1 detecting the end point and recognizing the motion by the comparative example shown in FIG. 6 was compared with the case 2 detecting the end point and recognizing the motion by the “pole sampling” of the present invention shown in FIG. 7. Recognition rates for each case are summarized in Table 1.

Referring to Table 1, the recognition rate of numbers with curves is relatively low due to the accelerometers that have similar behaviors of 0 and 6 and only measure linear acceleration. As a result, it can be seen that the endpoint detection method of case2 of the present invention exhibits a higher recognition rate than that of case1.

Next, referring to FIGS. 13 to 22, a method of implementing a coordinate transformation function using the mouse device of the present invention will be described. The aforementioned pole sampling (EPS) has been simplified to implement the coordinate transformation without the accelerometer trajectory estimation.

Since linear motion is a vector motion, it can be expressed by the direction of force and the magnitude of force. The direction of the force and the magnitude of the force with respect to the linear motion can be expressed as the sum of the scaling factors of the unit vectors of the Cartesian coordinate system (for example, the x-axis and the z-axis of FIG. 11). Detecting the axis where the force is measured and knowing the magnitude and direction of the force generated in the axis can determine the direction and scaling factor of the linear motion. The time when the action of the force already started can be determined by the moment the signal disappears.

13 to 22 show graphs of the acceleration signals output from the x-axis accelerometer and the z-axis accelerometer shown in FIG. 11 according to the trajectory of the mouse device, and a force vector. The graph shown in blue solid lines represents the signal of the x-axis accelerometer, and the graph represented in green solid lines represents the signal of the z-axis accelerometer. In the graph of the acceleration signal, the hollow circle represents the pole defined in general mathematics. In contrast, the solid circle represents the extreme value found through the EPS of the present invention and is denoted by reference numerals x1 to x4 and z1 to z4.

Through the simplified EPS method as shown, it can be seen that i) linear motion only, ii) on which axis the motion occurred. For example, if an extreme value extracted as EPS is generated in both the x-axis and the z-axis signal, the linear motion in the diagonal direction in which the movement occurs in both the x-axis and the z-axis can be known. Therefore, the coordinate transformation function can be implemented for the 3D mouse movement.

Also, iii) it can be seen in which direction the axis moves. That is, the direction of the movement can be known depending on whether the magnitude difference between the first point and the second point of the extreme value obtained by the EPS is positive or negative.

And iv) the magnitude of the force. In other words, the absolute value of the magnitude difference between the first and second points of the extreme value obtained by EPS is the magnitude of the force.

Also, v) the time at which the force acts can be seen as the end point extracted by EPS. If it is a diagonal motion, it is expressed as the vector sum of the x-axis and the z-axis. Therefore, the coordinate recognition result is calculated by simultaneously performing the above-described method on both axes.

Fig. 13 shows acceleration signals of the x-axis accelerometer and the z-axis accelerometer when the mouse device is linearly moved to the right. Referring to this, only four poles x1 to x4 for the x-axis signal are extracted as EPS. 14 shows the trajectory and the force vector at this time. The rectangle drawn above the force vector represents the magnitude of the force acting in the positive direction.

Fig. 15 shows acceleration signals of the x-axis accelerometer and the z-axis accelerometer when the linear movement is to the left. Referring to this, only four poles x1 to x4 for the x-axis signal are extracted as EPS. 16 shows the trajectory and the force vector at this time. The square drawn below the force vector represents the magnitude of the force acting in the negative direction.

Fig. 17 shows acceleration signals of the x-axis accelerometer and the z-axis accelerometer when linearly moved upward. Referring to this, only four poles z1 to z4 for the z-axis signal are extracted as EPS. 18 shows the trajectory and the force vector at this time. The rectangle drawn above the force vector represents the magnitude of the force acting in the positive direction.

Fig. 19 shows acceleration signals of the x-axis accelerometer and the z-axis accelerometer when linearly moved downward. Referring to this, only four poles z1 to z4 for the z-axis signal are extracted as EPS. 20 shows the trajectory and the force vector at this time. The square drawn below the force vector represents the magnitude of the force acting in the negative direction.

21 illustrates acceleration signals of the x-axis accelerometer and the z-axis accelerometer when the linear movement is performed in the diagonal direction from the upper right to the lower left. Referring to this, all four poles x1 to x4 for the x-axis signal and four poles z1 to z4 for the z-axis signal are extracted as EPS. 22 shows the trajectory and the force vector at this time.

The coordinate recognition method by the simplified EPS method is as follows. The information necessary for the coordinate transformation is the direction (D) for each axis, the magnitude of the force (F) for the direction, and the time (T) for which the operation was continued. It is assumed that the result of EPS processing the acceleration signal generated when the linear movement in the diagonal direction from the upper right to the lower left is as shown in FIG. 21, and poles of each axis are denoted by reference numerals x1 to x4 and z1 to z4. In this case, a method of obtaining a direction (D _x ), a force (F _x ), and a time (T _x ) of the movement with respect to the x-axis is as follows.

The result of recognizing the change of coordinate information by constructing a vector by the direction of the motion and the magnitude | size of a force calculated about each axis, and summating these vectors is illustrated in FIG. The result of the coordinate recognition is not limited to the vector sum of the x-axis and the z-axis as shown, and although not shown, the same holds true for one axis or three axes.

Once again, for example, the first value of the feature component (extreme value of the acceleration signal extracted with ESP) for the first axis corresponding to the x-axis is A _x (x (1)), and the second value of the feature component is A. _{x (x} (2)) when _{said, a x (x (2)} ) a x (x (1) in the a _{x (x} (1)) code and the a _{x (x} (2)) of the value obtained by subtracting from the The change of the coordinate information about the first axis is recognized based on the vector obtained by the absolute value of the value obtained by subtracting). In this case, the vectors may be obtained for a plurality of axes, and then the change of the coordinate information for the plurality of axes may be recognized based on the vector sum of the vectors.

And, wherein said characteristic component with respect to the first axis (e.g., x axis) (e.g. A _x (x (1)), A _x (x (2)), A _x (x (3)), A Based on the length of the interval where _x (x (4)) exists (for example, x (1) to x (4)), the time for which the operation for the first axis is continued (for example, T _x = X (4) -x (1)). If this is extended, the lengths of the sections in which the feature components exist for the plurality of axes can be obtained, and then, based on this, the time for which the operation for the plurality of axes is continued can be recognized.

As described above, the present invention aims at integrating a motion recognition function for button input of a general TV remote controller and a coordinate conversion function of a three-dimensional mouse, while minimizing complexity and maximizing intuition, particularly in silver generation. Provided is a suitable mouse device.

In order to maximize intuition, we focused on motion recognition using accelerometers and proposed algorithms for replacing button input functions and transforming coordinate information. We tried pattern recognition with the acceleration signal itself, not the trajectory estimation by integrating the acceleration, and proposed a method of performing end point detection and feature extraction simultaneously. In addition, we proposed a method to obtain the change of coordinate information without estimating trajectory by simplifying EPS.

Although embodiments according to the present invention have been described above, these are merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent embodiments of the present invention are possible therefrom. Therefore, the true technical protection scope of the present invention will be defined by the following claims.

1 is a side view showing the posture of the pen and the horizontal plane.

FIG. 2 is a graph showing an actual trajectory, an acceleration signal, and an estimated trajectory of the pen of FIG. 1;

3 to 5 illustrate ZVC in graphs and equations.

Figure 6 is a comparative example of the endpoint detection method presented for comparison with the endpoint detection method of the present invention.

7 is a graph for explaining an end point detection method by pole sampling according to the present invention.

8 to 10 are graphs for explaining the endpoint detection method of the present invention.

11 and 12 show the structure and operation of the mouse device of the present invention.

13 to 22 are diagrams for describing a method of implementing a coordinate transformation function using a mouse device of the present invention.

Claims (21)

Removing the DC bias of the acceleration signal output from the accelerometer; Low pass filtering to remove noise of the acceleration signal; Obtaining σ ^p (k) for a k th sample A (k) of the acceleration signal subjected to the low pass filtering step, where σ ^p (k) is a standard deviation or variance; Obtaining a local minimum σ ^p _LV (n) from the σ ^p (k); Obtaining a local maximum σ ^p _max (n) between two adjacent σ ^p _LV (n); Detecting a starting point and an ending point by comparing the sigma ^p _max (n) with a threshold sigma ^p _th ; Extracting a pole of A (k) from the sigma ^p (k) using a local minimum sigma ^p _LV (n); Endpoint detection method comprising a. The method of claim 1, When σ ^p _max (n) is greater than the threshold value, k (n-1) is extracted as the starting point, and when σ ^p _max (n) is less than the threshold value, k (n) is extracted as the end point. End point detection method characterized in that. The method of claim 1, And the local lowest point σ ^p _LV corresponds to an extreme value of the acceleration signal A (k). The method of claim 1, Finding the extreme value position of the acceleration signal A (k) and simultaneously detecting the starting point and the ending point in the sigma ^p _max (n) region. The method of claim 1, In calculating the standard deviation, the overlap length x is a value obtained by subtracting the moving length N from the length of a predetermined section, and the moving length N is an integer greater than or equal to 1 and less than or equal to the length of the predetermined section. . The method of claim 1, And extracting the start point and the end point by detecting an envelope of the signal σ ^p (k) and comparing it with the threshold value. delete It is possible to selectively perform the motion recognition function for button input and the coordinate conversion function for pointing according to whether the function classification button is on or off. When performing the coordinate conversion function by detecting the movement in the off state of the function classification button, the coordinates are converted by obtaining a starting point and an end point of an operation using an extreme point detection method (EPS), When performing the motion recognition function by detecting the movement in the on state of the function classification button, the start point and the end point of the operation are obtained by using an endpoint detection method (EPS: Extreme Points Sampling), and then the feature components are extracted. Mouse device characterized in that the movement is recognized by pattern recognition through a Zero Velocity Compensation (ZVC) algorithm. An accelerometer unit having at least one accelerometer and a button for distinguishing a function, and converting an acceleration signal output from the accelerometer into a digital signal; Obtaining A (k), which is low pass filtered data of the digital signal output from the accelerometer unit, and detecting a start point and an end point in a region of standard deviation or variance with respect to A (k) to recognize a change in coordinate information Control unit; Mouse device comprising a. 10. The method of claim 9, The control unit recognizes an operation through pattern recognition by extracting a feature component from the acceleration signal, wherein the feature component is an extreme value of A (k). 10. The method of claim 9, Feedback unit for transmitting the signal sent by the controller to the user information such as visual, auditory, tactile; Mouse device further comprises. Removing the DC bias of the acceleration signal output from the accelerometer; Low pass filtering to remove noise of the acceleration signal; Obtaining σ ^p (k) for a k th sample A (k) of the acceleration signal subjected to the low pass filtering step, where σ ^p (k) is a standard deviation or variance; Obtaining a local minimum σ ^p _LV (n) from the σ ^p (k); Obtaining a local maximum σ ^p _max (n) between two adjacent σ ^p _LV (n); Detecting a starting point and an ending point by comparing the sigma ^p _max (n) with a threshold sigma ^p _th ; Recognizing a change in coordinate information from the acceleration signal; A zero velocity compensation (ZVC) step of eliminating offset errors present in the start point and the end point; Recognizing motion through pattern recognition by extracting feature features of the acceleration signal; Method of operation of the mouse device comprising a. The method of claim 12, Whenever the local lowest point σ ^p _LV occurs while finding the starting point and the end point, the coordinate (k (n), Γ _lowpass [A (k (n))]) is ΔA (n) · ΔA (n-1) < Checking for extreme points satisfying zero; More, ΔA (n) is Γ _lowpass [A (k (n))-A (k (n-1))], And if said coordinate is not said pole, said local lowest point is discarded. Obtain σ ^p (k) for the acceleration signal output from the accelerometer, where σ ^p (k) is the standard deviation or variance, By comparing the local maximum values of the σ ^p (k) region with a threshold value, the starting point and the end point are found to recognize a change in coordinate information. And recognizing motion by removing offset errors present in the start point and the end point and extracting a feature component corresponding to an extreme value of the acceleration signal. The method of claim 14, When the first value of the feature component is A _x (x (1)) and the second value of the feature component is A _x (x (2)) with respect to the first axis, A _x (x (2)) minus A _x (x (1)) minus sign and A _x (x (2)) minus A _x (x (1)) minus value And recognizing a change in the coordinate information on the first axis based on the vector obtained by the method. The method of claim 15, Obtaining each of the vectors for a plurality of axes, and then recognizing a change in the coordinate information for the plurality of axes based on the vector sum of the vectors. The method of claim 15, And recognizing a time duration of the operation on the first axis based on a length of a section in which the feature component exists with respect to the first axis. The method of claim 17, Obtaining a length of a section in which the feature component exists for a plurality of axes, and then recognizing a time duration of the operation for the plurality of axes. Obtain σ ^p (k) for the acceleration signal output from the accelerometer, where σ ^p (k) is the standard deviation or variance, Finding a starting point and an end point by comparing local maximums of the σ ^p (k) region with a threshold value, recognizing a change in coordinate information, and extracting a feature component of the acceleration signal existing between the starting point and the end point A method of operating a mouse device, characterized in that the recognition of the movement through the recognition. An accelerometer unit having at least one accelerometer and a button for distinguishing a function, and converting an acceleration signal output from the accelerometer into a digital signal; Obtaining A (k), which is data obtained by removing the bias of the digital signal output from the accelerometer unit and performing low pass filtering, detecting a starting point and an ending point in a region of standard deviation or variance with respect to the A (k), A controller configured to recognize an operation through pattern recognition by removing an offset error existing within a section defined by a start point and the end point, and extracting a feature component from the acceleration signal; Mouse device comprising a. delete

Images