KR101234094B1 - Apparatus for motion based pointing providing haptic feedback and control method thereof - Google Patents

Abstract

The present invention relates to a motion-based pointing device for providing tactile feedback and a method of controlling the same, comprising: a motion detecting unit detecting a user's movement, a pointing means including a tactile output unit for providing a tactile feedback to the user, and And a terminal means for manipulating a pointer displayed on a screen from the movement of the user sensed by a pointing means, and outputting corresponding tactile information to the pointing means according to the movement of the pointer and the screen change according to the manipulation of the pointer. The pointing means may be tactile output according to the tactile information output from the terminal means. According to the present invention, it is possible to manipulate the objects on the screen and the graphical user interface elements through the movement of the hands and to feel the tactile feeling, thereby giving the user interface a tactile feedback function for the operation of the pointing device, which was dependent only on visual information. By doing so, there is an effect of increasing the usability and the recognition rate of the user.

Description

Apparatus for motion based pointing providing haptic feedback and control method

The present invention relates to a motion-based pointing device that provides tactile feedback and a method of controlling the same. The present invention provides a tactile feedback that enables a user to manipulate objects on the screen and graphical user interface elements through the movement of a hand, and at the same time, feels haptic. The present invention relates to a motion-based pointing device and a control method thereof.

In the case of a commonly used pointing device, for example, a remote controller that can be operated remotely may be used.

The function of controlling the screen according to the rotation direction and the movement of the remote control using a gyro sensor or the like is implemented. In addition, a plurality of gyro sensors are provided to detect movement in various directions and control a screen according to shaking of the remote controller.

Recently, a technology has been developed to provide a tactile stimulus to the remote controller so that the user can feel the tactile feeling while controlling the screen.

However, at present, a technique for providing tactile feedback for allowing a user to recognize a fine adjustment when a screen is manipulated using a pointer has not been implemented. That is, when the user controls the screen using the pointer, it is not easy for the user to detect the direction or the position of the pointer from the tactile feedback.

SUMMARY OF THE INVENTION An object of the present invention for solving the above problems is a motion-based pointing device that provides tactile feedback that enables the user to manipulate objects on the screen, graphical user interface elements through the movement of the hand, and at the same time feels tactile. The control method is provided.

In addition, another object of the present invention, by controlling the pointer on the screen connected to the terminal means by using an angle change measured according to the movement of the user's hand or wrist can be utilized as a pointing device to replace the existing mouse, TV The present invention also provides a motion-based pointing device that provides tactile feedback that can be used as a remote controller for controlling a sound device, and a control method thereof.

Motion-based pointing device for providing tactile feedback according to the present invention for achieving the above object, the motion detection unit for detecting the user's movement, and the tactile output for providing tactile feedback to the user in the long axis direction or left and right directions A pointing means provided on the screen and a pointer displayed on a screen from the movement of the user sensed by the pointing means, and the corresponding tactile information according to the movement of the pointer and the screen change according to the manipulation of the pointer. And a terminal means for outputting to the means, wherein the tactile output unit is one or more to generate a collision or vibration in at least one of up, down, left, right, front and back according to the tactile information output from the terminal means. It characterized in that it comprises a linear vibrator.

The tactile output unit may collide the vibrating body of the linear vibrator in one direction or both directions, or generate vibration.

The tactile output unit may generate a short vibration or a vibration in which its intensity gradually increases or decreases when the vibration occurs.

The tactile output unit may collide the vibrating body of the linear vibrator in the opposite direction or generate a vibration every time the tactile information is output in response to a movement of the pointer or a screen change according to the manipulation of the pointer. .

The haptic output unit may generate a collision in a direction corresponding to the movement of the pointer.

The tactile output unit is characterized in that the two linear vibrators are disposed perpendicular to each other in the two axis direction, to generate a collision or vibration using the vibrating body of the linear vibrator.

The tactile output unit is characterized in that the three linear vibrators are disposed perpendicular to each other in the three axis direction, to generate a collision or vibration using the vibrating body of the linear vibrator.

The motion detection unit may include an angular velocity sensor measuring an angular velocity in a three axis direction from the movement of the pointing means, and an acceleration sensor measuring acceleration in the three axis direction from the movement of the pointing means.

The pointing means further comprises an input unit for receiving a control command from a user, and a communication unit for transmitting and receiving a signal with the terminal means.

The terminal means recognizes the coordinate position on the screen corresponding to the movement of the user sensed by the pointing means and outputs a control signal for controlling the operation of the pointer, and according to the movement of the pointer or the pointer manipulation. And a tactile information extraction unit configured to extract a tactile information corresponding to a movement or a screen change of the pointer recognized by the gesture recognition unit.

The tactile information extracting unit may extract tactile information in response to at least one change in the position, the moving distance, the moving speed, the moving angle, the shape of the pointer, and whether the screen is selected using the pointer.

The tactile information extracting unit extracts tactile information in response to at least one of a shape, a size, a position, and an angle of a window, a menu, an icon, an emoticon, a button, a figure, and an area selected by the pointer displayed on a screen. It features.

The tactile information extracting unit may extract tactile information according to whether the object indicated by the pointer is executable on the screen.

The terminal means further comprises a display unit having a screen, and a communication unit for transmitting and receiving a signal with the pointing means.

On the other hand, the control method of the motion-based pointing device for providing tactile feedback according to the present invention for achieving the above object, the step of detecting the movement of the pointing means according to the user's movement, the screen in response to the movement of the pointing means Outputting a control signal for controlling the operation of the pointer displayed on the screen; recognizing a movement of the pointer on the screen or a screen change according to the manipulation of the pointer; And correspondingly outputting the tactile information, and outputting the tactile information using the linear vibrator of the pointing means according to the output tactile information.

The recognizing may include recognizing a change in at least one of a position, a moving distance, a moving speed, a moving angle, a shape, and whether a screen is selected using the pointer.

The recognizing may include recognizing a change in at least one of a shape, a size, a position, and an angle of a window, a menu, an icon, an emoticon, a button, a figure, and an area selected by the pointer displayed on the screen.

The recognizing may include recognizing whether a target indicated by the pointer is executed on a screen.

The tactile output may include colliding the vibrating body of the linear vibrator in one or both directions, or generating a tactile output by generating vibration.

The tactile output may include generating a collision or vibration in at least one of up, down, left, right, front and back directions using a vibrator of the linear vibrator.

According to the present invention, there is an effect of using the tactile feedback to improve the performance and usability of the user interface of various devices.

In addition, the present invention has the effect of increasing the usability and the recognition rate of the user by providing a tactile feedback function to the user interface for the operation of the pointing device, which was dependent only on visual information.

In addition, the present invention reduces the ambiguity generated when using a pointing device based on motion recognition through tactile feedback, and has an effect of enabling accurate manipulation.

Hereinafter, specific embodiments of the present invention will be described with reference to the accompanying drawings.

1 is a view showing the operation of a motion-based pointing device for providing tactile feedback according to the present invention.

As shown in FIG. 1, a motion-based pointing device for providing tactile feedback according to the present invention includes a control device such as a computer, or a terminal means 200 such as a monitor, a TV, etc. And pointing means 100 for controlling the displayed pointer.

For example, when the user wants to control the screen operation of the computer monitor, the pointing means 100 is connected to the terminal means 200 such as a computer or a monitor to take a specific motion, and thus the operation screen is operated by operating the pointer of the monitor. And so on.

At this time, the pointing means 100 is provided with a sensor that can measure the movement of the user's hand, the user grasps the pointing means 100, each axial rotation (roll. Pitch, yaw) or shaking (shake) ) To feel the touch when the size, position, scroll, etc. of the window window 102 displayed on the screen or to move the menu, and to operate more comfortably and delicately than when there is no tactile feedback. Will be.

At this time, the shaking motion is, in general, the left and right movement (yaw) of the wrist corresponds to the horizontal movement of the pointer 104 of the screen 101, the vertical movement (pitch) is the vertical of the pointer 104 on the screen 101 Corresponds to directional movement. In FIG. 1, an example of shaking in the long axis direction is illustrated, but not only the long axis direction but also the shaking in the left, right, up, down, and diagonal directions is also applicable.

2 is a block diagram showing the configuration of the pointing means 100 of the motion-based pointing device for providing tactile feedback according to the present invention, Figure 3 is a perspective view showing the configuration of the pointing means 100.

2 and 3, the pointing means 100 according to the present invention includes an input unit 110, a motion detecting unit 120, a tactile output unit 130, a control unit 140, and a communication unit 150. do.

The input unit 110 is implemented in the form of a plurality of key buttons, and is used as various command input buttons when replacing functions such as left and right clicks and scrolling of a mouse or when used as a remote controller.

The motion detector 120 includes an inertial sensor for measuring the movement of the user's wrist and the hand. Here, the inertial sensor includes an angular velocity sensor for measuring the angular velocity in the three axis direction, and an acceleration sensor for measuring the acceleration in the three axis direction.

In this case, the angular velocity sensor may utilize only two axes when adjusting the position of the pointer displayed on the screen using only the up, down, left and right movements of the wrist, and the acceleration sensor may freely design one axis to three axes according to the purpose. have.

The tactile output unit 130 transmits tactile information to the user based on the linear vibrator. In this case, the tactile output unit 130 may be provided in plurality.

The tactile output unit 130 is disposed in a direction perpendicular to the first tactile output unit 131 and the first tactile output unit 131 that outputs tactile information regarding the movement in the long axis direction of the pointing means 100. And a second tactile output unit 133 for outputting tactile information about the movement in the male direction of the reference numeral 100.

The first tactile output unit 131 has a linear vibrator disposed in the longitudinal direction of the pointing means 100 to generate shock or vibration at both ends in the longitudinal direction of the pointing means 100.

Preferably, the linear vibrator of the first tactile output unit 131 is disposed as close as possible to the place where the index finger and the middle finger touch when the pointing means 100 is gripped by hand.

The second tactile output unit 133 generates a shock or vibration at both ends of the linear vibrator in the horizontal direction of the pointing means 100.

Of course, in the exemplary embodiment of the present invention, although the tactile output unit 130 is composed of the first tactile output unit 131 and the second tactile output unit 133, the present invention is not limited thereto. It may be implemented by only 130, or three or more tactile output unit 130 may be implemented.

The operation principle of the linear vibrator of the tactile output unit 130 will be described with reference to FIG. 5.

The communication unit 150 performs communication with the terminal means 200 to be controlled.

The controller 140 controls the operations of the input unit 110, the motion detector 120, the tactile output unit 130, and the communication unit 150.

In addition, when the motion detector 120 detects a user's movement by a user's hand, wrist, arm, or the like, the controller 140 may detect the motion information detected by the terminal means 200 connected through the communicator 150. Output

In addition, when the tactile information corresponding to the motion information is received from the terminal means 200 connected through the communication unit 150, the controller 140 transmits the received tactile information to the tactile output unit 130. As a result, the tactile output unit 130 generates impulse or vibration in accordance with the tactile information from the controller 140.

4 is a block diagram showing the configuration of the terminal means 200 of the motion-based pointing device for providing tactile feedback according to the present invention.

As shown in FIG. 3, the terminal means 200 according to the present invention uses the communication unit 210, the terminal control unit 220, the display unit 230, the motion recognition unit 240, and the tactile information extraction unit 250. Include.

The communication unit 210 is connected to the pointing means 100 and receives the movement information of the user sensed by the pointing means 100 and a control command input from the user. In addition, the communication unit 210 provides tactile information from the tactile information extracting unit 250 to the connected pointing means 100 in response to the received motion information.

The display unit 230 displays an operation screen, a pointer, or the like according to a control signal from the pointing means 100 connected through the communication unit 210.

The motion recognition unit 240 detects the control command input from the user from the signal received through the communication unit 210. In addition, the motion recognition unit 240 recognizes the user's motion based on the user's motion information received through the communication unit 210.

In this case, the terminal controller 220 controls the movement of the pointer displayed on the display unit 230 in response to the operation recognized by the gesture recognition unit 240.

In addition, the terminal controller 220 transmits the motion information recognized by the motion recognizer 240 to the tactile information extractor 250. The tactile information extractor 250 extracts tactile information corresponding to the motion information transmitted from the terminal controller 220. In this case, the tactile information extractor 250 outputs tactile information through the tactile output engine.

The tactile information output by the tactile information extraction unit 250 is transmitted to the communication unit 210 by the terminal controller 220 and transmitted to the pointing means 100.

5 is an exemplary view referred to to explain the operating principle of the linear vibrator.

First, (a) of FIG. 5 illustrates a linear vibrator, in which a mass moves linearly by using a magnetic field.

At this time, the linear vibrator disposed in the vertical direction generates vibration to the upper or lower side of the pointing means 100 according to the movement of the mass body, and the linear vibrator disposed in the left and right direction moves to the left side of the pointing means 100 according to the movement of the mass body. Or to generate vibration to the right.

The linear vibrator applied to the present invention not only responds faster than the eccentric rotary vibrator used when generating a tactile stimulus, but also has less influence of inertia, so that the operation stops quickly after the electric signal is stopped. There is an advantage in delivering short and tactile information.

FIG. 5B shows a digital signal, and FIG. 5C shows a vibration signal of the linear vibrator generated corresponding to the digital signal of (b).

Referring to (b) and (c) of FIG. 5, when the linear vibrator is generally controlled using digital signals HIGH and LOW, as in the case of A, when the signal of HIGH is supplied, the mass moves to one side. When the LOW signal is supplied, the mass moves to the other side.

In this case, the linear vibrator generates an impact when a mass body hits one side while a momentary change of the signal occurs.

In addition, the linear vibrator is less affected by the inertia that remains after the electrical signal is interrupted. That is, as in the case of B, even if the period of the signal is short, the interference between the current signal and the previous signal is relatively small.

In addition, the linear vibrator causes the mass to vibrate without colliding with the wall surface if the period of signal generation is made very short, such as C.

6 to 19 are exemplary views referred to for describing the operation of the motion-based pointing means 100 for providing tactile feedback according to the present invention.

6 illustrates an example of an operation of providing tactile feedback using a linear vibrator of the first tactile output unit 131 while moving a pointer displayed on a screen of the terminal means 200 using the pointing means 100 according to the present invention. It is shown.

As shown in FIG. 6A, when the pointer displayed on the screen of the terminal means 200 shows continuous movement from the starting point, the pointing means 100 for controlling the movement of the pointer, that is, the movement of the pointer, is used. In order to move, the tactile information is output through the first tactile output unit 131 to transmit the degree of movement according to the movement of the hand or the wrist to the user.

The tactile information extraction unit 250 of the terminal means 200 outputs the tactile information whenever the position of the pointer occurs more than a specific change amount.

Therefore, the user can determine the position of the pointer based on tactile information as well as visual information, so that the position of the pointer can be precisely and comfortably operated.

In addition, since the tactile information extraction unit 250 of the terminal means 200 adjusts the frequency of tactile output generation in proportion to the moving speed of the pointer, the user can intuitively experience the moving speed of the pointer. .

The tactile information extractor 250 of the terminal means 200 may repeatedly output a short HIGH / LOW signal at the time of generating the tactile output as shown in FIG. At this time, the linear vibrator of the pointing means 100 causes the two-way reciprocating collision instantaneously to the vibrating body according to the tactile information from the terminal means 200.

In this case, there is an advantage that the position of the mass is restored to its original position even after one operation.

On the other hand, the tactile information extraction unit 250 of the terminal means 200 outputs a signal for generating a LOW or HIGH signal only once, which is opposite to the previous state, at the time of generating the tactile output as shown in FIG. can do. In this case, there is an advantage that can reduce the energy consumption while causing a faster reaction.

That is, since the terminal means 200 generates an event every time a pointer movement occurs, the pointer means 200 moves the object on the screen, draws a picture, or writes the text using the pointing means 100. Whenever a motion event of P, occurs, a two-sided collision of the linear vibrator may occur.

In addition, as well as the movement of the pointer, when the wrist is rotated by the unit angle or the position is changed by the unit position in the pointing means 100 itself, it is possible to expect the effect of improving the operability by generating a tactile output.

7 illustrates an example of an operation of providing tactile feedback using a linear vibrator of the second tactile output unit 133 while moving a pointer displayed on a screen of the terminal means 200 using a pointing means according to the present invention. .

FIG. 7A is a graph showing in which direction the pointer moves in the horizontal direction from the starting point and in what direction.

As such, when the pointer shows continuous movement in the vertical direction, the tactile information extractor 250 of the terminal means 200 outputs tactile information corresponding thereto. At this time, the second tactile output unit 133 reacts to the tactile information from the terminal means 200 to move the vibrating body of the linear vibrator to generate a collision.

The linear vibrator of the second tactile output unit 133 is arranged to correspond to the left and right rotation of the user's wrist or the left and right movement of the hand, and the pointer displayed on the screen of the terminal means 200 interlocked with the pointing means 100. Each time a position occurs above a certain change amount, dispatch occurs, as explained in FIG.

In other words, the linear vibrator of the second tactile output unit 133 may generate a momentary bidirectional reciprocating collision by repeating a short HIGH / LOW signal at the time of generating the tactile output, as shown in FIG. As shown in (c), when the tactile output is generated, a collision occurs according to a signal generated only once with a LOW or HIGH signal that is opposite to the previous state.

On the other hand, the linear vibrator of the second tactile output unit 133 may generate a collision in response to the tactile generation signal as shown in FIG.

That is, when the pointer is moving to the right, the haptic information extracting unit 250 of the terminal means 200 applies a momentary HIGH to the second tactile output unit 133 and then outputs a signal that gradually decreases, so that the mass moves to the right. Causes a collision, but when it returns to the left side, it returns slowly to avoid the impact. On the other hand, when the pointer moves to the left side, the tactile information extractor 250 of the terminal means 200 applies a HIGH signal that gradually increases to the second tactile output unit 133, and then instantaneously supplies a signal that changes to LOW. The mass moves to the right, but there is no impact, and the moment is impulsive to the left.

In this case, if the control to move gradually so that an impact does not occur is possible in a short time (typically within 20 msec), the user does not feel the time difference but only the immediate left or right impact. Therefore, the method of FIG. 7D has an advantage of intuitively informing the user in which direction the user is moving with tactile information.

Although not shown in the present invention, such a method may be applied to vertical movement by vertically arranging the linear vibrator of the tactile output unit 130 vertically.

8 to 15 illustrate an embodiment of providing tactile feedback according to a movement of a pointer in response to an operation screen displayed on a screen of the terminal means 200.

First, FIG. 8 illustrates an example of providing tactile feedback in a process of moving a position of a screen or an icon on a screen.

FIG. 8A illustrates an operation displayed on a screen, and FIG. 8B illustrates a tactile output signal according to the operation of FIG.

As shown in FIG. 8A, an icon representing a file, a shortcut, a folder, or the like is displayed on the screen connected to the terminal means 200, and an open window is displayed to selectively adjust its position through manipulation of a pointer. Can be.

When the button for selecting the window displayed on the screen is manipulated by the pointing means 100, the tactile information extracting unit 250 of the terminal means 200 displays the pointing means (tactically) to indicate tactilely that the window is selected as shown in (A). A signal for causing the linear vibrator of the tactile output unit 130 of 100 to collide once to the upper side is output. Thus, the tactile output unit 130 generates a collision of the linear vibrator according to the input signal.

In addition, when the user's hand or wrist holding the pointing means 100 moves to move the selected window, the motion recognition unit 240 of the terminal means 200 senses the movement of the pointing means 100 and (B). Move the pointer on the screen together. Accordingly, the tactile information extractor 250 outputs tactile information proportional to the moving speed, and the tactile output unit 130 of the pointing means 100 provides bidirectional tactile feedback using a linear vibrator according to the tactile information.

In addition, when the window is moved to a desired position by using the pointing means 100 as shown in (C), and the button is released to select the pointer, the terminal means 200 reproduces the feeling that the selected window is separated. It outputs tactile information that causes the built-in linear vibrator to collide downward. Therefore, the linear vibrator of the tactile output unit 130 generates a lower collision according to the output tactile information.

In this case, the tactile output unit 130 may selectively operate among the first tactile output unit 131 and the second tactile output unit 133 according to the moving position and the direction of the pointer.

As such, using the tactile feedback of the pointing means 100 according to the present invention, it is possible to intuitively inform the user whether an object is selected or whether the object is moving.

9 shows an example of providing tactile feedback while resizing a window. FIG. 9A illustrates an operation displayed on the screen, and FIG. 9B illustrates a tactile output signal according to the operation of FIG.

As shown in (A), when a button of the pointing means 100 is manipulated to select a window corner vertex or an outer boundary position of the window, the tactile information extractor 250 causes the linear vibrator to collide upward. Outputs tactile information. At this time, the linear vibrator strikes the vibrating body upward to reproduce the feeling that the momentary window is selected and attached to the pointing device 100.

In addition, while moving the pointing means 100 to change the size of the selected window as shown in (B), the tactile stimulus is output whenever the unit size increases or decreases.

In addition, in the case of manipulating the button for releasing the window selection after changing the window size as desired as shown in (C), the feeling of the window falling off is instantaneously reproduced by colliding the mass of linear vibration downward.

If necessary, the mass of the linear vibrator may be gradually moved upward after the operation is completed, so that the user does not feel any tactile collision.

10 illustrates an example of providing tactile feedback while manipulating a scroll bar of a window window. FIG. 10A illustrates an operation displayed on a screen, and FIG. 10B illustrates a tactile output signal according to the operation of FIG.

Referring to FIG. 10, when the pointer selects the scroll bar using the pointing means 100 as shown in (A), the instantaneous pointer is generated on the stroke bar by generating an upward collision using the linear vibrator of the tactile output unit 130. Reproduce the feeling of sticking.

Further, while moving the selected scroll bar by moving the pointing means 100 as shown in (B), by providing a bidirectional tactile feedback using a linear vibrator of the tactile output unit 130 per unit movement of the pointer, the user feels a tactile stimulus. To do that.

On the other hand, when the scroll bar reaches the limit of upward or downward movement, as shown in (C), the linear vibrator is controlled by a very short signal to generate a short and strong vibration momentarily, indicating that the scroll bar has reached the top or bottom position. Provide feedback.

After changing the position of the scroll bar as desired, the bottom collision is generated by using the linear vibrator of the tactile output unit 130 as shown in (D) to reproduce the feeling of the scroll bar falling off the pointer.

In this case, since the scroll bar is operated vertically, the linear vibrator may be disposed in the vertical direction to provide tactile feedback indicating that the scroll bar is moving in a specific direction as described with reference to FIG.

11 illustrates an example of providing tactile feedback according to an operation of maximizing or minimizing a window size. FIG. 11A illustrates an operation displayed on a screen, and FIGS. 11B and 11C illustrate a tactile output signal according to the operation of FIG.

Referring to FIG. 11, when a specific icon is selected and executed using the pointing means 100 as shown in (A), the vibrating body of the linear vibrator is collided upward in accordance with the user's motion to reproduce the feeling of clicking the icon. .

Subsequently, when the window is opened as shown in (B), the linear vibrator of the tactile output unit 130 is collided downward and then collided upward to reproduce the feeling of the window being floated upward.

In addition, when a button for minimizing or maximizing a window is manipulated using the pointing means 100, such as (C) or (D), a linear vibrator of the tactile output unit 130 is used to reproduce the feeling of pressing the button. A collision occurs momentarily, and then a collision is generated once again using a linear vibrator to convey the feeling that the window is minimized or maximized.

In addition, as shown in (E), when the button for closing the window using the pointing means 100 is manipulated, a feeling of closing the window is reproduced by generating a bottom collision using the linear vibrator of the tactile output unit 130.

At this time, as shown in (c) of FIG. 11, it is also possible to transmit a tactile feeling to the user by generating a vibration instantaneously for each of (B), (C), (D), and (E).

Of course, the embodiment of the tactile feedback generated in each case of (A), (B), (C), (D), and (E) is not limited thereto, but may be implemented in various forms.

12 illustrates an example of providing tactile feedback during menu navigation and popup menu manipulation. FIG. 12A illustrates an operation displayed on the screen, and FIGS. 12B, 12C and 12D show tactile output signals according to the operation of FIG.

In FIG. 12, when the pointer moves through the menu or the menu list by using the pointing means 100, tactile feedback is used to accurately know whether the menu is changed or in which menu. The embodiment of FIG. 12 may be used for manipulating a pop-up menu on a graphical user interface, moving pages using a pop-up menu during a presentation, or moving a channel through a menu when watching media such as a TV.

In the case of FIG. 12B, when the first start menu is selected, an upper collision is generated by the linear vibrator of the tactile output unit 130 as shown in A, and when the new menu list popup is executed, the tactile output unit 130 as shown in B is illustrated. To generate a reciprocating collision. In addition, when the pointer moves to 'Menu1' or 'Menu2' in a new menu list popup, a reciprocating collision is generated using a linear vibrator of the tactile output unit 130 as shown in C and D. FIG.

Thereafter, when the menu pops up, a harmful pop-up menu is executed, and when the pointer moves to a new menu 'Sub-Menu2', a reciprocating collision is generated using a linear vibrator of the tactile output unit 130 as shown in E and F, respectively. Let's do it.

12 (b), if the reciprocating collision is generated in both directions at every tactile output, in the case of FIG. 12 (c), when an event such as A, B, C, D, E, F occurs, the tactile output unit 130 Using a linear vibrator of) alternately generates an upward collision and a downward collision.

On the other hand, in the case of (d) of FIG. 12, when an event such as A, B, C, D, E, or F occurs, a predetermined vibration is applied using a linear vibrator of the tactile output unit 130 every time the event occurs. By making it easy for the user to recognize the occurrence of the event.

13 illustrates an example of providing tactile feedback as the pointer passes through menus and icons. FIG. 13A illustrates an operation displayed on a screen, and FIGS. 13B, 13C, and 13D illustrate a tactile output signal according to the operation of FIG.

As shown in FIG. 13, when the pointer moves according to the movement of the hand or wrist holding the pointing means 100, tactile feedback is provided when the pointer crosses a boundary of a menu or an icon. In this case, the user can intuitively know where his pointer is located.

In FIG. 13A, the pointer sequentially moves on a 'File' menu, an 'Edit' menu, a 'save' icon, and a 'preview' icon. At this time, when the pointer passes the boundary of the 'File' menu, 'Edit' menu, 'Save' icon, and 'Preview' icon, the tactile information output unit generates events such as A, B, C, and D.

In the case of FIG. 13B, when events such as A, B, C, and D occur, reciprocating collisions are generated in both directions using a linear vibrator of the tactile output unit 130.

Meanwhile, in the case of FIG. 13C, when an event such as A, B, C, or D occurs, the upper and lower collisions are alternately generated by using the linear vibrator of the tactile output unit 130.

Meanwhile, in the case of FIG. 13D, when an event such as A, B, C, or D occurs, the user generates a predetermined vibration by using the linear vibrator of the tactile output unit 130 whenever the event occurs. Make it easy to recognize the event.

The embodiment of FIG. 13 illustrates an example of providing tactile feedback only when the pointer enters a boundary of a menu or icon, but may provide tactile padback both when the pointer enters and exits a boundary of a menu or icon. have.

The embodiment of FIG. 13 may be used as a method of providing tactile feedback at the moment when the mouse is positioned over the hyperlink area when searching a website or the like.

14 shows an example of providing tactile feedback while selecting several letters. Here, FIG. 14A illustrates an operation displayed on a screen, and FIG. 14B illustrates a tactile output signal according to the operation of FIG.

In the embodiment illustrated in FIG. 14, when the user wants to select several letters using a pointer operation, the number of letters selected is increased by one to solve the problem of difficulty in accurate selection because the location of the pointer is not precisely manipulated. It generates a tactile output each time, enabling accurate character selection.

In other words, when a sentence such as (a) of FIG. 14 is displayed on the screen, and the user wants to select 'act' using the pointer, after the manipulation of the selection button, the user moves the pointer to select the 'a' character and then the tactile output unit ( A collision in one direction is generated by using the linear vibrator of 130), and when 'c' is selected, a collision in the other direction is generated by using the linear vibrator of the tactile output unit 130. Similarly, when 't' is selected, a collision in the opposite direction is generated again by using the linear vibrator of the tactile output unit 130.

As described above, when a plurality of letters are selected using the pointer, each time the letters are additionally selected, the pointer device generates a collision in the opposite direction by using the linear vibrator of the tactile output unit 130.

Of course, as in the above-described embodiments, the pointing means 100 generates a reciprocating collision in both directions using a linear vibrator of the tactile output unit 130 each time a character is selected, or generates a predetermined vibration. You can also make it easy to recognize an event.

Therefore, the user can intuitively feel the sense that the letters are selected by the Hangul characters.

Similarly, when a predetermined letter area is selected as shown in FIG. 15A or underlined as shown in FIG. 15B, the unit area selected according to the movement of the pointer is changed or underlined. Whenever is changed, as in the above-described embodiments, the pointing means 100 generates a collision in both directions or in one direction by using the linear vibrator of the tactile output unit 130 whenever a letter is selected, By generating vibrations, the user can easily recognize the occurrence of an event.

14 and 15 may provide tactile feedback whenever the number of letters selected by the pointer decreases, and the same technique may be applied to a text viewer or a web page viewer.

16 shows an example of providing tactile feedback while manipulating the pose of the 3D object displayed on the screen by using the pointing means 100 according to the present invention.

FIG. 16A illustrates an operation of manipulating a three-dimensional object displayed on a screen using the pointing means 100 according to the present invention, and FIG. 16B illustrates a three-dimensional operation corresponding to the movement of the pointing means 100. It shows a graph according to the movement of the object and the corresponding tactile output signal.

In particular, FIG. 16 (b) is a graph showing a change in the orientation of the three-dimensional object displayed on the screen according to the movement of the pointing means 100. The motion recognition unit 240 of the terminal means 200 is a pointing means. The three-axis angular change of 100 is matched to the angular change of the three-dimensional object displayed on the screen. At this time, the tactile information extracting unit 250 monitors the angular change of the three-dimensional object, and outputs tactile information as shown in FIG. 16 (c) whenever the angle changes by more than the unit angle when the angular change occurs from the starting point. Thus, the pointing means 100 generates a tactile output.

In this case, the pointing means 100 may be arranged back, front, bottom, left, and right by using the three tactile output units 130, and then may generate a tactile output signal for each unit angle change in each direction. Of course, if a change occurs at any angle using only one tactile output unit 130, a simple method of tactile output may be applied.

On the other hand, Figure 17 is an exemplary view referred to in explaining the operation method of the pointing means 100 according to the present invention.

As shown in FIG. 17, the pointing means 100 according to the present invention not only can measure the rolling of the long axis, but also the long axis direction, the up, down, left, right and diagonal directions. You can measure the shaking change with. In the rotational movement around the long axis, the clockwise rotation may correspond to a function of turning a page to the next page in a document viewer or the like, and may correspond to switching to the next channel in the media.

The movement of the pointing means 100 is recognized by the terminal means 200 and according to the tactile information extracted from the tactile information extracting unit 250, collision is generated using the linear vibrator of the first tactile output unit 131. Generate.

In addition, in the case of using the second tactile output unit 133, as described in the embodiment of FIG. 7, only the collision occurs in one direction (for example, the right direction), thereby switching to the next page or the next channel. Make it more intuitive.

On the other hand, the rotation in the counterclockwise direction may correspond to the function of turning the page to the previous page in the document viewer or the like, and may correspond to switching to the previous channel in the media. In this case, contrary to the above-described embodiment, a collision occurs on the opposite surface by using the linear vibrator of the first tactile output unit 131 or a left surface collision of the second tactile output unit 133 is generated. In this case, the clockwise rotation and the counterclockwise rotation within 90 degrees may be measured by an acceleration sensor that detects a change in gravity direction, not an angular velocity sensor.

In addition, it can be applied to the operation of increasing or decreasing the loudness, in which case the vibration is generated using the linear vibrator, but the louder the louder the sound, the weaker the weaker the sound. Make the size change more intuitive.

Meanwhile, when the pointing means 100 according to the present invention is shaken in the long axis direction, the moving direction is recognized by causing a collision in the shaking direction through the first tactile output unit 131, so as to actually feel a physical click. It can also provide

18 shows an example of providing tactile feedback while performing a figure work using the pointing means 100 according to the present invention.

FIG. 18A illustrates an operation of changing a size of a figure by using a pointer. As in the embodiment of FIG. 9, the tactile output unit 130 uses a linear vibrator to increase a figure or more by unit coordinates. Every time it gets smaller, it can cause a reciprocating movement or a collision on one side.

Meanwhile, FIG. 18B illustrates an operation of rotating a figure using a pointer. In FIG. 18B, when the figure is rotated using a pointer, the tactile output unit 130 may generate a reciprocating movement or a collision to one side whenever the unit is rotated by a unit angle or more using a linear vibrator.

19 illustrates an example of providing tactile feedback according to an available state of a menu or an icon on which a pointer is located when a pointer is manipulated using the pointing means 100 according to the present invention.

As shown in FIG. 19, when the Internet is generally used or when various programs are used, when the pointer is placed on various links indicating hyperlinks, screen switching, and execution, the pointer becomes a hand. This indicates that the link is in a connectable state.

Accordingly, in the present invention, when the pointer is manipulated using the pointing means 100, when the link of the point where the pointer is located is connectable or an icon or the like is executable, the tactile feedback is provided.

For example, if the pointer is brought to HyperLink2 as shown in FIG. 19A, the shape of the link is changed or the shape of the pointer is changed, as in ' HyperLink2 '. In this case, the tactile information extractor 250 of the terminal means 200 extracts the tactile information, thereby outputting a corresponding tactile signal from the pointing means 100.

In addition, even when the shape and color are changed by taking the pointer to 'Link2' as shown in FIG. 19B, the tactile information extraction unit 250 of the terminal means 200 extracts the tactile information, thereby indicating the pointing means 100. ) Will output the corresponding haptic signal.

In addition, even when the pointer is moved to the play button icon as shown in FIG. 19C and the shape or color of the play button icon changes, the tactile information extracting unit 250 of the terminal means 200 extracts the tactile information. The pointing means 100 outputs a corresponding haptic signal.

Meanwhile, FIG. 20 illustrates an example of providing tactile feedback according to the movement of the pointer when the pointer is manipulated using the pointing means 100 according to the present invention. Specifically, the pointing means 100 according to the present invention is used. This is an example of providing tactile feedback when taking a commanding motion.

As shown in FIG. 20, the user can make the same movement as holding the pointing means 100 in his hand and conducting the beat in the air. In this case, the terminal means 200 controls the movement of the pointer displayed on the screen according to the movement of the pointing means 100 and outputs the tactile information corresponding thereto. Thus, the pointing means 100 can produce an effect of teaching commanding practice by generating a tactile stimulus corresponding to the commanding movement.

In particular, the commanding movement is shown on the screen in response to the position of the pointer on the screen, and it may also generate a momentary short vibration or collision within 100msec when a specific beat starts. In this case, a visual, tactile, and arm movement can be accompanied to provide a rhythm and command training system with a very high learning effect.

The motion-based pointing device for providing tactile feedback according to the present invention is used as a pointing device to replace an existing mouse by controlling a pointer on the screen connected to the terminal by using an angle change measured according to the movement of the user's left and right and upper and lower wrists. It may be used as a remote control for controlling a TV or an audio device.

Referring to the operation of the present invention configured as described above are as follows.

21 is a flowchart illustrating an operation flow for a control method of a motion-based pointing device for providing tactile feedback according to the present invention.

Referring to FIG. 21, when the user holds the pointing means 100 in his hand and takes a specific motion, the pointing means 100 detects a motion according to the movement of the user (S1900) and detects the motion by the terminal means 200. The signal is transmitted (S1910).

The terminal means 200 recognizes an operation or a control command of the pointing means 100 from the motion detection signal received in step S1910 (S1920). Thereafter, the terminal means 200 controls the movement of the pointer displayed on the screen in response to the operation or control command recognized in step S1920, and controls the operation screen accordingly (S1930).

In addition, the terminal means 200 detects the tactile output pattern based on the movement of the pointer displayed on the screen and the screen change according thereto (S1940), and extracts the corresponding tactile information (S1950).

The terminal means 200 transmits the tactile information extracted in step S1950 to the pointing means 100 (S1960).

Meanwhile, the pointing means 100 receiving the tactile information from the terminal means 200 controls the linear vibrator according to the received tactile information to output the tactile sense (S1970, S1980).

As described above, the motion-based pointing device and the control method for providing the tactile feedback according to the present invention are not limited to the configuration and method of the embodiments described as described above, but the embodiments may be modified in various ways. All or part of each of the embodiments may be configured to be selectively combined to make it possible.

1 is a view showing the operation of the motion-based pointing device for providing tactile feedback according to the present invention.

2 and 3 are views for explaining the configuration of the pointing means of the motion-based pointing device for providing tactile feedback according to the present invention.

4 is a view referred to for explaining the configuration of the terminal means of the motion-based pointing device for providing tactile feedback according to the present invention.

5 to 20 are views for explaining the operation of the motion-based pointing device for providing tactile feedback according to the present invention.

21 is a flowchart illustrating an operation flow for a control method of a motion-based pointing device for providing tactile feedback according to the present invention.

Claims (20)

Pointing means having a motion sensing unit for sensing a movement of a user and a tactile output unit for providing tactile feedback in the long axis direction or the left and right directions to the user; And A terminal means for manipulating a pointer displayed on a screen from the movement of the user sensed by the pointing means, and outputting corresponding tactile information to the pointing means according to a movement of the pointer and a screen change according to the manipulation of the pointer; Including; The haptic output unit may include one or more linear vibrators for generating a collision or vibration in at least one of up, down, left, right, front and back directions according to the tactile information output from the terminal means. The tactile information includes information about a location of the pointer, a moving distance of the pointer, a moving speed of the pointer, a moving angle of the pointer, a shape of the pointer, whether a screen is selected using the pointer, and an object indicated by the pointer on the screen. Whether it is executable, a window displayed on the screen, a menu displayed on the screen, an icon displayed on the screen, an emoticon displayed on the screen, a button displayed on the screen, a figure displayed on the screen, a shape of an area selected by the pointer, and And a change in at least one of a size of an area selected by the pointer, a position of the area selected by the pointer, and an angle of the area selected by the pointer. The method according to claim 1, The tactile output unit, Motion-based pointing device for providing a tactile feedback, characterized in that the oscillating body of the linear vibrator in one direction or both directions, or generates a vibration. The method according to claim 2, The tactile output unit, A motion-based pointing device for providing tactile feedback, wherein when a vibration occurs, a short vibration or a vibration that gradually increases or decreases in intensity. The method according to claim 1, The tactile output unit, Whenever the corresponding tactile information is outputted according to the movement of the pointer or the screen change according to the manipulation of the pointer, the vibrating body of the linear vibrator collides in the opposite direction or generates vibration. Motion based pointing device. The method according to claim 1, The tactile output unit, And a collision generated in a direction corresponding to the movement of the pointer. The method according to claim 1, The tactile output unit, And two linear vibrators disposed perpendicular to each other in two axial directions to generate collisions or vibrations using the vibrating body of the linear vibrator. The method according to claim 1, The tactile output unit, And three linear vibrators disposed perpendicular to each other in three axial directions to generate a collision or vibration using the vibrating body of the linear vibrator. The method according to claim 1, The motion detection unit, And an angular velocity sensor measuring an angular velocity in a three axis direction from the movement of the pointing means, and an acceleration sensor measuring an acceleration in the three axis direction from the movement of the pointing means. . The method according to claim 1, The pointing means, An input unit for receiving a control command from a user; And And a communication unit configured to transmit and receive a signal to and from the terminal means. The method according to claim 1, The terminal means, Recognizing the coordinate position on the screen corresponding to the movement of the user sensed by the pointing means to output a control signal for controlling the operation of the pointer, and to recognize the screen change according to the movement of the pointer or the pointer operation A motion recognition unit; And And a tactile information extractor configured to extract tactile information corresponding to a movement of the pointer or a screen change recognized by the gesture recognition unit. delete delete delete The method according to claim 1, The terminal means, A display unit provided with a screen; And And a communication unit configured to transmit and receive a signal to and from the pointing means. Detecting a movement of the pointing means according to the movement of the user; Outputting a control signal for controlling the operation of the pointer displayed on the screen in response to the movement of the pointing means; Recognizing a screen change according to a movement of the pointer or a manipulation of the pointer on a screen; Outputting tactile information in response to movement of the pointer or screen change recognized in the recognizing step; And And tactile outputting using a linear vibrator of the pointing means according to the output tactile information. Recognizing the step, Whether or not the object indicated by the pointer is executed on the screen, the position of the pointer, the moving distance of the pointer, the moving speed of the pointer, the moving angle of the pointer, the shape of the pointer, whether the screen is selected using the pointer, and A window displayed on the screen, a menu displayed on the screen, an icon displayed on the screen, an emoticon displayed on the screen, a button displayed on the screen, a figure displayed on the screen, a shape of an area selected by the pointer, an area selected by the pointer And a change in at least one of a size, a position of the region selected by the pointer, and an angle of the region selected by the pointer. delete delete delete 16. The method of claim 15, The tactile output step, The control method of the motion-based pointing device for providing a tactile feedback characterized in that the oscillating body of the linear vibrator collides in one or both directions, or generates vibration to tactile output. 16. The method of claim 15, The tactile output step, Control method of a motion-based pointing device for providing tactile feedback, characterized in that the collision or vibration in the direction of at least one of up, down, left, right, front, back using a vibrating body of the linear vibrator.

Images