JP2008015564A - Displacement informing device and gui program linking with displacement informing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a displacement informing device, informing an operator of the occurrence of blur when blur occurs due to deviation from a linear moving reference line in the operator's aerial operation of a displacement informing device (such as a mouse), that is, moving it linearly in the vertical and lateral directions. <P>SOLUTION: In this displacement informing device, input device bodies 1, 10 held by the operator's hand H and moved in the air to input data to a monitor screen are provided with gyro sensors 2, 12 for detecting that the input device bodies 1, 10 move with deviation from the moving reference lines La, 1b, driving means 3, 13, 31 operated according to commands from the gyro sensors 2, 12, and tactile sense stimulus means 4, 17 for stimulating the fingers H held by the operator by the driving means 3, 13, 31 to inform the occurrence of blur. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention aims to prevent erroneous input due to unforeseen movement of the operator's fingers and camera shake when inputting data into a small information device or shooting an image with a camera. The present invention relates to a displacement notification device that presents information related to the movement of a finger to the operator's tactile sense and enables the operator to correct the movement of the finger using the tactile sense presentation information as a clue.

  Devices that input data to a personal computer (PC) include a mouse and a trackball. These are used to specify menu items displayed on the monitor screen of the display device and to input coordinate data. Conventionally, an attempt has been made to add a device for presenting tactile information to a mouse or a trackball. For example, when the pointer specifies an object on the screen, the haptic sense presentation device of Patent Document 1 acquires property information of the object, converts the acquired property information into haptic information, and generates the haptic sensation. It is added to the mouse. This frees the operator from gazing at the screen.

In the tactile sensation presentation device described in Patent Document 2, when a mouse provided with light emitting means is spatially operated, the PC measures and calculates the position of the light emitting means. Based on the calculation result, the position and movement of the light-emitting means are detected, and tactile information can be presented to the mouse correspondingly. The techniques of Patent Document 3 and Patent Document 4 are both related to a trackball mouse, and the former adds a feeling of clicking each time the pointer passes through the border between adjacent menus on the menu selection screen. is there. In the latter case, when the pointer moves to the next menu after passing through the menu boundary, the trackball is sure to be in the selected menu area so that it does not overrun another menu area that is not intended to be selected. It is intended to be stopped.
JP 2001-296950 A JP 2005-78310 A JP 2004-265134 A JP 2004-287903 A

  However, since the techniques of Patent Documents 1 and 2 are not in the form of operating the mouse in the air, the operator is restrained by the desk. In addition, the techniques of Patent Documents 3 and 4 can swing the trackball mouse in the air, but the direction in which the operator always moves when the operator swings the mouse back and forth, left and right, or up and down. Alternatively, blurring (displacement) from the orbit occurs. This blur (displacement) causes problems such as the user cannot select a desired menu item smoothly.

  The present invention has been devised in view of the above problems. When an operator moves a displacement notification device such as an input device or an imaging device in the air in the up / down, left / right, and front / rear direction, the blur is detected. Then, a displacement notification device is provided that notifies the operator's tactile sensation that the blur has occurred, and allows the operator himself to intuitively correct the position of the device based on the notified information. Is the first purpose (main purpose).

  Similarly, for the 6-axis movement including the rotation of the device body, the reference point, the reference direction, and the reference trajectory are set in the same manner, and the position and angle of the device body from this are reported to the operator's tactile sense. A second object is to provide a displacement notification device.

  Furthermore, when the operator is required to keep the position of the displacement notification device constant, for example, when it is desired to prevent blurring of the captured image by holding the camera at a certain position at a certain angle, or for a manual such as a robot hand. The tactile sensation that the operator presents from the displacement notification device when it is desired to hold the pointing device in a certain position during the operation or when it is desired to move these devices in a certain direction along the specified trajectory without blurring. Tactile presentation that guides you so that you can keep the position of your fingers constant and move along a specific straight line or trajectory by incorporating the presentation information into the feedback loop of your senses and movements. A third object is to provide a displacement notification device that changes and presents information according to the position and orientation of the apparatus body.

  It is a fourth object of the present invention to provide a graphical user interface (GUI) program used in combination with the displacement notification device of the present invention so that the menu item selection can be accurately performed using the displacement notification device held hollow. Objective.

  The present invention informs the operator that a shake (displacement) has occurred during the movement operation of the data input device or the like by the operator, and provides tactile information for correcting this.

  In particular, in order to correct this displacement, the operator detects the displacement at any time, changes the stimulus amount continuously according to the magnitude of the displacement, presents it to the operator as needed, and the operator determines the magnitude and direction of the displacement. Assuming that virtual reference points, reference lines, and reference planes are assumed in the outside world so that it is easy to grasp intuitively, it is necessary to present the blurs from the reference points, reference lines, and reference planes in an intuitive manner. It is a feature.

  Furthermore, the present invention is as described above so that when the menu is selected or the item that is highlighted in the menu is changed, the movement amount below a predetermined threshold is regarded as blurring and the operator can correct it. It is desirable to present the tactile information in a certain form, but if it exceeds the predetermined threshold and is judged to be a move to change the menu item that is intentionally selected or highlighted, the menu has changed. And a clear tactile stimulus that informs that the highlight has moved to another item, and the reference point itself is reset according to the position of the new menu or item that has been switched. Yes. In this way, a menu or an item can be displayed by using a tactile stimulus that changes continuously for correction and a tactile stimulus that changes rapidly to clearly notify that the reference point has been changed. Tactile feedback at the time of blurring is possible without impairing the operability of positioning.

  The features of the displacement notification device and the GUI program according to the present invention will be specifically described below.

  (Claim 1) In order to solve the above problems, the present invention takes the following measures. That is, the displacement notification device according to the present invention is a displacement notification device that an operator operates while grasping in the air and detects a position in a two-dimensional or three-dimensional space, and the device body is virtually connected to the outside world. A displacement detection sensor that detects displacement from a provided reference index, a drive unit that operates based on detection information of the displacement detection sensor, and a toucher that touches an operator's finger by the drive unit, Tactile stimulation means for notifying an operator of a displacement direction or a displacement amount of the apparatus with respect to the reference index, and the driving means moves the tactile while interlocking or synchronizing with the movement of the displacement of the apparatus main body. Thus, the displacement direction and the displacement amount of the apparatus main body are notified through an operator's sense of touch. Thereby, the operator can be given a clue for correcting the movement of the main body of the displacement notification apparatus (hereinafter referred to as the apparatus main body) contrary to the operator's intention in a stage where the shake is minute.

  In the present invention, when the operator grips the apparatus body and moves the apparatus body by moving the wrist or arm in the horizontal direction or the vertical direction while moving the wrist or arm, the displacement detection sensor built in the apparatus body is two-dimensional or three-dimensional. The reference index virtually set in the dimensional space, that is, how much displacement is generated from the reference point, reference direction, or movement reference line, reference plane, or trajectory set in any direction, its displacement direction and displacement Measure. Then, a control signal corresponding to the displacement calculated by the CPU based on the measured value and the accumulated amount of the measured value is output to the driving means. As a result, the driving means is actuated to drive the tactile stimulation means, and tactile stimulation is applied to the hand or finger (finger).

  As a result, the operator is notified that the apparatus main body has shifted from the reference point, the movement reference line, the reference plane, and the reference direction. For this reason, the operator gives the movement direction and movement amount of the apparatus main body or the rotation direction and rotation amount from the tactile stimulation means even though the operator moves or rotates the apparatus main body without any clue. When an unexpected displacement is sensed while being sensed through a stimulus, the trajectory is corrected so as to correct it, and the apparatus main body can be accurately moved or rotated in a three-dimensional space. What is important here is that the tactile stimulus presenting means does not apply the tactile stimulus to the fingers at random intervals but continuously and continuously in real time in conjunction with the movement of the apparatus main body. If a tactile stimulus is given in such a way, the operator immediately grasps any displacement that occurs, and corrects the direction and amount of movement of the device body before the device body greatly deviates from the reference index. can do. Furthermore, it is important to devise a tactile stimulus form so that the direction and amount of displacement can be easily grasped intuitively.

Here, the “reference index” means a reference point, a reference line, a reference plane, or a reference direction.
“Displacement” means a change in position and includes so-called movement and rotation.
The “displacement notification device” means a device having a function of detecting displacement such as movement or rotation, and an input device for a computer or a photographing device such as a camera also has such a function as a displacement notification device. included.

  (Claim 2) When the cursor is moved or the item to be highlighted is changed by the relative position change amount or angle change amount instead of the absolute position or angle of the apparatus main body described above, When moving the main body of the device, the cursor will always move, or the highlighted item will change. It is necessary to be able to move the main body of the apparatus without moving or changing them. Therefore, only when the apparatus main body is moved while touching a predetermined switch or touch sensor (for example, a capacitance sensor) with a finger, the cursor moves or the item to be highlighted changes. In particular, it is preferable to activate the function only when a finger touches the touch element and is touched.

  (Claim 3) In addition, whether the displacement from the reference index is an unexpected displacement caused against the operator's intention, or a displacement caused by an intentionally performed operation for the purpose of selecting a menu item, etc. In the former case, tactile information that changes continuously as appropriate to correct the displacement, and in the latter case, clearly indicate that the selected item has been switched. It is desirable to present discontinuously and rapidly changing haptic stimuli suitable for Therefore, when the amount of displacement exceeds a predetermined threshold, it is determined that the operator has intentionally moved the device body to another target point, and the tactile stimulus is suddenly changed to present that the reference point changes. Like that. At the same time, the position of the device body at the time when the threshold value is exceeded is set as a reference position for selecting the newly selected menu item, and the tactile stimulus is returned to the neutral stimulus form. If the amount of displacement from the newly set reference point is within the threshold, the tactile stimulus to assist the correction of the displacement, and if it exceeds the threshold, the tactile stimulus to indicate that another item has been moved To be presented.

  Here, “continuous” means that it is sufficient if the user can feel continuous. The term “discontinuous” means that it is sufficient if the user can feel that the user is discontinuous or a sudden change compared to the previous movement.

  (Claim 4) To correct the displacement of the apparatus main body, the operator can intuitively determine from which tactile stimulation means the tactile stimulation means how much the apparatus main body should be moved. Therefore, in the real world, touch with a tactile stimulation means to reproduce the tactile sensation that is felt when moving the device body while touching a reference object that is actually stationary at the reference position in the outside world with a part of the finger. Move the child to stimulate the tactile sensation of the fingers. In the real world, when a part of the finger holding the device body is touching a reference object stationary in the outside world, if the device body is moved in a certain direction, the reference object touches the finger while touching the finger. Will move in the opposite direction to the main body. The movement continuously moves in conjunction with the movement of the apparatus body. In the present invention, the toucher is mounted on the main body of the device, but when the main body of the device is moved, the portion in contact with the finger of the toucher is opposite to the direction of the movement in conjunction with the movement of the main body of the device. It is configured to move in the direction and reproduces the tactile sensation when the reference object is touching the finger.

  (Claim 5) When the touch element is moved relative to the apparatus main body, a gap is formed between the touch element and the apparatus main body, and there is a possibility that dust or water may enter from there to break down the apparatus. Therefore, the space between the touch panel and the main body of the device is covered with a film such as plastic silicone rubber to prevent the intrusion of dust and water.

  (Claim 6) If the actuator is to be driven directly by a motor or the like, the motor must be rotated at a high speed when the actuator is to be rapidly returned to the neutral point, and the rotational speed is limited. , The return speed of the tentacle will be slow. Therefore, by using a cam in which the distance from the rotating shaft to the outer periphery is unevenly formed according to the angle from the rotating shaft, the cam is driven by pushing up the outer periphery of the cam, and the outer peripheral radius of the cam is increased. If the radius changes suddenly at a certain angle, the tactile will quickly return to the neutral point after the cam reaches the specified rotation angle without increasing the motor rotation speed. Can be made.

  (Claim 7) The tentacles notify the displacement in the horizontal direction (horizontal direction) of the apparatus main body, notify the displacement in the vertical direction, notify the displacement in the front-rear direction, and so on according to the direction of displacement. Although it may be provided separately, the stimulus is complicated for the operator, and it may be difficult to grasp the direction and amount of displacement. Therefore, the driving means has been devised so that one common tentacle can be moved in two orthogonal directions to make it easier for the operator to intuitively grasp even a two-dimensionally moving displacement. However, this may cause another problem that makes it easy to confuse displacement in the two directions.

  (Claim 8) The stick is a stick, and the tip of the stick is moved back and forth, left and right or up and down, the movement of the base is reduced, and the gap between the base and the main unit is further reduced. By covering it with a plastic film that easily deforms, such as silicon rubber, dust and water resistance are ensured. Also, when the device body is moved, the tip of the stick tilts with respect to the device body, and if the tip of the stick moves in the direction of a reference point virtually set in the outside world, By moving the apparatus main body while touching a part of the apparatus, it is possible to grasp the relative movement of the apparatus main body with respect to the virtual reference index in the outside world. As a result, the operator can move the input device body while correcting the movement so as to approach the reference index.

  (Claim 9) An optical movement amount detection sensor used in an optical mouse pad or a sensor for optically detecting the position of a light source or a marker as a sensor for detecting that the apparatus main body is displaced from the reference index. Was used.

  (Claim 10) As a sensor for detecting that the apparatus main body is displaced from the reference index, a camera for photographing the movement of the apparatus main body or a camera mounted on the apparatus main body for photographing the movement of the outside world is used. The movement of the device body is detected from the change of the image taken by the camera.

  (Claim 11) A gyroscope or an acceleration sensor mounted on the apparatus main body is used as a sensor for detecting that the apparatus main body is displaced from the reference index. A gyroscope can be used to detect rotation about three axes in the three-dimensional space of the apparatus body. Further, acceleration in the three-axis direction in the three-dimensional space of the apparatus main body can be detected using the acceleration sensor.

  (Claim 12) The preferred amount of tactile stimulation varies depending on the operator. In addition, young people with high finger control ability can accurately position the device within a narrow space and perform operations such as menu selection. Using a wide range, the tolerance of displacement is increased to allow the position error of the main body of the device, and menu selection and other operations are performed. Also, depending on the type of information to be input and the situation, there are cases where you want to roughly read out a large range of information and when you want to specify information within a narrow range precisely. It is preferable that the correspondence relationship between the movement amount of the cursor and the highlight with respect to the movement amount of the main body can be changed. Therefore, the threshold value used when changing the reference position of the displacement notification device, or the sensitivity changing means for changing the correspondence that determines the amount of movement of the tentacle and the amount of movement of the cursor and highlight with respect to the amount of displacement generated in the device main body. Threshold adjustment means that can be adjusted is provided.

  (Claim 13) It is difficult to move the apparatus main body held in the air in the three-dimensional space as intended. In particular, when the movement of the apparatus main body and the movement of the cursor displayed on the monitor screen cannot be visually carefully confirmed, it is almost difficult to move the apparatus main body accurately. For example, when the menu displayed on the monitor screen is changed by moving the device main body in the front-rear direction, and the item to be highlighted in this menu is changed by the operation moving the device main body in the left-right direction, the operator Even if he intends to move the device main body only in the left-right direction in order to change the item to which the highlight is applied, it may frequently occur that the device main body moves in the forward / backward direction by mistake. If the displayed menu is changed due to this blur, the operator cannot input as desired. Therefore, in order to prevent such inconvenience, a displacement alarm device that notifies the motion blur in the front-rear direction through the tactile sensation is introduced to the device main body, and the motion is controlled when the device main body moves slightly in the front-rear direction. The operator so that the operator can correct the blur. When the operator moves the device body in the front-rear direction beyond a predetermined threshold without correcting the shake, the operator switches the menu by assuming that the device body has moved in the front-rear direction with the intention of switching the menu. To. In addition, when the threshold is exceeded, the touch panel is discontinuously and rapidly returned to the neutral position to notify the operator that the menu has been switched, and this time, in order to perform the same displacement notification in the switched menu, The accumulated movement amount or rotation amount is reset, and the device position immediately after the threshold is exceeded is newly set as a reference index. Note that this is not limited to the menu selection, but is always a problem encountered when information 1 and information 2 are independently input using the movement of the apparatus main body in two directions 1 and 2, respectively.

  (Claim 14) When the displacement notification device is mounted on a photographing device, a gyroscope that has already been introduced as a sensor for detecting a change in the position and angle of the device main body to compensate for blurring of a photographed image of the camera. By sharing the acceleration sensor or the image sensor for the displacement notification device, the cost of the displacement notification device is reduced.

  (Claim 15) Preferably, based on the displacement speed of the apparatus main body detected by the displacement detection sensor, the amplitude or the period of the movement of the tentacle driven by the finger stimulation means may be changed. For example, when the apparatus main body is moved slowly, the movement of the touch element is reduced.

  Furthermore, by changing the amount of movement of the curl according to the displacement speed, it is possible to flexibly perform a wide range of menu selections to a narrow range selection within a limited finger movement range.

  (Claim 16) A GUI (graphical user interface) program used together with the displacement notification device described above, which highlights a predetermined item in a menu and displays the item separately from other items. At this time, the item to be highlighted or the cursor pointing to the item is moved based on the displacement direction and displacement amount of the apparatus main body up and down, left and right, or front and rear.

  The computer on which the GUI program is installed is installed at a location different from the displacement notification device, and includes not only the case where data is exchanged by wireless or wired communication, but also the case where it is provided in the displacement notification device. . An example of the latter is a photographing device such as a digital camera.

  (Claim 17) When the GUI program is mounted on the computer device side that communicates with the displacement notification device, the communication processing of this GUI program is a one-way communication processing for receiving information sent from the displacement notification device. In addition, when the number of communications is also limited to save power, it is desirable to configure the displacement notification device and the GUI program so as to overcome such communication processing limitations. Therefore, in order to realize the selection of the menu items described above using unidirectional communication in which a signal transmitted from the displacement notification device is received by the computer device, the displacement notification device has a displacement amount of the device body. When an event that exceeds a predetermined threshold occurs, information on the occurrence of the event and the displacement direction of the apparatus body is transmitted to the computer apparatus by a one-way communication process, and the computer apparatus that has received this information receives the information. Immediately after that, the GUI program is configured to change the item to be highlighted in the menu to another item in a direction corresponding to the displacement direction of the apparatus main body. As a result, the communication is executed only when the above event occurs, so the power required for the communication can be saved, and the communication is unidirectional. The menu selection operation can be performed.

  (Claim 18) When bidirectional communication can be performed at any time between the displacement notification device and the computer device on which the GUI program is installed, the displacement notification device transmits the displacement amount of the device body detected by the sensor to the computer device. Since it can be transmitted at any time, before the displacement exceeds a predetermined threshold, the computer device prepares to change the item to be highlighted in the menu to another item in a direction corresponding to the displacement direction of the device body. be able to. For example, the cursor can be moved before completely changing the item to be highlighted to allow the operator to check whether the cursor is moving in the correct direction. Moreover, after completing the change of the item to be highlighted on the computer device side, a signal notifying the completion is transmitted to the displacement notification device side, and after receiving this signal, the tentacles are discontinuous. By quickly returning to the neutral position, the operator can proceed with the operation while confirming that the highlighted item has been changed.

  When holding or moving an input device or a photographing device in the air in a two-dimensional or three-dimensional space, feedback information regarding the position and movement of the device has conventionally been provided only through vision, so visual attention is always given during operation. However, according to the present invention, the displacement detection sensor built in the input device or the photographing device main body detects the movement of the device, and drives the finger that stimulates the finger based on the detection result. , Feed back the reference point, reference line, or blur from the reference plane that occurs when holding the device in the air or moving it to input data to the operator's tactile sense Even if it is not, it is possible to tactilely check the shake and correct the position and movement of the apparatus main body before the shake exceeds the threshold.

  On the other hand, when the amount of movement of the main body of the device exceeds the threshold value even though such correction information is provided, the operator has the intention to cross the border between adjacent menus on the menu selection screen. It is determined that the device has been moved, and the highlight of the menu item is moved to the next item, and the tactile stimulation means is moved abruptly to return to the neutral position to notify that the highlight has moved to another item. This time, with respect to the item that moved, the tactile feedback was given by distinguishing the change of the menu item with blur and intention in the same way, using the position of the main unit after the movement as a reference point. The person can operate the input device accurately, or can take an image without blurring the image using the photographing device. In addition, the manipulation of the robot arm can accurately give the position and trajectory of the pointing device through the sense of touch.

[First embodiment]
Hereinafter, a first embodiment of the present invention will be described in detail with reference to FIGS. FIG. 1 is an external view showing a state in which an input device main body 1 that is a displacement notification device is gripped by a hand H and operated in the air. The input device main body 1 has a small shape that can be grasped by the hand H, and is used as an aerial mouse (controller) for controlling a control target such as a television, a game machine, a computer monitor screen, or a robot. is there. The basic form of the displacement notification device has an input device main body 1, and the input device main body 1 is mounted with a known gyro sensor 2 that can read two-dimensional or three-dimensional spatial coordinates on an electronic circuit board (not shown). Configured. Note that the gyro sensor basically detects rotational motion, and an acceleration sensor can also be used when detecting translational motion. The electronic circuit board incorporates computer elements such as CPU, RAM, ROM, etc., and driving means 3 (two 3a and 3b in the present embodiment) comprising a battery, a stepping motor or a servo motor for radio control (not shown), A tact switch for giving a command to execute an application, and a transmission / reception unit for wirelessly receiving and transmitting information (data) such as position information, operation signals, and control signals between the control target and the input device body 1 by radio waves, or An infrared LED, a PD, and a tact switch for instructing execution of an application are mounted.

  Sensors such as the gyro sensor and the acceleration sensor have a function of detecting a change in spatial coordinates based on the shake (movement) when moving while holding the hand body, such as shaking the device body held by the hand H, When moving the input device body 1 with the hand H in the air, the reference point or linear reference line (or more generally, a curved orbit may be used as the reference line) or the device from the reference plane The amount of shake that the main body is displaced is calculated by a computer, and a drive signal is output to the drive means 2a and 2b based on the calculation result. Here, “reference plane” and “reference line” mean that when the device body 1 is moved up and down, left and right, and back and forth, it is desirable for the device to move on that plane or on that line. It is a line (orbit). When moving the device body along this trajectory in three-dimensional air, the person's hand floating in the air cannot feel tactile cues about the direction of movement, amount of movement, or speed. It is difficult to move like moving. The only perceptual means that can be used to correct the trajectory is sight, and you should keep your nerves focused and always pay attention to the movement of the device body or the cursor movement on the monitor screen reflecting the movement of the device body during operation. I must.

  For example, as a reference plane, La in FIG. 3 that serves as a reference for the vertical movement of the device body (in FIG. 3, the surface is actually shown as a dotted line for the convenience of the page), and a reference figure for the forward and backward movement. A reference plane such as 4 Lb is conceivable. “Blur” or “displacement” refers to the amount of deviation when the position of the device body deviates (displaces) from this reference plane.

  Note that there is an error in detection information such as the current gyro sensor, and it is difficult to accurately detect the amount of movement from the reference position. Therefore, especially when the displacement notification device is mounted on an input device and used, the reference plane, reference point, and reference line used in the above description are very small even if they are not actually fixed at fixed positions in the space. A virtual reference plane, reference point, or reference line that may move continuously or continuously. Even if this virtual reference plane, reference point, and reference line do not remain constant, if they move slowly and continuously in time, human perception is incomplete, so the movement is noticed. Therefore, the input operation can be performed without confusion. Also, when mounted on a shooting device, if the reference point moves slowly enough to not affect shooting, the shot image will be blurred even if the shooting device moves to that reference point. There is nothing. Therefore, the positions of the reference plane, reference line, and reference point do not necessarily have to be kept constant, and are not a reference point that is truly fixed in the three-dimensional space, but move minutely or continuously during the movement or rotation of the device. It may be acceptable. However, if it moves discontinuously and greatly, it will be troublesome because people will be confused during the operation and will not be able to operate correctly.

  Thus, even if the actual reference position is moving, the operator determines the information to be input based on the form of the tactile stimulus fed back by the tactile stimulus means described below, not depending on the position of the device body. In this case, the input information determined by the operator based on the form of the tactile stimulus and the actually input information are always consistent, so that no problem occurs. However, even in this case, if the change in the reference position is not slow and continuous, the operator will feel a change in the form of tactile stimulation that is difficult to predict during the input operation. It will be confusing to the operation.

  The input device main body 1 is provided with tactile stimulation means 4 for notifying that a shake has occurred by applying a stimulus to fingers held by an operator. The tactile stimulation means 4 has a base that is swingably provided on the input device main body, and a free end that touches the belly of a finger that grips the input device main body 1 When the input device body is moved with a shake from the reference plane, the stick is tilted in the direction of the reference plane (swings). ing. Here, the stick tip does not always have to be exactly on the reference plane. Of course, it is preferable if you are riding, but as mentioned above, there are errors in the sensor, and because the movable range of the stick tip is limited, it may be impossible to always put it on the reference plane, In such a case, it is sufficient to make it possible to grasp the moving direction of the device body from the tilt direction of the stick. However, for the purpose of preventing blur of the photographing device, it is desirable that the tip of the stick is always on the reference line accurately.

  Specifically, in one embodiment, the stick touches the forefinger on the front side of the input device body 1 and swings up and down the position of the device body 1 up and down in the direction opposite to the shake ( Touch the top / bottom blur notification stick (hereinafter referred to as “index finger stick”) 4a to be notified by tilting, and shake the front and rear of the position of the device body 1 back and forth in the opposite direction to the blur (tilt). ) And a forward / backward blur notification stick (hereinafter referred to as “thumb stick”) 4b. The index finger stick 4a or the thumb stick 4b is tilted up and down or back and forth according to the amount of shake as described above according to the command of the gyro sensor 2, and the person uses this information to stimulate the finger. It is necessary to present a tactile stimulus in a form that can correct the movement of the hand of the user. In other words, if the blur is suddenly moved to notify that the blur has occurred, the human correction is not in time. First, an allowable range is set for the size of the shake, and a shake below a certain threshold is allowed, and the input device generates the same input even if such a shake occurs. Further, since the blur is small, it is necessary to reflect it on the amount of tilt of the stick, and to continuously reflect the size of the blur on the amount of tilt and present it to the finger. In addition, it is desirable to apply tactile stimulation to the finger in a form that makes it easier for a person to intuitively understand the movement of the device body. For that purpose, it is preferable to generate and apply a stimulus so as to simulate the stimulus felt by the finger when the finger is moved in a state where one end of a stationary bar in the outside world is actually touching the finger.

  For example, when the device body is displaced, the stick may be tilted so that the tip of the stick that touches the finger stays at a reference point virtually set in the outside world. However, this need not be exact. For the time being, if the stick is tilted continuously without time lag in synchronization with the displacement of the device body, and the direction of the tilt is opposite to the movement of the device body, the person will stick the stick tip in the outside world anyway. And the illusion that the body is moving relative to it. If the type of information to be input is determined by the relative position of the device body relative to the tip of the stick rather than the absolute position of the device body in the outside world, the tip of the stick will actually be in the outside world for the purpose of an input device. There is no need to be stationary.

  Thus, based on the amount of shake when the device body 1 detected by the gyro sensor 2 deviates from the reference plane, a calculation by a computer is intervened as necessary, and the calculation result is output to the driving means 3a or 3b, and each stick 4a , 4b are tilted.

  However, the index finger stick 4a is stationary in a horizontal posture with respect to the outside in a normal state, and the horizontal posture is kept constant even when the input device main body 1 is moved back and forth and right and left within the reference plane La, and the thumb stick 4b is used as the input device. Even if the main body is moved vertically and horizontally within the reference plane Lb parallel to the monitor screen, the body stands still with the posture standing up and down kept constant.

  On the other hand, the input device body 1 is provided with a dial 5 for adjusting sensitivity or threshold. By operating the dial 5, the function that determines the correspondence between sensor sensitivity, device displacement, and stick tilt amount, or the threshold that determines the allowable range of blurring is adjusted, and the operator's preferences and senses, individual differences in motor skills, and input The characteristics of the displacement notification device can be adjusted according to the difference in information.

  The operation of the first embodiment will be described. FIG. 3 is an explanatory diagram for explaining how a stick (tactile) that stimulates a finger moves when the input device body 1 is displaced in the vertical direction. In FIG. 3, the reference plane La is indicated by a dotted line in the left-right direction on the paper surface for the purpose of illustration. However, when the operator faces and moves the monitor screen as the control target, It intersects perpendicularly to the screen and becomes a plane that spreads forward, backward, left and right.

  As shown in FIG. 3A, when the device main body is at the height of the reference plane La, the index finger stick 4a is horizontal. Further, when the input device body 1 is displaced upward or downward with respect to the reference plane La as shown in FIGS. 3B and 3C, the index finger stick 4a is tilted downward or upward, respectively, and the tip of the stick is always at the top. It is on the reference plane La. The operator has the illusion that the tip of the stick is the tip of a reference bar that is maintained at a constant height in the outside world, so that the displacement of the device body can be sensed intuitively and easily. Based on the perceived result, the hand is moved up and down so that the height of the device body is kept constant with respect to the tip of the reference bar that is virtually perceived (the height of the device body is corrected). Thus, the device body can be moved back and forth and right and left while correcting the trajectory so as to move along the reference plane La while keeping the height of the device body constant. Thus, without visually confirming, the device body can be moved back and forth and right and left within the reference plane La while constraining the height of the device body on the reference plane La using the tactile cues.

  Similarly, as shown in FIG. 4A (the “index finger stick 4a” is omitted in FIG. 4), the thumb stick 4b stands vertically, and the input device body 1 in the neutral state is moved forward from the reference plane Lb. Suppose that it is shifted to (biased on the monitor screen side). The amount of deviation is measured by a gyro sensor, and as shown in FIG. 4B, the driving means 3b is driven based on the measurement result, and the thumb stick 4b tilts (swings) toward the reference plane Lb. As a result, a stimulus is applied to the thumb of the operator, and notification is made that the device body has been displaced forward of the reference plane. Further, as shown in FIG. 4C, when the input device body 1 is displaced backward from the reference plane Lb, the thumb stick 4b swings forward, and the input device body 1 is displaced backward from the reference plane. Notify that.

  The operator who has been notified of the occurrence of blurring back and forth corrects the position of the device body by correcting the position of the input device body 1 so that the thumb stick 4b returns to the upright state as shown in FIG. It is possible to move freely in the vertical and horizontal directions within the surface while restraining within the reference surface Lb.

  As described above, according to the first embodiment, the operator holds the input device body 1 and uses the information given to the finger from the tactile stimulation means as a clue without visually checking the monitor screen or the device body. The device body can be moved while restraining the position of the device body within the reference plane. Combining the principles described in FIG. 3 and FIG. 4, if the device body is moved so that both the index finger stick and the thumb stick are kept in a neutral state, the movement is restricted to the intersection of the reference planes La and Lb. You can easily understand that you can. Further, as shown in FIG. 14, a stick 4c that touches a finger while being orthogonal to both the index finger stick 4a and the thumb stick 4b is introduced, and the displacement of the device body from the reference surface Lc orthogonal to both the reference surfaces La and Lb is also sensed by touch. If it can grasp | ascertain automatically, the position of a device main body can be restrained to the intersection of three reference planes. In this way, when inputting information by operations such as the position of the device body, movement, rotation, etc., the position and movement of the device body can be checked tactilely without having to visually confirm the position and movement of the body. An operation corresponding to the input data can be executed accurately and reliably while correcting. Further, since the dial 5 for adjusting the relationship between the displacement of the device main body and the amount of movement of the stick is provided, the degree of tilting of the sticks 4a and 4b with respect to the movement of the device main body is determined according to individual differences and the type of information to be input. It will be possible to adjust accordingly.

  Actually, the device main body may move even if the position of the device main body is corrected so that the stick stands upright due to a sensor error or restriction of the tilting amount of the stick. In such a case, rather than deciding the type of information to be input based on the absolute position of the main body in the outside world, it is better to decide the type of information to be input corresponding to the degree of tilt of the stick and the way the stick moves. good. That is, even if the position of the device itself has actually changed, if the degree of tilting of the stick is kept constant, it is considered that the position of the device body remains the same, and the type of information to be input It is determined. From the operator's perspective, it is not only the absolute position of the device itself, but the status of the operation that is being performed, based only on the tilting status of the stick. Consistency between the situation grasped by the person and the input information is maintained.

[Second Embodiment]
Next, a second embodiment will be described. As shown in FIG. 5 and FIG. 6, the basic form of the input device body 10 that is the displacement notification device according to the second embodiment is that the shake occurs with respect to the vertical shake that occurs in the input device body 10. It is configured to notify the operator. In the second embodiment, the circuit board 11 is built in substantially the same as the first embodiment. Further, the circuit board 11 has an infrared LED that emits infrared rays to a receiving unit on the monitor side or an infrared ray emitted from a light emitting unit on the monitor side as a communication means with the gyro sensor 12, the battery 13, the tact switch 14, and the monitor side. A light emitting / receiving unit 15 that functions as an infrared PD that receives light and a driving unit 16 are mounted. The configuration of the tactile stimulation means 17 is greatly different from that of the first embodiment.

  That is, the tactile stimulation means 17 in the second embodiment includes a disk-shaped movable plate 17a that is floatingly supported on the input device body 10 via a thin silicon rubber ring-shaped rubber film 17b, and a movable plate 17a. And a tilting means 18 which is disposed on the lower surface and tilts the movable plate 17a back and forth by the driving means 16. In this way, there is no gap between the main body and the movable plate, so that dustproofness and waterproofness are improved. The finger is stimulated using the tilting movable plate as a touch to notify the displacement of the device body.

  The movable plate 17a is provided with small projections 17c opposite to the front and rear peripheral edges of the upper surface thereof, so that the tilt direction of the movable plate 17a can be grasped more clearly. A displacement sensor may be detected only when a finger is touched by providing a capacitance sensor 19 for detecting the touch of the finger or a push button switch operated by pressing on the movable plate 17a. In this way, the operator can continue to move the cursor in a certain direction even in a narrow operation range.

  The tilting means 18 is disposed on the lower surface side of the movable plate 17a and the protrusion 18a on the lower surface side of the movable plate formed at a position corresponding to the small protrusion 17c on the upper surface side of the movable plate, and is orthogonal to the rotating shaft 16a of the driving means 16. The arm 18b extends in the rotational radius direction, and the roller 18c is rotatably attached to the arm 18b. Since the protrusion 18a exists on the track through which the roller 18c passes when the arm 18b rotates around the rotation shaft 16a, when the driving means 16 operates and the swing arm 18b swings (rotates) in any direction. The roller 18c comes into contact with any one of the projections 18a and pushes up the projection, so that the movable plate 17a is pushed up from below and tilted.

  In the second embodiment, the input device body 10 is gripped with the thumb placed on the movable plate 17a and moved in the air, but the position of the device body is virtually set in the outside during the movement. While on the reference plane, as shown in FIGS. 6A and 6B, the roller 18c and the protrusion 18a are in a positional relationship where they do not contact each other, so that the movable plate 17a is held in a horizontal state. ing. On the other hand, when the position of the device body deviates from the reference plane virtually set in the outside world, the arm 18b rotates around the rotation axis 16a according to the displacement amount (blur amount), and FIG. (D) Alternatively, as shown in FIGS. 6 (e) and 6 (f), the roller 18c contacts the protrusion 18a and pushes the movable plate 17a upward from below to tilt the movable plate 17a. As a result, stimulation is given to the thumb from the small protrusion 17c, and it can be seen that the operator is shaken. In this way, the input device body 10 is moved while being constrained within the reference plane, the cursor or highlight on the monitor screen is moved in accordance with the movement within the reference plane, and the tact switch 14 is pressed at an appropriate position. Thus, the application can be executed by inputting data to the control target.

  Thus, according to the second embodiment, the position of the input device is restricted within a specific reference plane by moving the input device while correcting the position of the input device so that the tilt angle of the movable plate 17a is kept constant. The reference plane can be moved two-dimensionally. For example, when the screen cursor is moved by such a two-dimensional movement in the reference plane and the device is moved so that the movable plate 17a changes the tilt angle, the content displayed on the screen is enlarged or It should be reduced. Then, using the tactile stimulus as a clue, it is possible to move only the cursor while keeping the scale of the content constant. On the other hand, if no tactile stimulus is given, it is difficult to restrain the device body within the reference plane, so even if you intend to move the device body within the reference plane to move the cursor, And the scale of the content changes carelessly, causing confusion in the operation.

  The swing arm 18b is configured to extend in a direction orthogonal to each other, but the extending direction is not limited to orthogonal, and may be extended in a C shape.

  The movable plate 17a is mounted with a touch sensor or a contact switch on a film that is turned on and off by a minute force, and detects whether or not a finger is on the movable plate 17a. For example, when the cursor is moved as described above, the movement of the main body is transmitted to the movement of the cursor only when a finger is on the movable plate 17a.

[Third embodiment]
In the example of the third embodiment shown in FIG. 7, the device body is moved and used in a state where the device body is inclined obliquely forward. This is because a pattern or a mark on the surface of the optical mouse pad 20 placed on the surface is optically detected, and the movement of the device is detected from the movement. As the optical sensor, an existing mouse coordinate input sensor or a CCD image sensor or the like may be used. In particular, when the present apparatus is mounted on a photographing device such as a camera, the image sensor originally provided in the photographing device may be shared for the purpose of the present apparatus. In this case, the movement of the photographing device body may be detected using the movement of the outside scene as a clue instead of the pattern of the mouse pad on the desk.

  In the embodiment shown in FIG. 7, the input device main body 10 moves the movement of the device main body 10 from the movement of the texture on the surface of the mouse pad that the sensor unit 21 moves on the optical mouse pad 20 at that time. measure. With this optical sensor, two-dimensional or three-dimensional position information is input, or the gyro sensor 12 or the like is also used separately to detect the movement and rotation of the device body 10. The method of tilting the movable plate 17a with respect to the movement and rotation of the device main body 10 detected in this way and the usage thereof are the same as in the second embodiment.

[Fourth embodiment]
As shown in FIGS. 8 and 9, the fourth embodiment differs from the second and third embodiments in that the method of tilting the movable plate 17a is biaxial and the movable plate 17a is tilted in two directions. Only the different configuration will be described below. In addition, the same code | symbol is attached | subjected to the same or equivalent member as 2nd embodiment.

  That is, the tilting means 30 is attached to the rotating shaft of the driving means 31 extending in the lateral direction of the lower surface of the movable plate 17a and the rotating shaft, and is in an opposite phase so that the movable plate 17a can be tilted, and from the rotating shaft. It is comprised with two cam bodies which formed the outer periphery shape so that the diameter of may become uneven. The diameter of the outer periphery of the cam body as viewed from the rotation axis of the cam varies depending on the direction and is uneven, but the way of changing the diameter relative to the direction is the movable plate 17a when the rotation axis is rotated. Determine the tilt pattern. The tilt pattern can be changed by changing the rotational speed of the rotary shaft. This rotation axis may be one axis, but if two pairs of cam bodies are rotated as described above using two axes crossing and intersecting with each other, the movable plate 17a is tilted independently around the two axes, A two-dimensional tilt pattern can be generated. Below, the case where it forms in a biaxial type is mentioned as an example, and is demonstrated.

  More specifically, two pairs of driving means 31 and rotating shafts driven and rotated by the driving means 31 are prepared. The two rotary shafts 32 and 33 are arranged so as to intersect with each other on the lower surface of the movable plate in an approximately crossed shape. The rotating shaft 32 is bent downward in a crank shape, and is arranged so that the rotating shaft 33 does not interfere therewith. Each of the rotating shafts 32 and 33 is provided with two cam bodies 34 and 35 so that the outer peripheral diameters are changed in opposite phases from each other at positions equidistant from the intersection. Four small protrusions 36 are provided on the upper surface side of the movable plate 17a that is tilted by being pushed up by these cam bodies 34 and 35, so that the stimulus to the finger due to the biaxial tilt can be clearly identified.

  According to the fourth embodiment, using one movable plate 17a, two-dimensional movement information can be transmitted to one finger placed on the movable plate 17a via the tilting state of the movable plate 17a. Since there is only one movable plate, it is easy and easy for the operator to grasp the two-dimensional movement information.

[Fifth embodiment]
In the fifth embodiment, as shown in FIG. 10, two marker light sources 43 provided on the upper portion of the monitor 40 are photographed using an optical sensor 42 provided on the input device body 10, and the photographed images are taken. The position, amount of movement, and amount of rotation of the input device main body 10 are to be calculated based on the position of the light source 43 shown inside. Based on the information on the position, movement, and rotation of the input device main body 10 calculated in this way, the tactile stimulation means drives the touch element (in this case, the movable plate 17a). For example, when the input device body 10 is moved left and right, the movable plate 17a tilts left and right, and when it is moved up and down, it tilts up and down. When a device can be moved freely in a three-dimensional space, the degree of freedom is too high to accurately grasp the movement of the device body. There is a case. However, if there is tactile stimulus information given through the tactile device, the device body can be accurately moved using this information as a clue.

  In FIG. 10, a menu selection screen 41 is displayed on the monitor screen 41. When the input device body 10 is moved up and down or left and right, the tactile stimulus is generated as described above, and the item displayed so that it can be distinguished from other items by highlighting it while accurately grasping the moving direction and the moving amount. (In the figure, the highlighted item is displayed with a bold line to distinguish it from the other items).

  According to the fifth embodiment, even if the operator does not stare at the monitor screen, the input device body 10 moves unexpectedly and the highlight position is changed by taking care that the tactile stimulus is kept constant. The problem of changing can be prevented, and even when the highlight moves, for example, if it moves up and down when moving to the left and right, it can be detected through tactile stimulation and the device body can be moved to correct the displacement. Allows you to select menu items accurately and quickly. Also, as will be discussed later, tactile stimuli can be generated so that it can be seen that the highlight crosses the menu item boundary.

[Sixth embodiment and GUI program]
Since the displacement notification device 1 and the GUI program of the present invention exhibit their characteristics while being related to each other, here, the GUI program will be described together with the sixth embodiment of the displacement notification device. In one embodiment of the GUI program, for example, a menu selection screen 41 as shown in FIGS. 11A, 11B, and 11C is displayed on the monitor screen. Here, any one of the first to fifth embodiments may be used as the displacement notification device, but a gyro sensor or the like that detects the position and direction, movement, and rotation of the input device body 10 is detected. Based on the amount of movement and the amount of rotation of the device main body, the tactile stimulation means drives the touch to stimulate the fingers. The feature of the sixth embodiment and the GUI program is that a special relationship is defined between the movement amount and rotation amount of the device body and the driving amount of the touch element. First, this is a problem that has also occurred in a conventional pointing device such as a mouse. However, in a three-dimensional pointing device that moves in the air, an operation error due to the following causes occurs more remarkably. In other words, even if you intend to select the menu item by moving the cursor to one of the menu items, the cursor position will be displaced during another operation such as pressing the click button. It will fall out of the item and an incorrect input will occur. In the air, the position of the device body becomes unstable, and the operator cannot obtain any tactile cues for grasping the position of the device body. On the other hand, with conventional mice and touchpads, a part of the fingers touches the surface of the reference desk or PC housing, and the mouse can tactilely grasp the finger position based on them. did it.

  In the present invention, in order to solve the above problem, the deviation notification device is configured with the goal of giving a virtual reference point to a finger in the air. This virtual reference point does not necessarily need to be actually stationary in the three-dimensional space. In practice, it is difficult to keep the reference point constant because of sensor errors and the limit of the movable range of the touch sensor, but the reference point given as the position of the touch sensor is relatively stable even if it fluctuates. If it fluctuates slowly and continuously, the person will recognize changes in the position and angle of the device body with reference to the position of the touching finger.

  The following method is used to reduce the error in the menu operation described above by using the virtual reference point given by the tentacle. In the following, moving the cursor means changing the highlighted item.

  (1) Using the position of the device body when the cursor is positioned at the center of the menu item area as the reference point (reference position), the device is in the reference point and the neutral is informed. To do.

  (2) When the device body is displaced from this reference point and the cursor is displaced from the center of the menu item area, the touch panel is moved to notify the displacement, and the operator moves the device body so that the touch panel returns to the neutral state. Thus, this displacement is corrected.

  (3) When the cursor is displaced from the center of the menu item area toward the periphery, it is linked to the displacement in real time so that the tentacles are continuously moved so that the cursor is positioned at the boundary between the menu items. Before arriving, the operator can grasp the displacement and correct it.

  (4) To allow the operator to intuitively grasp the direction and amount of displacement, a virtual reference object that is stationary in the external space is assumed and the finger touching it is moved. The tactile stimuli that you will sometimes feel are generated by moving the tentacles. For this purpose, for example, when the position of the device main body is displaced from the reference point, it is preferable to move the touch element in the direction opposite to the displacement direction in conjunction with the displacement. However, because the sensor error and the limit of the movable range of the toucher are difficult to move the toucher so that it actually stops in the space, the operator corrects the position of the device body as described above. However, there is no guarantee that the position can be maintained at a reference point in the outside world. Therefore, the information to be input is determined based on the relative position of the device body with respect to the touch, not based on the absolute position of the device body in the external world.

  (5) If the menu item area has a certain size, even if the device body is displaced and the cursor moves slightly, the cursor does not protrude from the menu item area. By correcting the position of the device body, it is expected that the cursor can continue to be maintained within the menu item area. If the cursor still protrudes from the menu item area and enters the next menu item area, it is determined that the operator has moved the cursor to intentionally select the next menu item, and the cursor is moved to the next menu item. Move to the item and highlight it.

  (6) As described above, when moving the cursor from one menu item area to the adjacent menu item area, it is not necessary to correct the displacement. It is better to notify the operator that the boundary has been passed. After passing through the boundary, the cursor position is automatically brought to the center of the area of the next menu item, and the toucher is preferably returned to the neutral state. Therefore, before reaching the boundary of the menu item area because the displacement is very small, the tentacle moves continuously in the opposite direction to the movement direction of the device body in conjunction with the displacement and passes through the boundary between the menu item areas. In addition, it is better to move it abruptly to return to the neutral state. Then, the position of the device main body is newly set as the standard position, and the position of the device main body and the movement of the toucher are associated with the displacement from the standard position by the above method.

  FIG. 11E is a graph showing the movement of the toucher with respect to the movement of the device body described above. FIG. 11E is a graph showing the relationship between the amount of movement of the input device body 10 (indicator) and the amount of increase from the reference position P of the small protrusions 36a to 36d (finger stimulators). As shown in FIG. 11 (a), when one of the menu items is highlighted, the frame of the menu item is displayed thick to distinguish it from the other items. If the contactor is in a neutral state, the device body is in the center of the selected area of the item (ie, the standard position), and even if the device position is slightly displaced, the item does not protrude from the selected area. Continue to be selected stably. If the device body is displaced from the standard position here, the toucher moves in a size that is linked to the displacement, so if you move the device body so that it returns to the neutral state when the toucher moves, the original menu You can continue to select items. When the operator wants to move the highlight to the next menu item, the operator ignores the notification of the displacement due to the movement of the touch element and continues to move the device body without correcting the displacement. Then, when the amount of displacement exceeds a certain threshold, the tactile device quickly returns to the neutral state, the device position at that time is set to the center (reference position) of the selection area of the adjacent menu item, and the highlight is also adjacent. Move to the menu item. FIG. 11B shows the state where the device body is moved in the positive direction of the y-axis, and when the displacement exceeds the threshold value, the highlight moves to the upper menu item, and FIG. 11C shows the device body. Is moved in the positive direction of the x-axis, and the highlight moves to the menu item on the right when the displacement exceeds a threshold.

  If bi-directional communication is performed between the input device main body 10 (or a displacement notification device mounted thereon) and the monitor 40 (or a GUI program mounted thereon), whether the displacement exceeds a threshold value or not is determined. Is determined by a calculation unit built in the monitor side, and the result is transmitted as a control signal to the input device body 10 to operate the driving means. Yes.

  If only one-way communication from the input device main body 10 (or the displacement notification device mounted thereon) to the monitor 40 (or the GUI program mounted thereon) is performed, it is determined whether the displacement has exceeded a threshold value. Is calculated by the calculation unit on the input device main body side, and based on the result, the actuator driving means is operated on the input device main body side, and the control signal is transmitted to the monitor side, and the displacement exceeds the threshold value. The cursor moves to the adjacent menu item in the direction corresponding to the displacement direction.

  In the sixth embodiment and the GUI program configured as described above, the input device body 10 is moved from (1) → (2) → (3) to select the menu 12 on the menu selection screen 41 as shown in FIG. An example of the case will be described. When the input device body 10 is moved upward from (1) to (2), the movable plate 17a tilts and the finger stimulator 36a rises. After passing the boundary, as shown by a point P in FIG. 11 (e), the movable plate 17a drops to the reference position and the movable plate 17a becomes flat. As a result, the operator can know that he has entered the area of the menu 11 as shown in FIG. Next, when the input device body 10 is moved rightward from (2) to (3), the finger stimulator 36b rises, and after passing through the boundary, the movable plate 17a drops to the reference position P, The target menu 12 is selected as shown in FIG. Here, when the tact switch 14 is turned on, the contents of the menu 12 are executed.

  As described above, since it is desirable to determine the amount of driving of the stylus with respect to the displacement of the device body, the device body (for example, an indicator) is continuously moved in one direction so as to cross a plurality of menu items. Sometimes, the driving amount of the touch element changes with a sawtooth waveform rising to the right as shown in FIG. When the speed of the device body is high, the period of this sawtooth waveform is reduced, and the amplitude is reduced as necessary. Even if the amount of movement of the device body is small, the cursor (highlight) crosses many menu items, Be able to reach distant menu items. Conversely, when the speed of the device body is slow, the cursor cannot reach the next menu item unless the period of the sawtooth waveform is increased and the amount of movement of the device body is increased. To be stable.

  The graph showing the relationship between the two may be a sawtooth waveform having a downward slope as shown in FIG. 12A and a rectangular waveform in FIG. 12C. However, it is inappropriate to use these when selecting a menu item. Yes, it is used to distinguish and give tactile information for reporting other situations.

[Seventh embodiment and GUI program]
In the seventh embodiment, a sensor for detecting the movement of the device main body in the z-axis direction and a contact for notifying the displacement of the device main body in the z-axis direction are introduced in the sixth embodiment. Here, the touch element for notifying the displacement in the z and y-axis directions is omitted, but only the function of the touch element for notifying the displacement in the z-axis direction will be described here, so that the monitor screen 41 as shown in FIG. When a plurality of menu sheets 50 to 52 are displayed, a function for selecting a sheet by moving the input device body 10 in the depth (Z-axis direction) is added. That is, the menu sheet displayed on the monitor screen 41 is arranged so as to overlap in the depth direction, and the input device body 10 is not only in the two-dimensional direction of the monitor screen but also in the depth direction (Z-axis direction) (1) → ( The menu sheet 50 to 52 can be selected by moving from 2) to (3).

  When the menu sheet is switched, the rising amounts of the touch elements 36a and 36c change as shown in the graph of FIG. 13E with respect to the displacement amount of the device main body (indicator) in the z direction. That is, when the device main body is at the reference position for selecting a specific sheet with respect to the z direction, the contactor is in a neutral state. However, if the device main body is displaced in the z-axis direction with respect to this reference position, the displacement is On the other hand, as shown in FIG. 13 (e), the movable plate is tilted continuously in association with the displacement so that the side corresponding to the moving direction of the device body of the toucher is raised. While the displacement is small, the operator can correct the device position to the reference position by using the tactile information as a clue. On the other hand, when the operator disregards the notification given by the toucher and further moves the device body and the displacement exceeds a certain threshold value, the toucher moves suddenly and becomes neutral as shown in FIG. Return to the state. The movement of the tentacle is perceived by the operator as a clear stimulus, and becomes notification information notifying that the menu sheet has been switched.

  When the menu sheet is switched, the position on the z-axis of the device body at that time is newly set as a reference position for the sheet. FIG. 13 shows an example in which the input device body 10 is moved in the depth (Z-axis) direction from (1) → (2) → (3). In (3), the target sheet 52 is selected, and the selection is confirmed by turning on the tact switch. Even after the menu sheet 52 is confirmed, if the device main body is moved in the x and y axis directions to select a menu item, the device main body is erroneously moved in the z axis direction if there is no tactile clue as described above. May also move and the menu sheet may switch to another sheet. However, if there is the above-mentioned tactile stimulus, the position of the device body is corrected so as to keep the toucher in a neutral state, and if it is moved in the x and y axis directions, the cursor is moved while keeping the selected sheet constant. To select menu items.

  As mentioned above, although the present invention has been described in detail according to each embodiment, the specific configuration is not limited to these embodiments, and the scope of the present invention can be changed even if there is a design change or the like without departing from the gist of the present invention. Is included.

  For example, by covering the entire input device main body 1 of each embodiment with silicon rubber, the input device main body 10 can be easily gripped, which is effective in protecting against external impact force, water, and dust.

  The dial as the volume adjustment mechanism in the first embodiment may be provided in the second to seventh embodiments.

  Further, in each embodiment, the finger is stimulated by tilting the stick or the movable plate, but the hand or the finger is stimulated not by tilting but by another form of movement instead of tilting. Also good.

  Moreover, you may make it the aspect which vibrates a stick, a movable plate, and a finger stimulator, or makes it generate | occur | produces a sound, or adds and uses together.

  In the above description, an input device having a function of transmitting data to a computer is described as an example of a displacement notification device. However, the present invention is not limited to this, and an imaging device such as a camera, a nuclear reactor, a medical machine, and a manufacturing facility. It can be applied as a control means for performing remote operation in robot operation or the like.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an external view illustrating a state where an input device (displacement notification device) is gripped according to a first embodiment of the present invention. It is the side view which showed the structure of the input device main body typically. It is an operation explanatory view explaining the state where the input device main body is moved and operated in the left-right direction. It is an operation explanatory view explaining the state where the input device body is moved up and down. According to the second embodiment, (a) is a side view and (b) is a plan view. It is a figure explaining the effect | action of 2nd embodiment. It is related with 3rd embodiment, (a) is a side view, (b) is a top view. It is related with 4th embodiment, (a) is a side view, (b) is a top view. It is effect | action explanatory drawing of 4th embodiment. It is an external view of 5th embodiment. In the sixth embodiment, (a) to (c) are menu selection screens, (d) is an external view of an input device body, and (e) is a graph showing the characteristics of a control signal. It is a graph which shows the modification of a control signal characteristic. According to the seventh embodiment, (a) to (b) are menu selection sheet selection screens, (d) is an external view of the input device body, and (e) is a graph of control signal characteristics. FIG. 14A is an external view of an input device provided with three tentacles for stimulating fingers to notify the movement of the device main body in the three-axis directions, and FIG. 14A illustrates a state in which the input device is gripped. FIG. 14B is a side view schematically showing the configuration of the input device main body.

Explanation of symbols

1 Input device body (displacement notification device body)
2 Gyro sensor 3 Driving means 4 Tactile stimulation means 4a Index finger stick (tactile stimulation means)
4b Thumb stick (tactile stimulation means)
4c Middle finger or ring finger stick (tactile stimulation means)
5 Dial (Volume adjustment means)
10 Input device body (displacement notification device body)
11 Electronic Circuit Board 12 Gyro Sensor 13 Battery 14 Tact Switch 15 Infrared LED
DESCRIPTION OF SYMBOLS 16 Driving means 16a Rotating shaft 17 Tactile stimulation means 17a Movable plate 17b Rubber film 17c Small protrusion 18 Tilt means 18a Cam part 18b Swing arm 18c Roller 19 Capacitance sensor 21 Movement amount detection sensor 30 Tilt means 31 Drive means 32, 33 Rotating shaft 34, 35 Cam body 36 Small protrusion 36a, 36b, 36c, 36d Finger stimulator 40 Monitor 41 Menu selection screen 42 Optical sensor 43 Light source for marker 44 Menu 50-52 Menu selection sheet H Hand (finger)
La Horizontal movement reference line Lb Vertical movement reference line

Claims (18)

  A displacement notification device that is operated while being gripped by an operator and detects a position in a two-dimensional or three-dimensional space, the displacement detecting device detecting that the device body is displaced from a reference index virtually provided in the outside A detection sensor, a driving unit that operates based on detection information of the displacement detection sensor, and a toucher that touches an operator's finger by the driving unit are moved to operate a displacement direction or a displacement amount of the device with respect to the reference index. Tactile stimulation means for informing the user, and the driving means operates the displacement direction or the displacement amount of the apparatus main body by moving the tentacle while interlocking or synchronizing with the movement of the displacement of the apparatus main body. Displacement informing device characterized by informing through the sense of touch of a person.   The displacement notification device according to claim 1, wherein a switch or a touch sensor that detects contact of a finger is provided on the touch element, and the displacement notification device functions only while the finger is touching the touch element. .   The tactile sense stimulating unit is configured to perform the above operation until the displacement amount of the apparatus main body reaches a predetermined threshold value or the cursor displayed on the monitor screen corresponding to the apparatus main body or the main body position reaches a predetermined boundary position. The operator continuously moves the tentacles in conjunction with the movement of the displacement of the apparatus main body, notifies the operator of a clue for correcting a slight deviation from the reference index, and exceeds the threshold value or the boundary position. The contactor is discontinuously returned to the neutral position, the accumulated amount of displacement is reset, and the apparatus main body position immediately after exceeding the threshold is newly set as the reference index. The displacement notification device according to claim 1 or 2.   The tentacle moves in the direction opposite to each displacement direction when the device main body is displaced left, right, front, back, or up and down in a three-dimensional space, and the finger rests on the reference index in the outside world. 4. The device according to claim 1, wherein the displacement direction and the displacement amount of the device are informed so as to cause an operator to feel an illusion as if the touching touching device is touched. 5. The displacement notification device described.   The tactile stimulation means is waterproof and dustproof by using a tentacle supported by a plastic film that is freely deformed in the apparatus body and sealing with the plastic film. Item 5. The displacement notification device according to any one of Items 1 to 4.   The drive means uses a cam in which the distance from the rotation axis to the outer periphery is unevenly formed according to the rotation angle, and drives the tentacles to change the substantial outer radius at a predetermined rotation angle. The displacement notification device according to any one of claims 3 to 5, wherein the tentacles are returned to the neutral position by relatively rapidly changing compared to the rotation angle.   The drive means includes a mechanism for moving one tactile element in two orthogonal directions, and notifies the tactile sense of the two-dimensional displacement direction and the displacement amount of the apparatus by the two-dimensional movement of one tactile element. The displacement notification device according to any one of claims 1 to 6, wherein the displacement notification device is characterized.   The tactile sense stimulating means includes a stick-like touch element formed so that a free end portion provided so as to swing freely touches a hand or a belly of a finger holding the device main body, and the device main body is The displacement notifying device according to any one of claims 1 to 7, wherein when moving from a reference index, the tentacle is tilted in the direction of the reference index.   The displacement notification device according to claim 1, wherein an optical position detection sensor is used as the sensor that detects that the device main body is displaced from a reference index.   As the displacement detection sensor for detecting that the device main body is displaced from a reference index, a camera for photographing the movement of the device main body or a camera mounted on the device main body for photographing the movement of the outside world is used. The displacement notification device according to any one of claims 1 to 9.   10. The gyroscope or acceleration sensor mounted on the apparatus main body is used as the displacement detection sensor that detects that the apparatus main body is displaced from a reference index. 10. Displacement notification device.   When the driving means continuously moves the tentacle so as to synchronize with or synchronize with the displacement of the apparatus main body, a correspondence relationship that determines the amount of movement of the tentacle with respect to the displacement generated in the apparatus main body is established. The displacement notification device according to any one of claims 1 to 11, further comprising a sensitivity changing unit for changing or a threshold adjusting unit capable of adjusting the threshold.   The predetermined information 1 is input by an operation of displacing the apparatus main body in a predetermined direction 1, and the predetermined information 2 is input and changed by an operation of displacing the apparatus main body in a predetermined direction 2 different from the direction 1. A displacement notification device used in the menu item selection method, wherein the device main body erroneously moves the device main body in the direction 1 in an attempt to input the information 1. As a means for preventing inconvenience that unintended information 2 is input by moving in the direction 2, the displacement detection sensor detects the displacement of the apparatus body in the direction 2, and the tactile stimulation means Until the amount of displacement of the device main body in the direction 2 reaches a predetermined threshold, the finger is continuously moved so as to be interlocked with the movement of the displacement of the device main body at any time, and the reference finger The operator is notified of a clue for correcting a slight deviation from the position, and when the threshold value is exceeded, the contactor is discontinuously returned to the neutral position, and information 2 is input and further accumulated. The displacement notification device according to any one of claims 3 to 12, wherein the displacement amount is reset, and the device position immediately after the threshold is exceeded is set as a new reference index.   The gyroscope, the acceleration sensor, or the image sensor introduced for blur compensation of a photographed image of a camera separately mounted on the photographing apparatus main body is shared as the displacement detection sensor. The displacement notification device according to any one of the above.   The amplitude or period of the movement of the tentacle driven by the finger stimulating means is changed based on the displacement speed of the apparatus main body detected by the displacement detection sensor. The displacement notification device described. A GUI (graphical user interface) program for a computer that operates based on data output from a displacement notification device,
Menu display processing for displaying a menu on the monitor screen of the display device;
Highlight control processing that highlights a predetermined item in the menu and displays that item separately from other items,
A communication process for communicating with the displacement notification device;
Items for moving the cursor displayed on the monitor screen or highlighting in the menu based on the information on the displacement direction and displacement amount of the displacement notification device main body received by the communication process in the up / down, left / right or front / rear direction And processing to change
GUI program for causing a computer to execute. The GUI program according to claim 16, wherein the communication process is a one-way communication process for receiving information sent from the displacement notification device,
When an event occurs in which the displacement amount of the displacement notification device main body exceeds a predetermined threshold value, information on the occurrence of the event and the displacement direction of the displacement notification device main body sent from the displacement notification device is received. A program for causing a computer to execute a process of changing an item to be highlighted in a menu based on the received information to another item in a direction corresponding to a displacement direction of the displacement notification device main body. The GUI program according to claim 16, wherein the communication process performs bi-directional communication with the displacement notification device as needed,
Information on the displacement of the displacement notification device main body sent from the displacement notification device is received at any time by the communication process, and when the information on the displacement or the accumulated information satisfies a predetermined condition, a predetermined function is executed. By the communication processing, the driving means of the displacement notification device is operated to return the tentacles to the neutral position discontinuously, and the accumulated displacement information is reset, and the device immediately after the threshold is exceeded. A GUI program for causing a computer to execute processing for transmitting a command for setting a position to a new reference index to the displacement notification device.

Images