JP2013030143A - Operation support device, mobile body information communication terminal, imaging device, and information processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation support device capable of supporting an operator to intuitively modify a posture of an operation object in a desired posture, and a mobile body information communication terminal, an imaging device, and an information processor having the operation support device.SOLUTION: An operation support device comprises: posture information detection means for detecting posture information of an operation object; information conversion means for converting a difference between the posture information detected by the posture information detection means and preset target posture information into tactile information; and tactile information transmission means for transmitting the tactile information converted by the information conversion means to an operation body operating the operation object.

Description

  The present invention relates to an operation support device that supports an operation performed by an operator on an operation target, and a mobile information communication terminal, an imaging device, and an information processing device that include the operation support device.

  In recent years, in mobile terminals such as mobile phones and digital cameras that are objects to be held and operated by an operator, information that supports operations performed by the operator on the mobile terminal is presented from the mobile terminal to the operator. Is known (Patent Document 1, etc.). For example, when the operator performs the operation of presenting the attitude information of the portable terminal to the operator and correcting the attitude of the portable terminal to the desired attitude, the inclination with respect to the desired attitude of the portable terminal is used as the attitude information of the portable terminal. There are those that display the degree on the screen and those that emit a sound to inform the operator when the posture of the mobile terminal becomes a desired posture. These are operations that convert the attitude information of the mobile terminal into auditory or visual information of the operator's senses, present it to the operator from the mobile terminal, and correct the attitude of the mobile terminal by the operator to a desired attitude Is supporting.

  However, when the operator cannot see the screen of the mobile device or the sound emitted from the mobile device is drowned out by the surrounding noise, the operator grasps the posture of the mobile device based on auditory information and visual information. Difficult to do. Therefore, there arises a problem that the operator cannot intuitively correct the portable terminal to a desired posture based on visual information or auditory information presented to the operator from the portable terminal.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an operation support apparatus that can assist an operator to intuitively correct an operation target to a desired posture, and an operation support apparatus thereof. It is to provide a mobile information communication terminal, a photographing apparatus, and an information processing apparatus.

  In order to achieve the above-described object, the invention of claim 1 is directed to posture information detection means for detecting posture information of an operation target, the posture information detected by the posture information detection means, and preset target posture information. And a tactile information transmission unit that transmits the tactile information converted by the information conversion unit to an operating body that operates the operation target. To do.

  In the present invention, the posture information of the operation target detected by the posture information detecting means is converted into tactile information by the information converting means, and the tactile information is presented to the operating body by the tactile information transmitting means. Thus, the operation object's posture information is presented to the operating body in contact with the operation object as tactile information among the operator's senses, and the operation by the operator to correct the operation object's posture to a desired posture is performed. Can help. Therefore, the operator is more intuitive based on the tactile information presented to the operating body from the tactile information transmission means than when the posture information of the operation target is presented using visual or auditory information among the senses of the operator. In addition, an operation for correcting the posture of the operation target to a predetermined posture can be performed.

  As described above, according to the present invention, there is an excellent effect that an operator can support an operation for intuitively correcting an operation target object to a desired posture.

The block diagram concerning control of an operation assistance apparatus. The figure which shows the example of mounting of the tactile information transmission means to various apparatuses. The figure which shows an example of the internal structure of an operation target object. The figure which shows an example of the internal structure of a controller. The figure which shows the process example inside a processor. (A) Explanatory drawing of posture of digital camera, (b) Explanatory drawing of posture of smartphone, (c) Explanatory drawing of posture of video camera. The figure which shows the whole flow which concerns on the operation assistance control of the operation target object. The figure which shows the process example inside a processor. The figure which shows the whole flow which concerns on the operation assistance control of the operation target object. The figure which shows the process example inside a processor. The figure which shows the whole flow which concerns on the operation assistance control of the operation target object. (A) The figure which shows the example of the haptic information conversion function which weakened the sensitivity, (b) The figure which shows the example of the haptic information conversion function which weakened the sensitivity, (c) The example of the haptic information conversion function which strengthened the sensitivity FIG. 4D is a diagram illustrating an example of a haptic information conversion function with increased sensitivity, and FIG. 5E is a diagram illustrating an example of a haptic information conversion function with decreased sensitivity. Explanatory drawing of the video camera in a time-series target position and orientation. (A) The figure which shows the example of the tactile information transmission means which performs tactile information transmission by the force of a normal direction, (b) The figure which shows the example of the tactile information transmission means which performs the tactile information transmission by the force of a tangential direction, (c) The figure which shows the example of the tactile information transmission means which performs tactile information transmission by the force of a tangential direction, (d) The figure which shows the example of the tactile information transmission means which performs the tactile information transmission by the force of a tangential direction. The figure which shows the example of the tactile information transmission means which performs tactile information transmission by a vibration stimulus. The figure which shows the example of the tactile information transmission means which performs tactile information transmission by electrical stimulation. (A) Explanatory diagram when the contact part of the tactile information transmission means is a part of the touch panel of the smartphone, (b) Explanatory diagram when the contact part of the tactile information transmission means is a part of the touch panel of the pad type personal computer . Sectional drawing which shows the internal cross-section of a touchscreen. Explanatory drawing when the movable part of a tactile information transmission means is a shutter button of a digital camera. Sectional drawing which shows the internal cross-section of the digital camera of the periphery of the shutter button which one part protruded outside from the housing | casing.

FIG. 1 is a block diagram related to the control of the operation support apparatus.
The operation target 101 is a portable terminal that can be held and operated by the operator 107. The posture information detection unit 102 has a function of detecting posture information of the operation target object 101 and outputting posture information. The information conversion unit 103 has a function of converting the difference between the posture information detected by the posture information detection unit 102 and the preset target posture information 100 into the tactile information 106. The tactile information transmission unit 104 has a function of transmitting the tactile information 106 converted by the information conversion unit 103 to the fingertip 109 of the operator 107 to act.

  Examples of the operation object 101 include a digital camera that is a portable imaging device, a mobile phone or smartphone that is a mobile information communication terminal, and a laptop personal computer or a pad personal that is a portable information processing device. A computer etc. are mentioned.

  Based on the tactile information given to the fingertip 109, the operator 107 performs an operation output 105 on the operation object 101 supported and held by the upper limb 108.

  Note that the fingertip 109 in the figure is a part where the tactile receptor of the hand of the operator 107 exists, and may be a palm or the like. The upper limb 108 may be any part of the body as long as it is a part that can be moved further, such as the arm, elbow, shoulder, wrist, and finger joint of the operator 107.

  For example, an inertial sensor can be used as the posture information detection unit 102. Thereby, the movement of the operation target object 101 can be detected in a state where the operation target object 101 is not grounded and unconstrained independently of the environment.

  Inertial sensors include two-axis sensors for detecting accelerations in the X-axis direction and Y-axis direction orthogonal to each other in a parallel plane, and in the X-axis direction and Y-axis direction orthogonal to each other in a parallel plane. An acceleration sensor such as a three-axis sensor that detects the heading acceleration and the heading acceleration in the Z-axis direction that is the normal direction of the surface can be used. By using the acceleration sensor, the posture of the operation target object 101 can be calculated from the components on each axis of gravity acceleration.

  Further, as the inertial sensor, an angular velocity sensor represented by a gyro sensor may be used. Thereby, the attitude of the operation target object 101 can be calculated by the reference attitude and the time integration.

  As another example of the posture information detecting means 102, the strength of a magnetic field on a two-dimensional coordinate composed of an X axis and a Y axis is detected in an environment where a magnetic field exists, or 3 composed of an X axis, a Y axis, and a Z axis. You may use the magnetic sensor which detects the intensity | strength of the magnetic field on a dimensional coordinate. In this case, a magnetic marker or the like may be installed in the environment, or a geomagnetic sensor using geomagnetism may be used. Thereby, the attitude | position of the operation target object 101 is computable from the intensity | strength of the magnetic field of each axis | shaft detected by the magnetic sensor.

  As yet another example of the posture information detection unit 102, an image sensor may be used. The attitude of the operation target 101 can be calculated by installing an image sensor on the operation target 101 or the environment and performing image processing on the obtained image data.

  As shown in FIG. 2, the tactile information transmission means 104 is in contact with the fingertip 109 of the operator 107 of various portable terminals (mobile phone 30, smartphone 40, digital camera 50, laptop personal computer 60, pad personal computer 70). It is mounted in a predetermined space (indicated by hatching in the figure) arranged in a modular manner or embedded.

FIG. 3 shows an example of the internal configuration of the operation target 101.
The posture information detection unit 102 is a posture sensor 202, the information conversion unit 103 is a controller 203, and the tactile information transmission unit 104 is an actuator 204.

  FIG. 4 is a diagram illustrating an example of the internal configuration of the controller 203. As shown in FIG. 4, in the controller 203, an A / D converter 301 that digitizes sensor signals, a processor 302 that performs information processing, a D / A converter 303 that analogizes calculation results, and an actuator 204 are driven. An actuator driver 304 that is a circuit is provided.

  The A / D conversion unit 301 has a function of digitally converting the analog output signal of the attitude sensor 202. The processor 302 has a function of performing a calculation process of calculating a difference between the converted sensor signal and preset target posture information and converting it into tactile information. The D / A conversion unit 303 has a function of converting tactile information into an analog signal. The actuator driver 304 has a function of driving the actuator 204 using the analog signal as a command value.

  Processing performed inside the processor 302 will be described with reference to FIG. The processor 302 is provided with an attitude calculation unit 401 and a tactile information conversion unit 402. The posture calculation unit 401 has a function of calculating posture information (an angle such as roll, pitch, yaw) from the sensor signal of the posture sensor 202. The tactile information conversion unit 402 has a function of converting a difference between the posture information of the operation target 101 calculated by the posture calculation unit 401 and the target posture information 100 into tactile information.

  6A is an explanatory view of the posture of the digital camera 50, FIG. 6B is an explanatory view of the posture of the smartphone 40, and FIG. 6B is an explanatory view of the posture of the video camera 500. The posture calculation unit 401 calculates posture information of the operation target from the sensor signal digitized by the A / D conversion unit 301. As the posture information of the operation target to be calculated, for example, the digital camera 50 that is the operation target 101 from three orthogonal physical quantities as shown in FIG. 6 (a), FIG. 6 (b), or FIG. 6 (c). And the roll angle, pitch angle, yaw angle, etc. of the smartphone 40 or the video camera 500.

The posture information of the operation target object 101 can be obtained from at least one of the roll angle, the pitch angle, and the yaw angle, but here, when the roll angle is focused as the posture information of the operation target object 101 Will be described. The roll angle calculated by the posture calculation unit 401 is θ _IN , the target posture is θ ₀ , the difference is Δθ = θ _IN −θ ₀ , and the tactile information conversion unit 402 performs a predetermined tactile information conversion function f (Δθ) with respect to Δθ. ) To obtain the tactile information x. Then, a command is issued from the processor 302 to the actuator 204 based on the tactile information x.

  For example, when considering a leveling operation in photographing with the digital camera 50, Δθ = 0 corresponds to a state in which the posture of the digital camera 50 is horizontal.

FIG. 7 shows an overall flow relating to the operation support control of the operation object 101.
When the operation support control is started, first, a target posture θ ₀ of the operation target 101 is set (S1). Next, the posture of the operation target 101 is detected by the posture sensor 202, and the analog output signal output from the posture sensor 202 is digitally converted by the A / D conversion unit 301 (S2). From the sensor signal digitized by the A / D conversion unit 301 in this way, the posture θ _IN of the operation target 101 is calculated by the posture calculation unit 401 (S3). Then, the posture calculated by the posture calculation unit 401 is θ _IN , the target posture is θ ₀ , and the difference is Δθ = θ _IN −θ ₀ , and the tactile information conversion unit 402 performs a predetermined tactile information conversion function f ( The tactile information x is calculated by performing transformation using (Δθ) (S4). The D / A conversion is executed by the D / A conversion unit 303 on the tactile information x calculated in this way, and it is sent to the actuator driver 304 as a command value, and moves in advance according to the command value. The actuator 204 is driven (S5). When the operation support is continuously performed (YES in S6), the process returns to the reading by the attitude sensor 202 and executes a series of control again. On the other hand, when the operation support is not continuously performed (NO in S6), the series of control is finished and the operation support is ended.

  By executing such control, it is possible to perform operation support so that the operator 107 can intuitively move the operation target 101 toward a desired posture.

Next, a case where the posture information is multidimensional including position information will be described.
Position 3 degrees of freedom in three-dimensional space, the position and orientation information of the six degrees of freedom pose 3 degrees of ^{freedom, v} theta = represented by a vector as [q1, q2, q3, q4 , q5, q6]. In the present specification, for the purpose of preparing the specification, the Berkle symbol represented by adding “→” above “θ” is represented by the subscript “ ^v ”.

The distance (scalar) between the two position / orientation vectors ^v θ ₁ and ^v θ ₂ is expressed as d ( ^v θ ₁ , ^v θ ₂ ). For example, a standard Euclidean distance or Mahalanobis distance may be used as the distance.

8, the position and orientation information calculated by the position and orientation information calculation unit 401 and ^v theta _IN, the target position and orientation information with ^v theta _0, error Δθ = d ^{(v θ} _IN from the target by the error calculation unit 403 from those , ^v θ ₀₎ is calculated. Then, the tactile information conversion unit 402 performs conversion using a predetermined tactile information conversion function f (Δθ) on Δθ to obtain tactile information x. Then, a command is issued from the processor 302 to the actuator 204 based on the tactile information x.

FIG. 9 shows an overall flow relating to the operation support control of the operation object 101.
When the operation support control is started, first, a target position / orientation ^v θ ₀ of the operation object 101 is set (S1). Next, the posture of the operation target 101 is detected by the position / orientation sensor 202, and the analog output signal output from the position / orientation sensor 202 is digitally converted by the A / D conversion unit 301 (S2). The position / orientation calculation unit 401 calculates the position / orientation ^v θ _IN of the operation target 101 from the sensor signal digitized by the A / D conversion unit 301 (S3). Then, the position / orientation calculated by the position / orientation calculation unit 401 is ^v θ _IN , the target position / orientation is ^v θ ₀ , and the error from the target calculated by the error calculation unit 403 is Δθ = d ( ^v θ _IN , ^v θ ₀ The tactile information conversion unit 402 converts the Δθ using a predetermined tactile information conversion function f (Δθ) to calculate the tactile information x (S4). The D / A conversion is executed by the D / A conversion unit 303 on the tactile information x calculated in this way, and it is sent to the actuator driver 304 as a command value, and moves in advance according to the command value. The actuator 204 is driven (S5). If the operation support is to be continued (YES in S6), the process returns to reading by the position / orientation sensor 202 and the series of control is executed again. On the other hand, when the operation support is not continuously performed (NO in S6), the series of control is finished and the operation support is ended.

Next, a case where the target position / orientation is set as time series data will be described.
In the position and orientation information, the position and orientation at a certain time t is expressed as follows: ^v θ (t) = [q ₁ (t), q ₂ (t), q ₃ (t), q ₄ (t), q ₅ (t) , Q ₆ (t)], and a vector including time t.

In FIG. 10, ^v θ ₀ (t _k ) is stored in the target position / orientation information storage unit 405 as target position / orientation information at each time t _k (k = 0, 1, 2,...) The target position and orientation information reading unit 404 reads ^v θ ₀ (t) of the position and orientation information 401 at the current time t from the target position and orientation information storage unit 405. Position the position and orientation information calculated by the attitude information computing section 401 and ^v theta _IN, and the target position and orientation information read ^v theta _{0 (t),} the error [Delta] [theta] = d from the target by the error calculation unit 403 from those ( ^v θ _IN , ^v θ ₀ (t)) are calculated. The haptic information conversion unit 402 performs conversion using Δt on a predetermined haptic information conversion function f (Δθ) to obtain haptic information x. Then, a command is issued from the processor 302 to the actuator 204 based on the tactile information x. As an example of the target position / orientation information storage unit 405, a table in which a component of ^v θ ₀ (t) for each time step is described is shown in Table 1.

FIG. 11 shows an overall flow relating to the operation support control of the operation object 101.
When starting the control operation support, first sets an initial value t ₀ to time t (S1). Next, the target position and orientation information reading unit 404 reads the target position and orientation ^v θ ₀ (t) of the operation target object 101 at the time t from the target position and orientation information storage unit 405 (S2). Next, the posture of the operation target 101 is detected by the position / orientation sensor 202, and the analog output signal output from the position / orientation sensor 202 is digitally converted by the A / D converter 301 (S3). The position / orientation calculation unit 401 calculates the position / orientation ^v θ _IN of the operation target 101 from the sensor signal digitized by the A / D conversion unit 301 (S4). The position and orientation calculated by the position and orientation calculation unit 401 is ^v θ _IN , the target position and orientation is ^v θ ₀ (t), the error from the target is Δθ = d ( ^v θ _IN , ^v θ ₀ (t)), The tactile information conversion unit 402 converts the Δθ using a predetermined tactile information conversion function f (Δθ) to calculate the tactile information x (S5). The D / A conversion is executed by the D / A conversion unit 303 on the tactile information x calculated in this way, and it is sent to the actuator driver 304 as a command value, and moves in advance according to the command value. The actuator 204 is driven (S6). If the operation support is to be continued (YES in S7), the time is updated (S8), the process returns to the target position / posture reading (S2), and a series of control is executed again. Increase the value of k in the time update tk by one. On the other hand, when the operation support is not continuously performed (NO in S7), the series of control is finished and the operation support is ended.

  Specific examples of the tactile information conversion function f (Δθ) are shown in FIGS. 12 (a), 12 (b), 12 (c), 12 (d), and 12 (e). The conversion of the posture deviation Δθ using the tactile information conversion function f (Δθ) is a conversion having a characteristic in the vicinity of Δθ = 0.

  As shown in FIGS. 12A, 12B, and 12C, by changing the width of the haptic information conversion function f (Δθ) in the vicinity of Δθ = 0, the operation target 101 becomes the target posture. As described above, it is possible to adjust the sensitivity of Δθ for ending the operation support.

  When conversion is performed using the tactile information conversion function f (Δθ) shown in FIG. 12A, the posture of the operation target 101 is slightly shifted from the target posture, not only when the posture deviation Δθ is zero. However, the operation support is ended on the assumption that the operation object 101 is in the target posture.

  When conversion is performed using the haptic information conversion function f (Δθ) shown in FIG. 12B, the haptic information conversion function shown in FIG. Even if the deviation of the operation target 101 with respect to the target posture whose allowable range is larger than when f (θ) is used, the operation support is ended assuming that the operation target 101 is in the target posture.

  On the other hand, when conversion is performed using the tactile information conversion function f (Δθ) shown in FIG. 12C, the operation support 101 is assumed to have reached the target posture only when the posture deviation Δθ is zero. Is terminated.

  That is, in the haptic information conversion function f (Δθ) shown in FIGS. 12A and 12B, the sensitivity of Δθ for ending the operation support is weakened if the operation object 101 assumes the target posture. In the tactile information conversion function f (Δθ) shown in FIG.

  For example, in the case where a leveling operation is considered as an operation object 101 by photographing with the digital camera 50, if the state in which the posture of the digital camera 50 is horizontal is set as a target posture, the operator 107 holds the digital camera 50 in his hand. Therefore, when taking a picture, it is very difficult to keep the posture of the digital camera 50 in a strictly horizontal state and keep the posture continuously.

  For this reason, when the sensitivity is strong, if the operation support is terminated only when the posture deviation Δθ = 0, the digital camera 50 is slightly displaced from the horizontal position when taking a picture. Tactile information such as vibration of a magnitude that can be felt by the operator 107 as operation assistance from the camera 50 is presented to the operator 107. For this reason, there is a possibility that the tactile information presented to the operator 107 as operation support may cause hand shake and hinder shooting. In particular, it is likely to occur when a hand holding the digital camera 50 is raised and a front subject is photographed.

  Therefore, even if the sensitivity is weakened and the posture of the digital camera 50 is slightly deviated from the horizontal, if the amount of deviation is within an allowable range for practical use, the posture of the digital camera 50 is substantially horizontal. As a result, small tactile information that is hardly felt is presented to the operator. As a result, it is possible to prevent the camera from being shaken due to tactile information presented from the digital camera 50 to the operator 107 for operation support, and to prevent the shooting from being disturbed. The photographing can be performed in a state or a substantially horizontal state.

  Further, as the tactile information conversion function f (Δθ), the functions shown in FIG. 12D and FIG.

  When the conversion is performed using the tactile information conversion function shown in FIGS. 12A, 12B, and 12C, the roll angle of the digital camera 50 that is the operation target 101 is horizontal. Thus, the same tactile information x is presented if the absolute value of the deviation Δθ of the posture is the same between the case where it is inclined in the direction of arrow A and the case where it is inclined in the direction of arrow B in FIG.

  On the other hand, when conversion is performed using the haptic information conversion function shown in FIGS. 12D and 12E, the roll angle of the digital camera 50 is tilted in the direction of arrow A in FIG. 6 with respect to the horizontal. If the absolute value of the positional deviation Δθ is the same between the case where the user is in the direction of arrow B and the case where the user is inclined in the direction of the arrow B, the haptic information x and the haptic information -x are respectively presented. That is, tactile information that indicates whether the posture of the digital camera 50 is inclined in the direction of the arrow A or the arrow B with respect to the horizontal is presented to the operator. As a result, the operator 107 can intuitively grasp in which direction the digital camera 50 is moved to correct the posture of the digital camera 50 to a desired posture from the presented tactile information. Therefore, the operator 107 can correct the posture by moving the digital camera 50 toward a desired posture more intuitively using the presented tactile information as a clue.

  Note that the tactile information conversion function f (Δθ) shown in FIG. 12D is a case where the sensitivity of Δθ for ending the operation support is strengthened assuming that the operation object 101 is in the target posture. The tactile information conversion function f (Δθ) shown in (e) is a case where the sensitivity is weakened.

  Further, the tactile information conversion function may be a conversion depending on not only Δθ but also its time differentiation (first floor, second floor).

  A plurality of tactile information conversion functions f (Δθ) described above are set and stored in advance in a memory (not shown) in the controller 203, and the conversion rule is changed according to the operator 107 and the operation scene. A tactile information conversion function f (Δθ) may be called and set. As a result, optimal tactile information corresponding to the operator 107 and the operation scene is presented to the operator 107, and operation support is performed so that the operator 107 can intuitively move the operation target 101 toward a desired posture. be able to.

Next, a case where a portable video camera having a moving image shooting function is shot and held will be described. FIG. 13 shows a video camera in the time-series target position and orientation θ ₀ (t) as shown in Table 1. When position information is arranged in a three-dimensional space in time series, one path 500 is formed. The video camera at the target position and orientation ^v θ ₀ (t ₀ ) at time t = t ₀ is _denoted by reference numeral 501, and the video camera at the target position and orientation ^v θ ₀ (t _N ) at time t = t _N is _denoted by reference numeral 502. It is expressed. A target position and orientation defined in such a time series is set in advance. In the same manner as described above, by providing the video camera with a function of presenting the deviation from the target as tactile information to the operator, it is possible to perform time-series video camera operation according to the target position and orientation. For example, a tactile information transmission unit may be provided at a part touched by the fingertip of the operator, such as a recording (REC) button of a video camera.

  Next, the haptic information conversion unit 402 in the controller 203 converts Δθ with a predetermined haptic information conversion function f (Δθ), and the haptic information is applied to the fingertip 109 of the operator 107 based on the obtained haptic information x. An example of the tactile information transmission unit 104 that presents the message will be described.

  FIG. 14A is a diagram illustrating an example of the haptic information transmission unit 104 that transmits haptic information by a force in the normal direction. As shown in FIG. 14 (a), the tactile information transmission means 104 has the movable part 11 in contact with the fingertip 109 of the operator 107, and its vertical displacement is x. That is, the tactile information x is the amount of displacement of the movable part 11 in the vertical direction. As a result, the tactile information can be presented to the fingertip 109 of the operator 107 in contact with the movable portion 11 by the force in the normal direction due to the vertical displacement of the movable portion 11, and the operator 107 can intuitively operate the target. Operation support that allows the object 101 to move toward a desired posture can be performed.

  FIG. 14B is a diagram illustrating an example of the haptic information transmission unit 104 that transmits the haptic information by a tangential force. As shown in FIG. 14B, the tactile information transmission means 104 has a disk 12 that is a movable part in contact with the fingertip 109 of the operator 107, and the rotation angle around the axis of the disk 12 is x. That is, the tactile information x is a rotation angle around the axis of the disk 12. As shown in the figure, the upper surface of the disk 12 may be the contact position with the fingertip 109, and the peripheral portion of the disk 12 may be the contact position with the fingertip 109. Thus, the tactile information can be presented to the fingertip 109 of the operator 107 in contact with the disk 12 by the tangential force caused by the rotation of the disk 12 around the axis, and the operator 107 intuitively sets the operation object 101. It is possible to perform operation support so that the robot can be moved toward a desired posture.

  FIG. 14C is a diagram illustrating an example of the haptic information transmission unit 104 that transmits the haptic information by a tangential force. As shown in FIG. 14 (c), the tactile information transmission means 104 has a movable part 13 having a convex shape on the surface in contact with the fingertip 109 of the operator 107, and the horizontal displacement of the movable part 13 is expressed as x. And That is, the tactile information x is a horizontal displacement amount of the movable portion 13 having a convex shape. As a result, the tactile information can be presented to the fingertip 109 of the operator 107 in contact with the movable portion 13 by the tangential force due to the horizontal displacement of the movable portion 13 having a convex shape, and the operator 107 can intuitively Therefore, it is possible to perform operation support that allows the operation object 101 to be moved toward a desired posture.

  FIG. 14D is a diagram illustrating an example of the haptic information transmission unit 104 that transmits haptic information using a tangential force. As shown in FIG. 14 (d), the tactile information transmission means 104 has a movable part 14 having a concave shape on the surface that is in contact with the fingertip 109 of the operator 107, and the horizontal displacement of the movable part 14 is expressed as x. And That is, the tactile information x is a horizontal displacement amount of the movable part having a concave shape. As a result, the tactile information can be presented to the fingertip 109 of the operator 107 in contact with the movable portion 14 by the tangential force due to the horizontal displacement of the movable portion 14 having a concave shape, and the operator 107 can intuitively Therefore, it is possible to perform operation support that allows the operation object 101 to be moved toward a desired posture.

  FIG. 15 is a diagram illustrating an example of the haptic information transmission unit 104 that transmits haptic information by vibration stimulation. As shown in FIG. 15, the tactile information transmission means 104 has a contact portion 15 made of an elastic body that contacts the fingertip 109 of the operator 107, and a vibrator 16 is attached to the contact portion 15. The amplitude of the driving force of 16 is assumed to be x. That is, the tactile information x is the amplitude of the driving force of the vibrator 16. Thereby, the tactile information can be presented to the fingertip 109 of the operator 107 who has contacted the contact portion 15 by vibration stimulation by vibrating the contact portion 15 by the vibrator 16, and the operator 107 can intuitively Operation support that allows the operation object 101 to be moved toward a desired posture can be performed.

  FIG. 16 is a diagram illustrating an example of the haptic information transmission unit 104 that transmits haptic information by electrical stimulation. As shown in FIG. 16, the tactile information transmission means 104 has a contact portion 17 made of an insulator with which the fingertip 109 of the operator 107 contacts, and a voltage or current supplied to the electrode 18 embedded in the contact portion 17. Let the amplitude be x. That is, the tactile information x is the amplitude of the voltage or current supplied to the electrode 18. Thus, by supplying voltage or current to the electrode 18, tactile information by electrical stimulation can be presented to the fingertip 109 of the operator 107 who has touched the contact portion 17, and the operator 107 can operate intuitively. Operation support that allows the object 101 to move toward a desired posture can be performed.

  14, FIG. 15, and FIG. 16, the module of the haptic information transmission means 104 has the various mobile terminals (mobile phone 30, smart phone 40, digital camera 50, laptop shown in FIG. 2. Is installed in a predetermined space (such as a hatched portion in the figure) where the fingertip 109 of the operator 107 of the type personal computer 60 and the pad type personal computer 70 can come into contact.

  Apart from these, an example will be described in which the tactile information transmission means 104 is incorporated in an existing mounted component.

  First, the contact portion 15 of the module of the tactile information transmission means 104 in the form of FIG. 15 is a part of the touch panel 41 of the smartphone 40 shown in FIG. 17A or the pad type personal computer shown in FIG. The case where it is a part of 70 touch panels 71 will be described.

  FIG. 18 shows an internal sectional structure of the touch panels 41 and 71. One side end in the horizontal width direction of the touch panels 41 and 71 provided so as to cover the display unit 80 on which an image is displayed is connected to an actuator 81 that displaces the touch panels 41 and 71 in the horizontal direction. The actuator 81 is provided on the housing 83. The other side end of the touch panels 41 and 71 in the horizontal width direction is installed in the housing 84 via an elastic body 82. Further, both end portions in the vertical width direction of the touch panels 41 and 71 are not supported and are in a floating state. The tactile information can be presented to the fingertip 109 of the operator 107 in contact with the touch panel 41, 71 by vibration stimulation by vibrating the touch panels 41, 71 in the lateral width direction by the actuator 81. Operation support that allows the smartphone 40 and the pad type personal computer 70 to be moved toward a desired posture intuitively can be performed.

  Next, the case where the movable part of the module of the tactile information transmission means 104 in the form of FIG. 14B is the shutter button (release button) 51 of the digital camera 50 shown in FIG. 19 will be described.

  FIG. 20 shows an internal cross-sectional structure of the digital camera 50 around the shutter button 51 partially protruding from the housing 53. The tactile information transmission means 104 can be displaced downward by being pressed against and touching the fingertip 109 of the operator 107, and a shutter button 51 that is a movable part that can rotate around the rotation shaft 51a, and a rotation shaft 51a. The module includes a rotation actuator 52 that functions as a push switch for releasing the shutter when the shutter button 51 is rotated and the shutter button 51 is pressed downward by the fingertip 109. The tactile information is presented to the fingertip 109 of the operator 107 in contact with the shutter button 51 by the tangential force from the shutter button 51 rotated around the axis by the rotary actuator 52. Accordingly, it is possible to perform operation support that allows the operator 107 to intuitively move the digital camera 50 toward a desired posture.

  In the digital camera 50 shown in FIG. 19, the module of the tactile information transmission means 104 in the form of FIG. 14A is displaced in the vertical direction while the shutter button 51 is displaced at the fingertip 109. It can also be applied as a module including an actuator that functions as a push switch that releases a shutter when pressed downward. Accordingly, the tactile information can be presented to the fingertip 109 of the operator 107 in contact with the shutter button 51 by the normal force from the shutter button 51 displaced in the vertical direction by the actuator. Accordingly, it is possible to perform operation support that allows the operator 107 to intuitively move the digital camera 50 toward a desired posture.

  Various mobile terminals such as the mobile phone 30, the smart phone 40, the digital camera 50, the laptop personal computer 60, and the pad type personal computer 70 can be equipped with a camera function as an application, and the operations described in this embodiment By employing the support device, it is possible to provide a function for supporting the leveling operation of the device by the operator 107.

  In addition, examples of applications that can be installed in various portable terminals include a navigation function based on map information, and a scene that presents a direction indicating a target route using sensor data such as a geomagnetic sensor, an inertial sensor, or GPS is assumed. The In this case, the function of presenting the deviation from the target direction to the fingertip 109 as tactile information can be realized by employing the operation support apparatus described in the present embodiment.

What has been described above is merely an example, and the present invention has a specific effect for each of the following modes.
(Aspect A)
In the operation support apparatus, the difference between the posture information detection unit such as the posture sensor 202 that detects the posture information of the operation target and the posture information detected by the posture information detection unit and the preset target posture information is used as the tactile information. Information conversion means such as a controller 203 for conversion, and tactile information transmission means such as an actuator 204 for transmitting haptic information converted by the information conversion means to an operating body such as a fingertip 109 for operating the operation target. According to this, as described in the above embodiment, the operator can intuitively move the operation object toward the desired position by using the tactile information presented about the position of the operation object as a clue. Can be corrected.
(Aspect B)
In (Aspect A), the posture information detection means is an inertial sensor. According to this, as described in the above-described embodiment, it is possible to detect the movement of the operation object without being grounded and restrained independently of the environment.
(Aspect C)
In (Aspect B), the inertial sensor is an acceleration sensor. According to this, as described in the above embodiment, the posture of the operation target can be calculated from the components at each axis of gravity acceleration.
(Aspect D)
In (Aspect B), the inertial sensor is an angular velocity sensor. According to this, as described in the above embodiment, the posture of the operation target can be calculated by the reference posture and the time integration.
(Aspect E)
In (Aspect A), the posture information detection means is a magnetic sensor. According to this, as described in the above embodiment, the attitude of the operation target can be calculated from the strength of the magnetic field of each axis.
(Aspect F)
In (Aspect A), the posture information detection means is an image sensor. According to this, the attitude of the operation target can be calculated by performing image processing on the obtained image data.
(Aspect G)
In (Aspect A), (Aspect B), (Aspect C), (Aspect D), (Aspect E) or (Aspect F), the tactile information transmission means exerts a normal force on the surface of the operating body. According to this, as described in the above embodiment, tactile information can be presented to the operating body by the force in the normal direction.
(Aspect H)
In (Aspect A), (Aspect B), (Aspect C), (Aspect D), (Aspect E) or (Aspect F), the tactile information transmission means exerts a tangential force on the surface of the operating body. According to this, as described in the above embodiment, tactile information can be presented to the operating body by a tangential force.
(Aspect I)
In (Aspect A), (Aspect B), (Aspect C), (Aspect D), (Aspect E), or (Aspect F), the tactile information transmission means applies vibration stimulation to the operating body. According to this, as described in the above embodiment, tactile information can be presented to the operating body by vibration stimulation.
(Aspect J)
In (Aspect A), (Aspect B), (Aspect C), (Aspect D), (Aspect E) or (Aspect F), the tactile information transmission means applies electrical stimulation to the operating body. According to this, as described in the above embodiment, tactile information can be presented to the operating body by electrical stimulation.
(Aspect K)
In (Aspect A), (Aspect B), (Aspect C), (Aspect D), (Aspect E), (Aspect F), (Aspect G), (Aspect H), (Aspect I) or (Aspect J) The information conversion means changes the conversion rule for each operator who operates the operation target with the operating tool. According to this, as described in the above embodiment, the optimal tactile information corresponding to the operator is presented to the operator so that the operator can intuitively move the operation target toward a desired posture. Operation support can be performed.
(Aspect L)
(Aspect A), (Aspect B), (Aspect C), (Aspect D), (Aspect E), (Aspect F), (Aspect G), (Aspect H), (Aspect I), (Aspect J) or In (Aspect K), the posture information includes any of the roll angle, pitch angle, and yaw angle of the operation target. According to this, as described in the above embodiment, the posture of the operation target can be grasped.
(Aspect M)
In mobile information communication terminals such as the mobile phone 30 and the smartphone 40, (Aspect A), (Aspect B), (Aspect C), (Aspect D), (Aspect E), (Aspect F), (Aspect G), The operation support device according to (Aspect H), (Aspect I), (Aspect J), (Aspect K), or (Aspect L) is provided. According to this, as described in the above embodiment, the operator can intuitively move the communication mobile terminal toward a desired posture using the presentation of tactile information about the posture of the mobile information communication terminal. it can.
(Aspect N)
In an imaging device such as the digital camera 50, (Aspect A), (Aspect B), (Aspect C), (Aspect D), (Aspect E), (Aspect F), (Aspect G), (Aspect H), (Aspect A) The operation support apparatus according to Aspect I), (Aspect J), (Aspect K) or (Aspect L) is provided. According to this, as described in the above embodiment, the operator can intuitively move the photographing apparatus toward a desired posture by using the presentation of tactile information about the posture of the photographing apparatus as a clue.
(Aspect O)
In an information processing apparatus such as a laptop personal computer 60 or a pad type personal computer 70, (Aspect A), (Aspect B), (Aspect C), (Aspect D), (Aspect E), (Aspect F), (Aspect A) The operation support apparatus according to Aspect G), (Aspect H), (Aspect I), (Aspect J), (Aspect K) or (Aspect L) is provided. According to this, as described in the above embodiment, the operator can intuitively move the information processing apparatus toward the desired posture by using the presentation of tactile information about the posture of the information processing apparatus as a clue.

DESCRIPTION OF SYMBOLS 11 Movable part 12 Disk 13 Movable part 14 Movable part 15 Contact part 16 Vibrator 17 Contact part 18 Electrode 30 Cellular phone 40 Smart phone 41 Touch panel 50 Digital camera 50 Digital camera 51 Shutter button 51a Rotation shaft 52 Rotation actuator 53 Case 60 Laptop Type personal computer 70 pad type personal computer 71 touch panel 80 display unit 81 actuator 82 elastic body 83 case 84 case 100 target posture information 101 operation object 102 posture information detection means 103 information conversion means 104 tactile information transmission means 105 operation output 106 Haptic Information 107 Operator 108 Upper Limb 109 Fingertip 202 Posture Sensor 203 Controller 204 Actuator 301 A / D Converter 302 Processor 30 D / A conversion unit 304 actuator driver 401 and orientation calculation unit 402 haptic information converting unit

JP 2010-239553 A

Claims (15)

Posture information detecting means for detecting posture information of the operation target;
Information conversion means for converting a difference between the posture information detected by the posture information detection means and preset target posture information into tactile information;
An operation support apparatus comprising: tactile information transmission means for transmitting the tactile information converted by the information conversion means to an operating body that operates the operation target. The operation support apparatus according to claim 1,
An operation support apparatus, wherein the posture information detecting means is an inertial sensor. The operation support apparatus according to claim 2,
An operation support apparatus, wherein the inertial sensor is an acceleration sensor. The operation support apparatus according to claim 2,
An operation support apparatus, wherein the inertial sensor is an angular velocity sensor. The operation support apparatus according to claim 1,
An operation support apparatus, wherein the posture information detecting means is a magnetic sensor. The operation support apparatus according to claim 1,
An operation support apparatus, wherein the posture information detection means is an image sensor. The operation support device according to claim 1, 2, 3, 4, 5 or 6,
The operation support apparatus, wherein the tactile information transmission means exerts a normal force on the surface of the operation body. The operation support device according to claim 1, 2, 3, 4, 5 or 6,
An operation support apparatus, wherein the tactile information transmission means exerts a tangential force on the surface of the operation body. The operation support device according to claim 1, 2, 3, 4, 5 or 6,
An operation support apparatus, wherein the tactile information transmission means applies a vibration stimulus to the operating body. The operation support device according to claim 1, 2, 3, 4, 5 or 6,
The operation support device, wherein the tactile information transmission means applies electrical stimulation to the operating body. The operation support device according to claim 1, 2, 3, 4, 5, 6, 7, 8, 9 or 10,
The operation support apparatus, wherein the information conversion means changes a conversion rule for each operator who operates the operation object with the operation body. In the operation support device according to claim 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 or 11,
The operation support apparatus, wherein the posture information includes any of a roll angle, a pitch angle, and a yaw angle of the operation target. In a mobile information communication terminal capable of communicating via a communication network,
A mobile information communication terminal comprising the operation support apparatus according to claim 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, or 12. In a photographing apparatus provided with photographing means for photographing a subject,
An imaging apparatus comprising the operation support apparatus according to claim 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, or 12. In portable information processing devices that can be carried,
An information processing apparatus comprising the operation support apparatus according to claim 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, or 12.