JPH06131114A - Three-dimensional data input device - Google Patents

Abstract

PURPOSE:To provide the three-dimensional data input device on which the position of a body and its attitude information can be inputted in the most familiar state when a person performs intellectual operation in daily life. CONSTITUTION:This three-dimensional data input device is equipped with a body 11 which constitutes an operation part, a 3-degree-of-freedom arm consisting of plural articulations JA1-JA6 provided with angle sensors SA1-SA3 and SB1-SB3 for detecting the angle of the body supporting both its ends and actuators MA1-MA3 generating reaction forces, and a control means 14 which recognizes the position and attitude of the body 11 with the detection signals of the angle sensors and calculates the reaction forces to control the generation torque of the actuators. A 6-degree-of-freedom force reaction can be generated by the angle sensors and actuators and when operation for pushing, grinding, cutting, and gripping the body, etc., is performed in space, the same operation as real operation can be obtained. Therefore, the operation which can present a resistance feeling and a weight feeling and is closer to the real operation is obtained and highly precise three-dimensional data input can be performed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional data input device using a pen-type force display capable of presenting a reaction force when performing modeling work in a virtual space.

[0002]

2. Description of the Related Art Artificial reality technology that enables a simulated experience in a virtual world by directly presenting synthetic information created by a computer to human sense organs dramatically improves communication between humans and computers. It is expected to improve. Currently, as a tool for adding an operation to a virtual world, a device such as a data glove that measures a finger movement and recognizes a gesture is often used. However, when man operates an object, the sense of force such as hardness and weight is indispensable. Then, the importance of feedback of power from the virtual world came to be recognized. The force display means a device for expressing such a feeling of force.

Most information media today are related to sight and hearing in terms of human senses. Visual and auditory information is transmitted through non-contact media such as light and sound, so it was relatively easy to realize a display device. However, tactile information does not have a good display because it requires mechanical interaction. In the field of robotics, master manipulators that can present force sensations to remote operations have been used, but these are not suitable for use as general interfaces with computers because of the large scale of the device.

From the viewpoint of hardware, the artificial reality generating technology is roughly divided into two directions. One is “immersive” that cuts off sensory information other than synthetic information of the virtual world by wearing a display device such as an HMD (Head Mounted Display). The other is the "open type" that aims to be comfortable in daily life without wearing special devices. The immersive type can present sensory information in a wide band by mounting many devices. The open type is naturally inferior to the immersive type in terms of expressiveness, but has an advantage in practical use.

Comparing the two types with a stereoscopic image display system as an example, a typical immersive type is the HMD, and an open type is a so-called “glassless” stereoscopic image using a lenticular lens. From the side of the user, it is desirable to be able to enjoy stereoscopic images without wearing anything, but when directly manipulating the displayed figures with hands, things that should be closer to your hands are hidden by your hands. The inconvenience of being invisible occurs.

That is, in the open type, the contradiction caused by the coexistence of the virtual world and the real world cannot be eliminated in principle. There are contradictory strengths and weaknesses between the two, and we cannot generally decide which is better. Therefore, it is necessary to balance in an optimum place according to the purpose.

The biggest difference between the immersive type and the open type when viewed from the side of the device that generates the artificial reality is whether or not the display is in contact with the human body. With respect to sight and hearing, or olfaction, these information are originally propagated in the air, and thus can be displayed in a non-contact manner.

However, somatic sensations (sensations received by skin and muscles) and vestibular sensations (balance sensations) cannot be displayed unless there is some mechanical interaction between the ground and the body. Therefore, in order to feed back these sensory information, it is necessary to take an immersive approach. If you want to display this information openly,
It is possible to use a method that mediates what is frequently touched in daily life.

[0009] For example, in a sensible machine that is popular in theme parks and the like, the vestibular sense information is expressed by moving a chair. No one would feel uncomfortable sitting in a chair. There are various other things that people touch on a daily basis, so if an actuator is attached to them and they are controlled by a computer, somatosensory information can be expressed in an open type.

To date, many devices for measuring the movement of human joints have been developed and put on the market. However, few devices with the function of feeding back force sense are commercially available. Devices for measuring joint movements have formed a market, albeit on a small scale, supported by the need for human factor analysis. However, although the force feedback device has been realized in a robotics laboratory, it has not been commercially available.

[0011]

In order to realize a force display, it is necessary to give mechanical interaction to the outside world. It is argued that there is no need for external devices if direct stimulation is given to the cerebral cortex, but this method causes various problems because the electrodes are embedded. The number of degrees of freedom and the movable range are important parameters in the design of the force display. The degree of freedom and range of motion of a force display greatly differ depending on the type of mechanism that constitutes it. The mechanism for realizing the force display is a multi-joint type that forms a manipulator using multiple links and joints, a thread type that realizes force feedback by thread tension, and a joystick that gives force feedback by attaching an actuator to the joystick. There is a mold. Comparing these three methods, the multi-joint type merit is indispensable in order to construct a versatile force display. In this case, the main design problems can be reduced to two points: how to make the mechanism elegant and how to use the thread form.

First, as a device for preventing the mechanism from becoming large, the parallel mechanism provides a powerful solution. A normal manipulator has joints directly connected like a human arm. In this case, the joint on the root side must support the entire weight of the joint on the tip side. Therefore, if the degree of freedom increases, a large actuator will have to be used. On the other hand, the parallel mechanism means a manipulator in which joints are connected in parallel, and the forces generated by a plurality of actuators are combined. Therefore, even if the actuator of each joint is small, a large force can be applied to the hand portion.

FIG. 9 is a diagram showing a configuration example of a desktop force display, and FIG. 10 is a diagram showing a configuration example of a force display using pneumatic pressure. Figure 10 proposes a force display that is intended for desktop use using a parallel mechanism (Proceedings of the 5th Human Interface Symposium (1989-
10) p1 to p4 “Artificial reality corresponding to force sense-construction of virtual space”). Further, as a device for reducing the weight of the actuator itself, there is a method of using a pneumatic cylinder. Bertea of Rutgers University has proposed a device that attaches a small air cylinder inside the data glove and applies force to the fingertips starting from the palm. However, all of these are troublesome to attach and detach, and there is a problem that the movable range is limited.

The present invention is based on the above consideration, and it is possible to input information on the position and orientation of an object in a state in which a person is most accustomed to when performing intellectual work in daily life. It is an object of the present invention to provide a three-dimensional data input device.

[0015]

To this end, the present invention comprises a plurality of joints provided with an object that constitutes an operating portion, an angle sensor that supports both ends of the object and an angle sensor that detects an angle, and an actuator that generates a reaction force. An arm having three degrees of freedom, and a control means for recognizing the position and orientation of the object by the detection signal of the angle sensor, calculating a reaction force generated in the arm, and controlling the torque generated by the actuator, It is characterized by generating a reaction force when an object is operated in a virtual space and inputting three-dimensional data.

[0016]

In the three-dimensional data input device according to the present invention, an object constituting the operating portion, an angle sensor for supporting both ends of the object and a plurality of joints provided with an actuator for generating a reaction force are used. Since an arm having a degree of freedom and a control means for recognizing the position and orientation of the object by the detection signal of the angle sensor, calculating a reaction force generated in the arm and controlling the torque generated by the actuator, The position and orientation of the object can be recognized by the sensor and the actuator, and the reaction force of 6 degrees of freedom can be generated in the arm, and when the object is pushed, scraped, cut, or grasped in the virtual space, You can get the same behavior as in reality. Therefore, it is possible to obtain a more realistic motion capable of expressing resistance sensation, weight sensation, etc., and it is possible to input highly accurate three-dimensional data.

[0017]

Embodiments of the present invention will be described below with reference to the drawings. 1 is a diagram showing an embodiment of a three-dimensional data input device of the present invention, FIG. 2 is a diagram for explaining the flow of signal processing performed by the three-dimensional data input device of the present invention, and FIG. 3 is a manipulator of the present invention. FIG. 4 is a diagram showing the basic configuration of the above and the force and torque applied to the grip portion, and FIG. 4 is a diagram showing the movable range of the manipulator. In the figure, 11 is a pen axis, 12 is a translation / rotation conversion mechanism, 13 is a graphic display, 14 is a computer, 15 is an AD converter, JA1-JA6, JB
1 to JB5 are joints, MA1 to MA3, MB1 to MB3 are motor actuators, SA1 to SA3, SB1 to SB
3 is an angle sensor, PA1 to PA6, PB1 to PB5 are power amplifiers, and WA1 to WA6 and WB1 to WB5 are pulse width modulation circuits.

FIG. 1 shows an example of the configuration of a pen-type force display that constitutes the three-dimensional data input device of the present invention.
(A), and the joints JA1 to JA6 are attached to the tip of the pen shaft 11.
, And joints JB1 to JB5 are connected to the opposite side via a translation and rotation conversion mechanism 12. Of these joints, joints JA2, JA3, JA5, JB2, JB
3, JB5 are pivot joints, and joints JA1, JA4, J
A6, JB1, and JB4 are rotary joints. Then, motor actuators MA1 to MA3 and M for generating reaction forces in the joints JA1 to JA3 and JB1 to JB3, respectively.
B1 to MB3, angle sensors SA1 to SA3 and SB1 to SB3 for detecting a turning angle and a rotation angle are provided.

That is, in the pen-type force display of the present invention, the tip of the pen shaft 11 and the fixed point are connected by three pivot joints JA2, JA3, JA5, and rotary joints are provided on both sides of the front pivot joint JA5. Insert JA4 and JA6, and insert the rotary joint JA1 between the fixed point and the swing joint JA2.
Have been inserted. Then, the three joints JA1 on the fixed point side
.. to JA3, the motor actuators MA1 to MA3 apply reaction forces while the angle sensors SA1 to SA3 detect angles. Further, on the side opposite to the tip of the pen shaft 11, the fixed point is connected via the translation and rotation conversion mechanism 12 by three pivot joints JB2, JB3, JB5, and the front pivot joints JB5 and JB3 are connected. The rotary joint JB4 is inserted between the fixed point and the rotary joint JB2. And 3 on the fixed point side
Angle sensors SB1 to S for one joint JB1 to JB3
While detecting the angle at B3, the motor actuator MB1
~ MB3 tries to give a reaction force.

Angle sensors SA1 to SA3, SB1 to SB
3 detects the angle, and motor actuators MA1 to M
FIG. 1B shows an example of the configuration of a signal processing system that generates a reaction force on A3 and MB1 to MB3. In FIG. 1B, the computer 14 performs generation of a virtual object, determination of contact between a pen and a virtual object, generation of reaction force to be returned to the pen, geometric calculation of manipulator, and drawing of three-dimensional graphic. Angle sensors SA1 to SA3, SB1
Each joint JA1 to JA3, JB1 to JB detected by SB3
3 is input, the posture of the pen shaft 11 is obtained, the reaction force in the virtual object is calculated, and each joint JA1 to JA3, J
B1-JB3 motor actuators MA1-MA3,
The drive signals of MB1 to MB3 are generated, and the display data of the operating environment of the virtual object is also generated. The AD converter 15 is an interface means between the angle sensor and the computer, and the angle sensors SA1 to SA3 and SB1.
~ SB3 analog detection angle signal to the computer 14
It is converted to a digital signal for processing by.
The pulse width modulation circuits WA1 to WA6 and WB1 to WB5 are
The drive signal generated by the computer 14 is pulse-width modulated for driving the DC motor, and the power amplifier PA1
-PA6 and PB1 to PB5 amplify the pulse-width-modulated drive signal to drive the motor actuators MA1 to MA3 and M.
B1 to MB3 are driven. Then, the graphic display 13 realizes modeling work in a virtual world by displaying, for example, an image of a virtual material and a virtual tool in response to the operation of the pen shaft 11.

The outline of the processing by the computer 14 will be described. First, the angle sensors SA1 to SA3 and SB1 to SB.
The signal of No. 3 is taken in (step S1) and the position and orientation of the pen shaft 11 are obtained (step S2). Next, the positional relationship between the virtual material and the pen is calculated (step S3), and if the pen is in contact, the reaction force applied to the pen is obtained based on the property of the material (steps S4 to S5). The torques to be generated by the motor actuators MA1 to MA3 and MB1 to MB3 of the joints JA1 to JA3 and JB1 to JB3 are obtained from the reaction force vector and output (step S6). Then, the shape data of the virtual material deformed by the pen is changed (step S7), and the images of the virtual material and the virtual tool are displayed on the graphic display 13 (step S).
8).

The human hand has three ways of translation and three ways of rotation, and has a total of six degrees of freedom. When such a degree of freedom is realized by an ordinary manipulator in which the joints are directly connected, the joint from the root must support the weight of the joint near the hand. Therefore, the overall hardware becomes large. In view of this, in the present invention, the size and weight of the entire manipulator is reduced by employing a mechanism called a parallel mechanism in which joints are connected in parallel as shown in FIG. 1 (A). Basically, a mechanism in which both ends of a pen-shaped gripping portion as shown in FIG. 3 are supported by manipulators having three degrees of freedom is adopted. As a result, the total degree of freedom is 6, so that a triaxial force F and a triaxial torque M can be generated in the grip portion.

Therefore, in the configuration of the present invention shown in FIG. 1A, two manipulators support the points A and B at both ends of the pen shaft. Point A corresponds to the position of the pen tip, and point B determines the attitude of the pen axis. In order to correspond to the rotational movement of the pen shaft in the axial direction, a mechanism for converting this rotational movement into a change in the distance between AB is provided. Therefore, if the positions of both AB points in the three-dimensional space are known, the position and orientation of the pen axis can be known. The position of point A is
The angles are measured by the angle sensors SA1 to SA3 of the joints JA1 to JA3, and the position of the point B is measured by the angle sensors SB1 to SB3 of the joints JB1 to JB3. Then, by appropriately controlling the force vectors at the points A and B generated by the motor actuators MA1 to MA3 and MB1 to MB3 of the joints JA1 to JA3 and JB1 to JB3, a force and torque with 6 degrees of freedom are applied to the pen shaft. Can be generated. Since the joints JA3 and JB3 can bring the actuator to the root by using the pantograph mechanism, the manipulator as a whole has the merit that the weight of the actuator is not in the hands of the operator. The light weight of the moving part in the air is important for the operation of the master manipulator.

FIG. 4 shows the movable range of the pen tip, and the pen tip can thus be moved within a certain hemisphere. The shorter the distance between the two points on both ends of the pen, and the longer the arm portion of the manipulator, the larger this movable range. However, since the length of the arm is in inverse proportion to the reaction force that can be generated on the pen tip, it is limited to some extent.

The present invention will be described in more detail with reference to specific operation examples. FIG. 5 is a view for explaining a reaction force generation method, FIG. 6 is a view for explaining a free-form surface shape operation, and FIG. 7 is a surface. FIG. 8 is a diagram showing an example of the model, and FIG. 8 is a diagram showing the magnitude of the reaction force.

Since A and B shown in FIG. 1A can move independently of each other, the reaction force to the pen can be obtained by using two points at both ends of the pen defined by each manipulator as action points. can get. For example, as shown in FIG. 5, a reaction force for translating the pen is obtained by making the direction vectors of these two points equal so that the torque in the axial direction of the pen is in opposite directions with the two points at both ends as the starting point and the ending point, respectively. This can be achieved by giving the vector of.

From FIG. 4, motor torques T ₀ , T ₁ and T ₂ (torques corresponding to θ ₀ , θ ₁ and θ ₂ , respectively) required to generate the force vector F acting on the end point h1 of the pen. ) Is given by the following equation.

[0028]

[Equation 1]

Then, the magnitude of the reaction vector is

[0030]

[Equation 2]

It becomes Substituting these into the following equation,
Each torque is calculated.

[0032]

[Equation 3]

The same applies to the end point h2 on the opposite side of the pen.

By design, the maximum reaction force that can be returned to the pen tip is about 6
Although the load was 00 g (when the current flowing to the motor was about 1.2 A), the actually obtained torque was a maximum of about 500 g in the horizontal direction that was not affected by gravity. Considering that the weight of the grip of the pen is about 220 g, sufficient torque was obtained.

Using such a system, a basic application for manipulating the shape of a free-form surface was manufactured.
As a result, the user can generate a free-form surface by pressing the surface with a virtual pen as shown in FIG.

As shown in FIG. 7, the surface model has four fixed points S1 to S4 and four action points A to D that are directly deformable.
It consists of 8 points, points a to h that can be deformed in an articulated manner. When each action point exceeds a certain amount of deformation, four points adjacent to that point, for example, in the case of action point A, points a, B, D,
h is also deformed by 1/2 of the deformation amount. This neighbor is
It is assumed that deformation occurs when the difference in displacement from the point of action exceeds a certain value.

When the pen deforms a curved surface, a reaction force proportional to the amount of deformation is returned. The direction of the reaction force is perpendicular to the surface (horizontal to the ground). The magnitude of the reaction force in this case is determined so as to be proportional to the amount of deformation of the curved surface as shown in FIG. The contact between the point of action and the pen is determined by whether or not the point of contact in the vicinity of each point of action in FIG. 7 is in contact, and the end of deformation is determined by the movement of the pen away from the surface.

A person is most familiar with a pen when performing intellectual work in daily life, and a tool such as a cutter knife or a spatula is generally used when performing work. Each of these has a bar-shaped grip portion, and at the time of work, a reaction force is felt through the grip portion. According to the present invention, as described above, both ends of the pen-like shape are supported by the arms composed of a plurality of joints, an angle sensor and an actuator are attached to the joints, and the position and posture of the pen are recognized to act between the ground and the hand. Since the force to generate is generated, a non-wearable force display can be realized.

The present invention is not limited to the above embodiment, but various modifications can be made. For example, in the above embodiment, the control means is composed of one computer, but it is needless to say that the control means may be composed of a plurality of computers for distributed processing according to the type and purpose of the application.

[0040]

As is apparent from the above description, according to the present invention, both ends of a rod-shaped object are connected by the arms having three degrees of freedom by connecting joints equipped with an angle sensor and an actuator in parallel in a three-dimensional space. Since it is supported, the load on each arm can be reduced, and the actuator at the joint can be downsized. Moreover, since the angle sensor and the actuator recognize the position and orientation of the object and generate a reaction force with 6 degrees of freedom in the arm, when the object is pushed, scraped, cut, grasped, or the like in the virtual space. , You can get the same behavior as reality. Therefore, it is possible to obtain a more realistic motion capable of expressing a resistance sensation, a weight sensation, etc., and it is possible to input highly accurate three-dimensional data. Furthermore, it becomes possible to input three-dimensional data with simple operations, and it can be applied to design support devices such as CAD and CG that perform actions such as cutting and pushing objects in industrial design, remote control devices for robots, etc. Can be expected.

[Brief description of drawings]

FIG. 1 is a diagram showing an embodiment of a three-dimensional data input device of the present invention.

FIG. 2 is a diagram for explaining the flow of signal processing performed by the three-dimensional data input device of the present invention.

FIG. 3 is a diagram showing a basic configuration of a manipulator of the present invention and a force and a torque applied to a grip portion.

FIG. 4 is a diagram showing a movable range of a manipulator.

FIG. 5 is a diagram for explaining a reaction force generation method.

FIG. 6 is a diagram for explaining a shape operation of a free-form surface.

FIG. 7 is a diagram showing an example of a surface model.

FIG. 8 is a diagram showing the magnitude of reaction force.

FIG. 9 is a diagram showing a configuration example of a desktop force display.

FIG. 10 is a diagram showing a configuration example of a force display using air pressure.

[Explanation of symbols]

11 ... Pen axis, 12 ... Translation and rotation conversion mechanism, 13 ... Graphic display, 14 ... Computer, 15 ...
AD converter, JA1 to JA6, JB1 to JB5 ... Joint, MA1 to MA3, MB1 to MB3 ... Motor actuator, SA1 to SA3, SB1 to SB3 ... Angle sensor, PA1 to PA6, PB1 to PB5 ... Power amplifier,
WA1 to WA6, WB1 to WB5 ... Pulse width modulation circuit

Claims (6)

[Claims] 1. An arm having three degrees of freedom, which comprises an object constituting an operation section, an angle sensor for supporting both ends of the object and an angle sensor for detecting an angle, and a plurality of joints provided with an actuator for generating a reaction force, the arm having three degrees of freedom, When an object is operated in a virtual space, the control device controls the torque generated by the actuator by calculating the reaction force generated in the arm by recognizing the position and posture of the object by the detection signal of the angle sensor. A three-dimensional data input device, wherein a three-dimensional data is input by generating a reaction force in the. 2. The three-dimensional data input device according to claim 1, wherein the arm that supports one end of the object includes three pivot joints and three rotary joints. 3. The three-dimensional data input device according to claim 1, wherein the arm that supports the other end of the object includes a conversion mechanism and three swing joints and two rotary joints that are connected via the conversion mechanism. And a three-dimensional data input device. 4. The three-dimensional data input device according to claim 2, wherein the angle sensor and the actuator are provided at one rotary joint and two pivot joints connected to a fixed end side of the arm. A three-dimensional data input device characterized in that 5. The three-dimensional data input device according to claim 1, wherein the control means has a graphic display and calculates a mutual positional relationship between a virtual tool and a virtual material from the position and the posture of the object. A three-dimensional data input device for displaying the virtual tool and the virtual material on a graphic display. 6. The three-dimensional data input device according to claim 5, wherein contact judgment between the virtual tool and the virtual material is performed based on the mutual positional relationship, and based on the properties of the virtual tool and the virtual material. A three-dimensional data input device for calculating a reaction force applied to the arm.