JP2782428B2 - Virtual object operation device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a virtual object operation device for operating a virtual object in a virtual world in the real world, and more particularly to a virtual object operation device for generating a reaction force according to the operation of a virtual object. .

[0002]

2. Description of the Related Art A conventional virtual object operating device is, for example,
(1) Ogi, Hirose: "Active data presentation based on somatic sensation", Proc. 9th Human Interface Symposium, p.531-538 (1993), (2) Calendar book: "New based on situational awareness" Proposal for Interaction Style ", WI
It is disclosed in SS'94 Transactions, p.9-17 (1994).

[0003] Conventionally, an image of an object is simply displayed on a display (monitor), and an operation such as bending or rotation of the displayed object cannot be performed.

If a bar code or the like is attached to an object taken in on a display (monitor), it is only possible to read additional information previously assigned to the code by reading the bar code. there were.

As a technique for operating an object image on a display (monitor), CAD (Computer Aided Des
As in ign), a pointing device such as a mouse is used to rotate or deform an object. However, in this case, in operations such as deformation,
He could only give commands and couldn't actually apply force to the object by hand. Further, in the case of metal, it is not possible to generate a reaction force against an operation such as deformation in accordance with the material of the object, as it is difficult to bend.

As a method of directly operating a virtual object, a virtual object is visually grasped using a glove-type data input device or the like, and the object is interlocked with the deformation of the hand. Deformation was being done. In this case, the reaction force generating function is not usually provided, and the reaction force is generated by adding a reaction force generation mechanism to the glove type data input device in order to generate the reaction force. Needed.

[0007]

However, as described above, a device for generating a force by adding a reaction force generating mechanism to a glove type data input device is mechanically complicated and requires time for attaching and detaching the device. There was a problem that it could not be easily used.

The present invention has been made to solve the above problems, and has a simple structure (mechanism).
It is an object of the present invention to provide a virtual object operation device capable of generating a reaction force according to an operation such as deformation of a virtual object. That is, an object of the present invention is to provide a simple structure (mechanism)
It is an object of the present invention to provide a virtual object operation device having a virtual object operation device capable of obtaining a real feeling and a texture as if a person operating a virtual object is actually operating the object.

[0009]

A virtual object operating device according to a first aspect of the present invention is a virtual object operating device for operating a virtual object in a virtual world in the real world, wherein the virtual object is displayed. The displayed virtual object is operated by moving the displayed display device.

As described above, the virtual object operation device according to the first invention can easily operate the virtual object only by moving the display device.

[0011] Preferably, when the display device is moved, a sense of weight corresponding to the weight of the object represented by the virtual object is generated.

As described above, since a sense of weight corresponding to the weight of the object represented by the virtual object is generated, a person who moves the display device can feel the weight as if he / she actually holds the object.

Preferably, when the display device is moved, a reaction force is generated according to the operation of the virtual object.

As described above, since the reaction force is generated in accordance with the operation of the virtual object, the person who moves the display device can obtain the real feeling and texture as if the user were actually operating the object.

A virtual object operation device according to a second aspect of the present invention is a virtual object operation device for operating a virtual object in a virtual world in the real world, and a display device for displaying the virtual object, and a display device. And support means for supporting the
The displayed virtual object is operated by moving the display device.

As described above, the virtual object operation device according to the second invention can easily operate the virtual object only by moving the display device.

Preferably, the supporting means includes a heavy feeling generating means for generating a heavy feeling according to the weight of the object represented by the virtual object.

As described above, since a feeling of weight corresponding to the weight of the object represented by the virtual object is generated, the person who moves the display device can feel the weight as if he / she actually holds the object. Also, with a simple configuration (mechanism) of the display device and the support means, the person who moves the display device can feel the weight as if he / she actually held an object.

Preferably, the supporting means includes a reaction force generating means for generating a reaction force according to the operation of the virtual object when moving the display device.

As described above, since the reaction force is generated in accordance with the operation of the virtual object, the person who moves the display device can obtain the real feeling and texture as if the user were actually operating the object. In addition, with a simple configuration (mechanism) of the display device and the support means, a person who moves the display device can obtain a real feeling and texture as if the user were actually operating an object.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a virtual object operation device according to the present invention will be described with reference to the drawings.

(Embodiment 1) A virtual object operating device according to Embodiment 1 of the present invention captures a virtual object on a display (monitor). Then, when performing any operation (action) on the captured virtual object, a reaction force corresponding to the operation (action) is generated. That is, the first embodiment
In the virtual object operation device according to the above, the operator can perform an operation (action) on the virtual object while feeling a reaction force corresponding to the operation (action) on the virtual object.

Here, the reaction force generally refers to a reaction that is received by a person who has performed an action on a certain object. In the virtual object operating device according to the first embodiment, the reaction force mainly means a force that feels the weight of the object.

Hereinafter, the virtual object operating device according to the first embodiment will be described in detail. FIG. 1 is a schematic diagram showing a virtual object operation device according to Embodiment 1 of the present invention.

Referring to FIG. 1, the virtual object operating device according to the first embodiment includes a small display (monitor) 1, a manipulator 3, and a universal joint (not shown). The manipulator 3 includes active rotation axes 3a, 3b,
3c.

The manipulator 3 and the small display (monitor) 1 are connected via a universal joint (not shown).

The active rotation shaft 3a rotates around the A axis. The active rotation shaft 3b rotates around the B axis. The active rotation shaft 3c rotates around the C axis.

The small display (monitor) 1 can rotate around the D axis along the plane of the small display (monitor) 1. The small-sized display (monitor) 1 can rotate around the E-axis along the surface of the small-sized display (monitor) 1 and perpendicular to the D-axis. Small display (monitor)
1 is perpendicular to the surface of the small display (monitor) 1 (D
About the F axis (perpendicular to the plane containing the axis and the E axis).

Small display 1 such as a handy monitor
Is mounted in front of the manipulator 3, and can fix the position of the small display 1 in the air like a Z-light for illumination. The manipulator 3 has several degrees of freedom, and has active rotary shafts 3a, 3b,
3c can generate a reaction force, whereby the operator of the small display 1 can obtain a feeling of weight corresponding to the weight of the object represented by the virtual object displayed on the small display 1. That is, the manipulator 3 forms a force feedback device. The force feedback device using the ultrasonic motor will be described later in detail.

The small display 1 is attached to the manipulator 3 so that its tip becomes a fulcrum. That is, the center point of the small display 1 coincides with the position of the distal end of the manipulator 3, and three axes (D axis, E axis, and F axis) can be rotated around the center point of the small display 1. Also, a small display 1 (manipulator 3)
Can move up, down, left and right (D axis, E axis, F axis directions).

The tip position of the manipulator 3 (the center point of the small display 1) is the bending angle of the three-dimensional position sensor or the manipulator 3 (rotation angle around A axis, rotation angle around B axis, rotation angle around C axis). Can be measured from

A procedure for performing an operation (action) on a virtual object using the virtual object operation device according to the present embodiment will be described.

FIG. 2 is a schematic diagram showing a virtual object operator using a virtual object operation device. The same parts as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

Referring to FIG. 2, an operator 5 carries out an operation (action) on a virtual object by holding small display 1.

FIG. 3 is a schematic diagram showing the use of the virtual object operation device in the presence communication conference system. The same parts as those in FIG. 2 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

Referring to FIG. 3, two screens 11a and 11b are provided on the two screen supporting portions 9a and 9b. Also, the screen support portions 9a and 9b
Each is provided with a speaker 7. Using computer graphics technology, a virtual world is created by the screens 11a and 11b. In the virtual world formed by the screens 11a and 11b, a person 13 (hereinafter, also referred to as "virtual person 13") and an object 15 (hereinafter, "virtual object 1") are displayed.
5 ").

On the other hand, in the real world, the operator 5
A virtual object operation device including a small display 1 and a manipulator 3 is used. In FIG. 3, the boundary between the virtual world and the real world is not clearly distinguished.

Using the virtual object operating device, the virtual object 15
On the other hand, the operator 5 performing the operation (action) first holds the small display 1 and points the small display 1 toward the virtual object 15 in the virtual world (in the virtual space). By doing so, the virtual object 15 is displayed in the small display 1.
Is displayed. Note that the small display 1 is, for example, a handy monitor. Small display 1
Is directed to the virtual object 15, for example, when shooting an object with a portable video camera.

In FIG. 3, as the virtual object 15,
Although “Mikoshi” is shown, in the following description, the virtual object 15 will be described as a simple object.

A touch panel is attached to the display surface of the small display 1, and a straight line or the like can be drawn on the image (virtual object 15) displayed on the small display 1. In addition, the small display 1 includes:
There is a switch, and by turning on / off the switch, the gripping function for the virtual object 15 displayed on the small display 1 can be turned on / off.

When the holding function of the small display 1 is off, the position of the small display 1 can be changed without changing the position of the virtual object 15 in the virtual world (in the virtual space). Therefore, the size of the small display 1 depends on the position and the rotation direction of the small display 1.
The position at which the user views the video (virtual object 15) displayed on the screen changes. That is, when the gripping function is off, the virtual object 15 can be viewed from any position by arbitrarily moving the small display 1.

When performing an operation (action) on the virtual object 15, the gripping function of the small display 1 is turned on. When the holding function of the small display 1 is on,
It is as if the virtual object 15 is grasped, and the position and the direction of the virtual object 15 in the virtual world (in the virtual space) can be changed. That is, by rotating and moving the small display 1 itself, the entire virtual object 15 can be rotated and moved, or a part of the virtual object 15 can be bent. As described above, the image on the small display 1 when the operation (action) is performed on the virtual object 15 is displayed simultaneously with the deformation of the virtual object 15 (operation / action on the virtual object 15).

In the case of rotation of the entire virtual object 15, when the small display 1 is rotated, the position on the surface of the virtual object 15 displayed on the small display 1 matches the viewpoint position of the small display 1. Is the center of rotation.

A case where a part of the virtual object 15 is bent, such as bending a corner of a paper, will be described. The gripping function of the small display 1 is turned off, the small display 1 is moved with respect to the virtual object 15, and a specific place (a place to be bent) of the virtual object is projected on the small display 1.

FIG. 4 is a view showing a small display on which a specific place (a place to be bent) of the virtual object is projected. The same components as those in FIG. 3 are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

A box is displayed as the virtual object 15. Consider a case where a hatched portion of a box as the virtual object 15 is bent. Turn on the gripping function and turn on the small display 1
Above, draw a line C corresponding to the bending line. Then, the small display 1 is rotated in the direction of arrow A. Then, in conjunction with the rotation of the small display 1 in the direction of the arrow A, the hatched portion of the box as the virtual object 15 bends in the direction of the arrow B.

Here, an ultrasonic motor used in the above-described force feedback device (manipulator 3) will be described. The ultrasonic motor is constituted by a plurality of ultrasonic generating rings, wherein an inner ring adjusts a rotation holding force and an outer ring generates a rotational force. Due to the double structure of the ring, it is possible to control the rotational force according to the change in load. Also,
Unlike an electromagnetic motor, it does not adversely affect a magnetic sensor such as a three-dimensional digitizer. Therefore, it is possible to generate a reaction force when grasping a virtual object or to use it as a three-dimensional pointing device. Such an ultrasonic motor is used as the active rotating shafts 3a, 3b, 3c as described above.

A force feedback device using an ultrasonic motor is disclosed in, for example, Japanese Patent Application Laid-Open No. 2-0274.
No. 18 (3D coordinate input control device),
No. 27420 (three-dimensional coordinate input control device) and Japanese Patent Laid-Open No. 2-026283 (rotational force control mechanism).

As described above, the virtual object operation device according to the first embodiment of the present invention has a gripping function, and only by moving the small display 1, can the virtual object projected on the small display 1 be moved. Thus, the operation (action) can be easily performed.

In the virtual object operating device according to the first embodiment, the manipulator 3 that generates a sense of weight (reaction force) according to the weight of the object represented by the virtual object is used. Therefore, the operator 5 who performs an operation (action) on the virtual object using the small display 1 can feel the weight as if he / she actually holds the object.

In the virtual object operating device according to the first embodiment, the small display 1 and the manipulator 3 are used, and with such a simple configuration (mechanism),
A heavy feeling (reaction force) can be generated, and the operator can feel a heavy feeling as if he / she actually holds an object.

Next, the virtual object operating device according to the first embodiment will be described focusing on experimental results.

The application field of the virtual world generated by the VR (Virtual Reality) technology is expected in various fields such as remote control, design and CAD. Extracting the process of object operation work (virtual object operation work) in the virtual space, which is indispensable in these applications, can be described as follows: (1) a phase of searching for an object from an appropriate viewpoint; This is a series of loops in which a phase for deforming and (3) a phase for confirming the operation result are a unit.

In many researches on virtual reality (virtual reality) so far, many search methods and operation methods have been independently proposed in accordance with the development in their intended application fields, but they are not always effective. It cannot be said that it provides an interface as a combination. When the balance of the input device or the output device is lost, the series of operations will be disrupted,
It cannot be a good interface as a whole.

Focusing on this operation flow, as a virtual object operation device according to the first embodiment, a virtual world object operation device (PDDM:
Palmtop Display for Dextrous Manipulation) will be described. The PDDM as a virtual object operating device according to the first embodiment is developed with an operation for assembling and deforming an object in a real-life communication conference system in mind. As a device capable of presenting a haptic sense, a design that is realized by the same device and that is not conscious of interruption of a work flow due to a change operation between these different function modes is adopted.

First, the outline of the PDDM as the virtual object operating device according to the first embodiment, the information presenting method and the operating method are positioned through the current device classification, and then the details of the prototype model will be described. further,
The experimental results of the operation environment evaluation using this prototype model will be described, and future application developments will be summarized.

Until now, the prototype system of the realistic communication conference that has been proposed has used an interface based on a fixed large screen type stereoscopic display, a globe device, and a magnetic sensor. Generally, in an operation environment based on a glove device, it is difficult to perform a detailed operation requiring high accuracy. The main cause is that the operation in the space does not have any constraint conditions unlike a desk or the like. As a countermeasure method, there are a method of intelligently constraining the operation itself, particularly in the case of contact between objects, and a method of adding a restriction to the operation using a force sense device.

The virtual object operation device according to the first embodiment aims at an operation support environment particularly using the latter haptic device. Although studies on other force displays have shown their effectiveness, the use of haptic devices for fixed large-screen displays has the following problems.

(1) There is a problem that a large-sized manipulator that can cope with a wide movable range is required, and (2) a visual field obstruction is caused by a mechanism on the front surface of the display.

To solve these problems, a portable display (Palmtop Disp.
A system in which a three-dimensional mouse-like object operation function and a force sense presentation function are added to (lay)) is the PDDM as the virtual object operation device according to the first embodiment. PDDM is
It has a structure in which some kind of force display is connected to a small liquid crystal television.

FIG. 5 is a diagram showing an expected use form when the PDDM according to the first embodiment is incorporated into a real-life communication conference system. The same parts as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

Referring to FIG. 5, in the real world, there is a PDDM including conference participant 17, small display 1 and manipulator 3. The PDDM shown in FIG.
Corresponds to the virtual object operation device in FIG. The area bb is
It is a virtual world with a large display. Virtual world bb
Includes conference participants 19 and 21 and object 23a
Exists. The arrow c indicates that one object is present in the computer, but two objects 23a and 23b are displayed in the local work space aa of the real world and the virtual world, respectively. This means that the region bb is a real-world conference participant 17
Since it can be said that the region bb is shared by the conference participants 19 and 21 in the virtual world, the region bb can be defined as a shared virtual space.

In the virtual world (virtual space) bb, other conference participants 19 and 21 and an object 23a are arranged. Here, if the conference participant 17 in the real world intends to make a change to a part of the object 23a (the shaded portion of the object 23a), first, the object 23a
Is copied to the local work space aa in front of the conference participant 17. The transferred object 23b and the original object 23
A link (cc) is attached to a, and the operation input to the copy (object 23b) is applied to the original object 23a as it is.

The conference participant 17 using the PDDM moves to an appropriate viewpoint in the work space aa and confirms the objects 23a and 23b to be changed. The small display 1 is moved in the direction in which the object 23a (23b) is to be deformed and moved, as if the part (part) to be operated of the object 23a (23b) was focused.

That is, the movement of the small display 1 is reflected on the deformation and movement of the object 23a (23b) in accordance with the constraint condition of the object 23a, and the operation reaction force is presented to the conference participant 17 through the force display. become. Note that the object 23a (23
Details of the method of transforming / moving b) are the same as those described with reference to FIG.

The PDDM according to the first embodiment as described above
Thereby, the above-mentioned problem is solved as follows. That is, (1) the force display does not require a wide movable range. (2) There is no mechanism for presenting a force sense between the information display surface and the viewpoint, and the view is not obstructed. (3) The system becomes compact by integrating the display system and the operation system.

Further, the following advantages are expected as compared with a typical force display in which force sense is directly presented on the arm of a user (conference participant 17) using PDDM.

That is, (1) as long as the operation target (object) is within the field of view, an object that cannot be reached can be operated with high accuracy in front of the body; (2) a user of the PDDM (participation in a conference) 17)
The advantages are that the operating environment is intuitive and (3) there is no need to attach (grasp) when not in use.

Next, these features of PDDM will be further described in comparison with other interface devices. PDD
M is an interface device having both an information presenting function and an operation input function, and will explain differences from other devices in terms of their respective functions.

The classification from the information presenting means will be described. FIG. 6 is a diagram showing the classification of the information display system (information display means).

In the means for presenting virtual space information by video,
The motion information of the user is linked to the control of a virtual camera in the virtual world to imitate the appearance in the real world.
In particular, depending on the user's position in the virtual space, a window type (Window type) that looks through the window from outside the virtual world,
It can be classified into a palmtop type (Palmtop type), an immersive type (Immersive type) that enters the virtual world, and a projection type (Projection type) that extends the virtual world into the real world. In FIG. 6, there is a proposal scheme, which indicates the PDDM according to the first embodiment. That is, the PDDM according to the first embodiment is
It is classified as a palmtop type.

If attention is paid to the degree of freedom and immersion in information search, the immersion type is literally the best.
However, in accordance with the improvement of the immersion sensation, the required load of the device increases psychologically and physically. Conversely, a window-type interface is relatively easy to implement, but lacks immersion.

The palmtop type employed in the first embodiment has intermediate features between them. If the information presentation screen is limited to a small liquid crystal surface, the information presentation ability may be inferior to that of a head-mounted (HMD) or large-screen system. However, the high degree of freedom obtained by moving the presentation surface with the hand does not necessarily lower the environmental recognition ability. Fitzmoris et al.'S Chameleon is widely known as a related work on Palmtop Display, but they map the movement of the palmtop display to search the information space. On the other hand, the PDDM according to the first embodiment is expanded and developed in that, in addition to the information search function in the virtual space, a function of operating the operation target itself with the palmtop display itself is added. It can be said that there is.

Next, the classification from the operation input means will be described. In other words, existing multidimensional operation input devices that are widely used are classified according to two axes of “reference coordinates of operation input” and “positional relationship between operation device and effect device”.

FIG. 7 is a diagram showing the classification of operation input devices (operation input means). Referring to FIG. 7, each axis has “reference coordinates of operation input (reference coordinate system of device)”.
The classification of whether the reference coordinate axis of the operation amount input by the user is a system fixed to the environment or a system fixed to the user is determined by the "positional relationship between the operation device and the effector" and the location where the user inputs the operation Represents a classification based on whether the locations where the effects appear are coincident or distant.

A description will be given of a “based on the world and overlap type”.

A device that is used in an environment where the place where the operating device moves and the case where the effector operates coincides, and is used in coordinate axes where the input of the operation is fixed to the environment, such as a touch panel or a glove type Devices such as data input devices belong to this category. In this category, if it is desired to move an object along an axis in the environment, the object is grasped and the operating device is moved in the axis direction. That is, the direction in the operation is defined based on the environment.

As a feature, an operation space suitable for the target environment is required, but an intuitive interface requiring no skill can be realized. In addition, since the input direction is defined in the space, the position of the user is an important factor. In general, this is not a problem because the movable area of the operation target is small, but in a wide space such as a walkthrough, means for moving to a place where the target exists for operation is required.

A description will be given of a “based on the user and separate” type.

A device used in a coordinate axis where the field where the operating device moves and the case where the effector operates does not match, and the input of the operation is fixed to the user, such as a joystick (JoyStick) or mechanical mouse ( Mechanical Mous
e) belongs to this category. In the above example, the operating device is moved along the coordinate system geometrically developed in the user environment, and the operation target object is indirectly grasped using an icon or the like.

That is, the reference direction referred to by the operation is defined on the user side. In general, the area required for moving the actuator is reduced by appropriate geometric transformation and time integration, but it requires skill because the semantic relationship between the input and the result is apart. However, once the conversion rules are mastered, a good input operation that does not depend on the position and orientation of the user can be performed thereafter. For ball-type data input devices, trackballs, tablets, and optical mice, the point that the coordinate axes of the operating system can be changed depending on the direction in which the device is placed is a base-on user type. It can be said, however, that this change is not easy, so we positioned it as an intermediate between the base-on-world type.

The positioning of the PDDM system according to the first embodiment will be described. In FIG. 7, there is a proposal method (palmtop type).
Of PDDM.

There are no icons in the operating environment of PDDM, and the object in the display area is directly focused on in an appropriate manner. This means that the environment is of the overlap type. Prior to the operation, the coordinate system for inputting the rotation or movement operation is always set in accordance with the coordinate system defined in front of the user in order to move the user to an appropriate viewpoint position in the local work space. Done. From this viewpoint, the PDDM according to the first embodiment
Can be said to be a base-on-user and overlap type during this classification. Therefore, realization of operation input using an intuitive operation environment and an operation device coordinate system belonging to the user is expected.

The configuration of the prototype PDDM according to the first embodiment will be described. FIG. 8 shows a PD according to the first embodiment.
It is a figure which shows the prototype of the PDDM manufactured in order to verify the concept of DM. The PDDM prototype in FIG. 8 corresponds to the virtual object operation device in FIG.

Referring to FIG. 8, PD according to the first embodiment
In the prototype of the DM, a small LCD 1 of 4 inches (weight 350 g) was used as the small display 1 in the display system.
f) is used. On the back surface, two push button switches that are operated by the fingertip of the left hand when the user grips are attached. In the prototype, the search mode and the grip mode are switched by the push button operation. Monitor position and orientation information
It is measured by a magnetic sensor (Fastrack ^™ ) attached to the main body. Electromagnetic noise generated from the liquid crystal monitor has some influence on the measurement by the magnetic sensor, but is removed by a digital low-pass filter.

The PDDM has active rotating shafts 3a, 3b, 3c.
, And the active rotation shafts 3a, 3b, 3c are composed of ultrasonic motors. That is, the PDDM employs a three-degree-of-freedom force display using an ultrasonic motor (Ultra Sonic Motor: hereinafter referred to as "USM") as an actuator. The USM is a motor that can control the holding torque and the rotation torque with a single unit, and has features such as not generating magnetic noise that is problematic during operation. An angle detecting mechanism is incorporated in the joint portion (the active rotation shafts 3a, 3b, 3c), so that force sense feedback is possible.
A small display 1 (liquid crystal monitor) at the tip of the manipulator 3 via a three-axis universal joint 24
, And is designed so that an operation reaction force acts on the center of gravity regardless of the attitude of the small display 1 (liquid crystal monitor).

In the configuration of this prototype, an operation reaction force of about 3 Newtons (N) can be output to the PDDM gripping section in three axial directions. Further, a contact sensor is arranged on the operation switch section of the small display 1 (monitor), and when the holding of the small display 1 (monitor) is detected, the maximum holding torque is applied to the USM to fix the mechanism section. Therefore, the small display (monitor) is stopped at the position where the user finally released the small display 1 (monitor). It is also expected to help reduce fatigue of the arm due to continuous operation of the force display.

FIG. 9 is a diagram showing the overall configuration of the PDDM system according to the first embodiment. Referring to FIG. 9, the PDDM system according to the first embodiment includes a virtual object operation device including a small display 1 and a manipulator 3,
Background display 25, small display position measurement sensor 27, head position measurement sensor 29, magnetic position measurement device 31, ultrasonic motor driver 33, control computer 35, first image generation computer 37, and 2 image generation computers 39.

The first image generation computer 37, the second image generation computer 39 and the control computer 35 are connected by Ethernet (Ether Net). The magnetic position measuring device 31 and the second image generating computer 39 are RS232C
Connected by cable.

Information from the manipulator 3 including the USM and other control computers 35 for processing input / output (I / O) and magnetic sensors (position measurement sensor 27 for small display and sensor 29 for head position measurement) are transmitted to The processing is performed by the first image generation computer 37 and the second image generation computer 39. A processing loop including a magnetic sensor (small display position measurement sensor 27 and head position measurement sensor 29) and a haptic system is processed at about 100 Hz, and image generation by computer graphics (CG) is separate from this processing loop. It becomes a real-time animation.

The ultrasonic motor driver 33 drives the USM included in the manipulator 3.
The control computer 35 controls the ultrasonic motor driver 33. The small display position measurement sensor 27 is for measuring the position of the small display 1 and uses a magnetic sensor. The head position measuring sensor 29 is for measuring the position of the head of the operator who operates the small display 1, and uses a magnetic sensor.

The classification relating to the viewpoint of the operation environment will be described. As the operation method after linking the target object to the local work space, four methods can be proposed depending on the method of reflecting the operation input to the object and the movement of the display image.
In each case, there can be proposed a method in which a part of the displayed video that is linked to the movement of the PDDM is set as a target object, a method in which the part is used as a background, and two methods in between.

The characteristics of each method are described below as “input operation reflection method”, “virtual camera viewpoint position and direction”,
“Matching of operation by rotation” and “Matching of display after opening” will be described.

The fixed view in background (hereinafter, referred to as “FVB”) will be described.

The movement of PDDM is directly reflected on the object. The virtual camera is always maintained in the geometric relationship between the PDDM and the background when grasped. Therefore, the background is always fixed, and the object may go out of the field of view with the operation. In addition, the operation direction and the moving direction of the object are deviated by the rotation operation, and the gaze directions do not match after the object is released. The virtual camera is for inputting an image on a large display (for example, the rear display 25 in FIG. 9).

Semi-Fixed View in
Object: hereinafter, referred to as “SFVO”).

The movement of PDDM is directly reflected on the object. The virtual camera translates the geometrical relationship between the PDDM and the object when it is grasped, with respect to only the positional relationship. Therefore, the object is always displayed at the center of the screen, but the appearance changes, and the background changes with the movement. Due to the rotation operation, the operation direction and the display direction of the object are shifted, and the line of sight does not match after the object is released.

The object fixed A (Object Centered Ma)
gic Hand: hereinafter referred to as “OCMH”).

The movement of the PDDM is reflected in the movement of the object by the magic hand metaphor. Therefore, the geometric relationship between the PDDM and the object when grasped is always maintained. The object is always displayed at the center of the screen in the posture at the start of grasping, and the background changes with movement and rotation.
However, regarding the rotation, since the camera position changes around the object position, there is no contradiction between the operation direction and the display direction of the object by the rotation operation. However, the gaze directions do not match after the object is released.

Object Fixation B (Camera Centered Ma
gic Hand: hereinafter, referred to as “CCMH”).

The movement of the PDDM is reflected in the movement of the object in the metaphor of the magic hand in the same manner as in the OCMH system. However, the rotation changes the object position around the camera position. Therefore, the operation direction does not deviate from the display direction of the object due to the rotation operation, and the line-of-sight direction matches after the object is released.

FIG. 10, FIG. 11, FIG. 12, FIG. 13, FIG.
The motion of the target object when the same operation is performed in the above-described four-operation method, the motion of the virtual camera that captures an image of the virtual space displayed on the PDDM (small display), and the The state of appearance change will be described.

FIG. 10 is a diagram showing an initial state of a small display, a virtual camera, and an object. FIG.
(A) shows the positional relationship between the first object 43, the second object 45, the virtual camera 41, and the small display 1. FIG. 10B shows an image on the small display 1 in the positional relationship shown in FIG.

Referring to FIG. 10A, first object 43 and second object 43 are displayed on a large display (not shown).
Object 45 is represented. Virtual camera 41
Is inputting a first object 43 and a second object 45 on the large display. The small display 1 is also facing the first object 43 and the second object 45 on the large display.

By rotating the small display 1 around the A1 axis or moving it in the direction of arrow A2,
The second object 45 can be rotated or translated. Hereinafter, such an operation of translating and rotating the second object 45 will be referred to as a “gripping operation”. In addition, small display 1
Although not shown, it is assumed that the manipulator 3 shown in FIG.

The arrow a indicates the direction vector of the small display 1 before the holding operation. The arrow b indicates the direction vector of the virtual camera 41 before the grip operation. Small display 1
On the upper side, the first object 43 or the second object 45 input by the virtual camera 41 is displayed. FIG. 10 shows an initial state.
No gripping operation was performed.

FIG. 11 is a diagram showing a positional relationship between an object, a virtual camera, and a small display and an image on the small display in the FVB system. FIG.
(A) shows the positional relationship between the first object 43, the second object 45, the virtual camera 41, and the small display 1 after the gripping operation. FIG. 11B shows an image on the small display 1 after the gripping operation. FIG.
The same reference numerals are given to the same parts as 0, and the description thereof will be omitted as appropriate.

Referring to FIG. 11A, small display 1 shown by a broken line shows an initial state, and small display 1 shown by a solid line shows a state after a gripping operation. Arrow c
Indicates a direction vector of the small display 1 after the gripping operation. Arrow d indicates the direction vector of the virtual camera after the gripping operation.

When the small display 1 is translated in the direction of arrow A2 and rotated about the A1 axis, the second object 45 is also translated in the direction of arrow B2 and rotates about the B1 axis. In this case, the movement of the small display 1 is not reflected on the virtual camera 41. That is, the virtual camera 1 remains in the initial state.

FIG. 12 is a diagram showing a positional relationship between an object, a virtual camera, and a small display and an image on the small display in the SFVO system. FIG.
(A) shows the first object 43 after the gripping operation,
3 shows the positional relationship between the object 45, the virtual camera 41, and the small display 1. FIG. 12B shows an image on the small display 1 after the holding operation. Note that FIG.
The same reference numerals are given to the same parts as those described above, and the description thereof will be appropriately omitted.

Referring to FIG. 12A, the small object 1 is translated in the direction of arrow A2 and is rotated about the axis A1, so that the second object 45 is also translated in the direction of arrow B2. Then, it rotates around the B1 axis.

In this case, the virtual camera 41 reflects only the parallel movement of the small display 1 and does not reflect the rotation. That is, the virtual camera 41 moves only in parallel with the small display 1.

FIG. 13 is a diagram showing a positional relationship between an object, a virtual camera, and a small display, and an image on the small display in the OCMH method. FIG.
(A) shows the positional relationship between the first object 43, the second object 45, the virtual camera 41, and the small display 1 after the gripping operation. FIG. 13B shows an image on the small display 1 after the holding operation. Note that FIG.
The same reference numerals are given to the same parts as those described above, and the description thereof will be appropriately omitted.

Referring to FIG. 13A, by moving the small display 1 in the direction of arrow A2 and rotating it around the A1 axis, the second object 45 also moves in the direction of arrow B2.
And rotates about the B1 axis.

In this case, the virtual camera 41 translates and rotates around the second object 45.

FIG. 14 is a diagram showing a positional relationship between an object, a virtual camera, and a small display and an image on the small display in the CCMH method. FIG.
(A) shows the positional relationship between the first object 43, the second object 45, the virtual camera 41, and the small display 1 after the gripping operation. FIG. 14B shows an image on the small display 1 after the gripping operation. FIG.
The same parts as those in 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

Referring to FIG. 14A, virtual camera 41
Reflects the parallel movement of the small display 1 in the direction of the arrow A2 and the rotation about the A2 axis. That is, the virtual camera 41 also translates and rotates integrally with the small display 1.

The second object 45 is a virtual camera 4
1 and the small display 1 as a center, translates in the direction of the arrow B2, and rotates around the B2 axis. In other words, it can be considered that the second object 45 is captured by the virtual camera 41 and the magic hand extending from the small display 1.

The results of FIGS. 11 to 14 are summarized. The FVB, SFVO, and OCMH systems achieve the same results, but display different images. CC
In the MH method, unlike the other three methods, the line of sight (direction) vector of the virtual camera 41 always coincides with the direction vector of the small display 1. The following evaluation experiments were conducted on the operability of these methods.

First, the experimental environment will be described. An operability evaluation test of the PDDM system according to the first embodiment is performed using the prototype system described above. Here, the purpose is to discuss how the difference in the design of the operation system affects the workability, particularly with the same concept, and to obtain a selection standard of the operation method in the subsequent development.

As a task (work), a work for arranging a cube arranged in an arbitrary posture at a position in front of the subject in an arbitrary posture as quickly as possible at a target position displayed as a polyline is set, and the task (work) completion time and work are set. Evaluate by accuracy. The distance to the target is about 50c
With m, the initial position is the same for each trial, but the attitude is different for each trial. The experimental procedure consisted of four methods (FVB, SFV
O, OCMH, and CCMH) are divided into two sets, and a comparative test is performed for each combination.

FIG. 15 is a diagram for explaining the experimental procedure of the evaluation experiment. Referring to FIG. 15, 2 to be compared first
The two operation methods are explained, and the subject is allowed to freely operate (teaching) until the subject understands the operation. Next, the experiment proceeds in the order of practice phase 1, practice phase 2, and test phase. In each phase, two operation methods are presented in eight trials. This procedure is performed for all test subjects for each combination of operation methods. In addition, questionnaires regarding “operability”, “confirmation of work status”, and “intuitiveness of operation” were conducted during the breaks in each phase and after the end of the experiment to provide a reference when evaluating these objective data. .

The subject is a male 3 who regularly uses a computer.
Performing on the name. In each trial process, the position and orientation of the target and the state of button operation are recorded every 3 seconds and at the time of self-report of operation completion.

The experimental results will be described and the results will be discussed. Due to the restriction of the movable range of the PDDM, there is a large operation that cannot be completed by a single operation, and a switching operation of the grip, which is a subdivided operation, is observed. This operation indicates the timing of repeating the units of each phase of observation, operation, and confirmation. Analysis of the time required to complete a task (work) during each trial by the “time per operation” divided by the number of grip switching operations. As a result of the statistical test, a significant difference was found in the inter-subject factors in the test phase. However, a significant difference cannot be shown in the factors between operation methods.

The fact that the time required for one gripping operation is uniform irrespective of the operation method is presumed to be that the learning in the test phase has become substantially uniform among the tasks in the test phase after understanding each operation method. In addition, during the experiment, some subjects tended to stick to the work accuracy particularly as the learning progressed. This experiment focuses on more general object operation efficiency than on work accuracy, and the subject's self-reported work completion time must be modified by some criteria.

Here, when the work end position data in each trial in the test phase is analyzed by the subject and the operation method, a superior difference is found by a statistical test in the factors between the subjects, but in the factors between the operation methods, There is no significant difference. Therefore, a 95% confidence interval of the operation accuracy of each subject was estimated from all position data at the end of the trial obtained for each subject, and a state in which the target was within the allowable interval was regarded as task completion. “Modified task (work) completion time”
The explanation will proceed.

FIG. 16 shows a four-operation system (FVB, CCM).
H, SFVO, OCMH)
It is a figure which shows a completion time. The horizontal axis shows the four-operation method, and the vertical axis shows the average work completion time in the test phase. The hatched bars indicate the working time within the allowable error range, that is, the data after correction. That is, it is a time for the cube to enter the target position within a certain error range, instead of making the cube exactly enter the target position displayed by the polyline.

A simple black frame indicates the free working time, that is, the data before correction. In other words, it is time for the cube to fit exactly into the target position where the polyline is displayed. Arrow A indicates the result of subject A. Arrow B indicates the result of subject B. Arrow C indicates the result of subject C. The straight line (error bar) indicated by the hatched bar indicates the variation (sample error) in the experimental results.

There was a significant difference in the factors between the operation modes and the completion time of the correction task by the statistical test. In particular, the CCMH method requires much processing time compared to the other methods, but the OCMH method and the SFVO method, which are almost the same method, are different from the other two methods in terms of work time, differences between individuals, and differences between trials. Small.

FIG. 17 is a diagram quantifying the subject's impressions in the four-operation system. The a-axis shows four systems. b
The axis indicates the content of the questionnaire for the subject. That is, "intuition of operation", "operability", and "operation status confirmation". “Intuition of operation” is whether or not it is close to moving a real object. “Ease of operation” is the ease of operation, that is, the ease with which the target can be brought. “Confirmation of work status” is the ease with which a target can be grasped. On the c-axis, “intuition of operation”, “operability”, and “confirmation of work status” are evaluated by scores.

Referring to FIG. 17, it is apparent that there is a strong negative opinion regarding the operability and intuitiveness of the operation of the CCMH method. The CCMH is a system in which an object on an extension of a small display (monitor) in the direction of the viewpoint is grasped by a virtual manipulator and operated. Since this is equivalent to an example in which a camera is mounted at the tip of the manipulator and remote operation is performed, we expected that this method would be more intuitive in terms of grasping the situation and operation than in other virtual methods. However, in this result, the evaluation was subjectively and objectively lower than other methods.

When considering each of the FVB, SFVO, and OCMH systems, all of them subjectively give high evaluations to the OCMH system for operability. In this method, (1) the viewpoint at the time of operation changes drastically by moving the object,
It was predicted that the operation would be confused because of the features such as (2) the greatest inconsistency of the viewpoint at the time of release, and (3) the operation direction and the movement direction were preserved even by rotation. .

However, in the opinion of the subject, there was a report that a good operation feeling was obtained as if the environment were moved with the object fixed. This means that while searching, the operation is based on a reference frame fixed in space,
This suggests that the reference frame was completely operated on the user base from the space at the time of grasping, and that the mode change was not so conscious. In such an evaluation experiment, a problem concerning operability is particularly focused on by the PDDM, and a result is obtained in which the proposed four methods are compared and the superior difference is seen in the same operation input / output environment.

The above will be described together. In the first embodiment, PDDM as a virtual object operation device is proposed as an operation interface in the immersive communication conference system, and the positioning of the PDDM is clarified through feature classification of conventional display devices and operation devices. Next, a prototype PDDM was manufactured, and the proposed operation methods were compared with each other based on the results of simple object operation work, and promising operation methods were confirmed.

As described above, the virtual object operation device according to the first embodiment of the present invention has a grasping function, and is capable of moving a small display only to move a virtual object projected on the small display. Operation (action) can be easily performed.

In the virtual object operation device according to the first embodiment, a manipulator that generates a sense of weight (reaction force) according to the weight of the object represented by the virtual object is used.
Therefore, an operator who performs an operation (action) on the virtual object using the small display can feel the weight as if he / she actually holds the object.

In the virtual object operating device according to the first embodiment, the small display 1 and the manipulator are used. With such a simple configuration (mechanism), a feeling of weight (reaction force) can be generated, and Can feel the feeling of weight as if he actually held an object.

In the virtual object operating device according to the first embodiment, a three-dimensional object can be easily duplicated in a virtual space.

(Embodiment 2) Three-degree-of-freedom force sense presentation apparatus 1
In the virtual object operating device realized by the base, that is, one manipulator, the reaction force components in the directions of the three axes of space can be displayed, but the rotation components around each axis cannot be displayed. For example, referring to FIG. 1, a reaction force component in the D-axis, E-axis, and F-axis directions can be displayed, but a rotation component around the D-axis, E-axis, and F-axis cannot be displayed. This is because a three-degree-of-freedom haptic device (manipulator) is capable of presenting weight by gravity and presenting a shock force accompanying a collision, but is capable of performing an object (operation (action)). This means that when the object is regarded as a rigid body, it cannot express the rotational torque around the center of gravity that generates the rotational motion.

The rotation torque in the operation of the virtual object operation device itself is not sufficient for real operation (real feeling) in terms of operational feeling and gripping feeling in the operation of a rigid object (virtual object to be subjected to an operation (action)). It has the effect of reinforcing.
On the other hand, since an active mechanism is required for a total of six degrees of freedom in a total of a position (a component in the direction of the three axes of space) and a posture (a component of a rotation around the three axes of space), the virtual object operation device itself is generally very difficult. It is a complicated and large mechanism. The virtual object operation device according to the second embodiment of the present invention solves such a problem. That is, the virtual object operation device according to the second embodiment has a simple structure and aims to realize a force sense presentation function with six degrees of freedom.

FIG. 18 is a schematic diagram showing a virtual object operating device according to the second embodiment of the present invention. The same parts as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

Referring to FIG. 18, a virtual object operating device according to the second embodiment has three manipulators 101 and 10.
3,105, universal joints 107,109,
111 and the small display 1. The manipulator 101 includes a free rotation axis 101a and an active rotation axis 1
01b and 101c. The manipulator 103 includes a free rotation shaft 103a and active rotation shafts 103b and 103c.
including. The manipulator 105 includes a free rotation shaft 105a.
And 105b and 105c. Small display 1
Shows a virtual object.

Here, the manipulators 101, 103,
105 corresponds to the manipulator 3 of FIG. The free rotation axes 101a, 103a, 105a are shown in FIG.
Correspond to the active rotary shaft 3a. Active rotating shaft 101b, 1
Reference numerals 03b and 105b correspond to the active rotation shaft 3b in FIG. The active rotating shafts 101c, 103c, 105c are shown in FIG.
Correspond to the active rotating shaft 3c. The universal joints 107, 109, 111 correspond to universal joints (not shown) in FIG. Therefore, the manipulators 101, 103, and 105 correspond to the manipulator 3 in FIG.
Is similar to However, the active rotating shaft 3a in FIG.
In contrast to the ultrasonic motor, the free rotation shaft 1 shown in FIG.
01a, 103a, and 105a do not include ultrasonic motors.

Free rotation shafts 101a, 103a, 105a
Can rotate around the A-axis (the central axis of the cylinder). The active rotation axes 101b, 103b, and 105b can rotate around the B axis (the central axis of the cylinder). Active rotating shafts 101c, 10
3c and 105c can rotate around the C axis (the central axis of the cylinder). Universal joints 107, 109, 11
1 each have three degrees of freedom.

As described above, the virtual object operating device according to the second embodiment has a single axis (one degree of freedom) free rotation axis 101.
a, a three-degree-of-freedom manipulator 101 including a one-axis (one-degree-of-freedom) active rotation axis 101b and a one-axis (one-degree-of-freedom) active rotation axis 101c; A three-degree-of-freedom manipulator 103 including an active rotation axis 103b having one axis (one degree of freedom) and an active rotation axis 103c having one axis (one degree of freedom), and one axis (one degree of freedom).
Free rotation axis 105a, one axis (one degree of freedom) active rotation axis 105b, and one axis (one degree of freedom) active rotation axis 105c
And a manipulator 105 having three degrees of freedom.

That is, the virtual object operation device includes three manipulators having three degrees of freedom. The manipulator 101 and the small display 1 are connected via a universal joint 107 having three degrees of freedom. The manipulator 103 and the small display 1 are connected via a universal joint 109 having three degrees of freedom.
The manipulator 105 and the small display 1 are connected via a universal joint 111 having three degrees of freedom. The active rotation shafts 101b, 101c, 103b,
103c, 105b, 105c are the active rotating shafts 3 of FIG.
An ultrasonic motor similar to the ultrasonic motor used as a, 3b, 3c. Then, a reaction force is generated by the ultrasonic motor.

In the virtual object operating device having such a mechanism (configuration), the mechanism of the grip portion (connection portion to the manipulator) of the central small display 1 is simplified (the universal joints 107 to 111 are used). ), The mechanism weight (the weight of the grip portion and the small display) is reduced, and at the same time, the position (D-axis direction, E-axis direction, F-axis direction) and posture (rotation around D-axis, rotation around E-axis) Rotation and rotation around the F-axis). That is, a wide movable range with six degrees of freedom is obtained.

The three manipulators 101 to 10
5 active rotary shafts (ultrasonic motors) 101b, 1
Since 01c, 103b, 103c, 105b, and 105c act simultaneously, the D-axis, as compared with the case where one manipulator is used (compared with the output when three active rotary shafts (ultrasonic motors) are used), A large reaction force in the E-axis and F-axis directions and a reaction force (rotational force) around the D-axis, E-axis, and F-axis can be generated. The reaction force in the D-axis, E-axis, and F-axis directions is a reaction force accompanying the parallel movement of the small display 1. D axis, E
The reaction force (rotational force) about each of the axis and the F-axis is a rotational torque output such as tilting or twisting the small display 1.

As a problem of the virtual object operating device according to the second embodiment, the rotation axis is substantially 18 in total.
Because of the existence of the number, there is a mechanical singularity inside the movable range. The mechanical singularity refers to a spatial position where the degree of freedom decreases with the overlapping of the rotation axes. In other words, the position of the small display 1, that is, the manipulator 101
There are positions where the reaction force cannot be generated depending on the state of to 105, but those positions are mechanical singularities. Regarding this problem, by considering the arrangement of the manipulators 101 to 105, the small display 1 (manipulator 101
The problem can be solved by providing a mechanical singularity in an area that is not frequently used even within the movable range of (1) to (5).

Here, the fact that there are 18 rotation axes will be described in detail. One manipulator (force sense presentation device) has six rotation axes. That is, A axis, B axis, C axis,
These are the three rotation axes of the universal joint.
Since three manipulators having such six rotation axes are provided, the number of rotation axes of the entire virtual object operation device is 18 in total.

As described above, the virtual object operating device according to the second embodiment includes three manipulators 101 to 105 each having six degrees of freedom. And further, each of the three manipulators 101 to 105
Active rotary shaft 101 composed of an ultrasonic motor that generates a reaction force
b, 101c, 103b, 103c, 105b, 105
c.

As described above, with a simple configuration including three manipulators each including a free rotation axis and an active rotation axis and one small display, a force sense presentation function having six degrees of freedom can be realized, and operation of a virtual object can be realized. A corresponding reaction force can be generated.

That is, with a simple configuration (structure and mechanism),
By moving the small display 1, a person who performs an operation (action) on the virtual object can obtain a real feeling and a texture as if he / she were actually operating the object.

For example, the difference between the case of folding paper and the case of bending sheet metal can be felt by the magnitude of the reaction force when the small display 1 is operated. Further, the virtual object operation device according to the second embodiment includes the configuration of the virtual object operation device according to the first embodiment, and thus has the same effects as the virtual object operation device according to the first embodiment. For example,
In a state where the operator of the small display 1 takes in the object on the small display 1, the operator can feel the weight of the object.

In the virtual object operating device according to the second embodiment, the procedure (method) of performing an operation (action) on the virtual object is the same as that of the virtual object operating device according to the first embodiment. For example, a procedure (method) for rotating the virtual object or bending a part of the virtual object is the same as that of the virtual object operation device according to the first embodiment.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing a virtual object operation device according to a first embodiment of the present invention.

FIG. 2 is a schematic diagram showing a virtual object operator using a virtual object operation device.

FIG. 3 is a schematic diagram showing the use of a virtual object operating device in the presence communication conference system;

FIG. 4 is a diagram showing a small display on which a specific place of a virtual object is projected.

FIG. 5 is a diagram showing an assumed use form when the PDDM according to the first embodiment is adopted in the presence communication conference system.

FIG. 6 is a diagram showing a classification of an information display system (information display means).

FIG. 7 is a diagram showing classifications of operation input devices (operation input means).

FIG. 8 is a diagram showing a prototype of a PDDM manufactured to verify the concept of the PDDM according to the first embodiment.

FIG. 9 is a diagram showing an overall configuration of a PDDM system according to the first embodiment.

FIG. 10 is a diagram showing an initial state of a small display, a virtual camera, and an object.

FIG. 11 is a diagram showing a positional relationship between an object, a virtual camera, and a small display and an image on the small display in the FVB method.

FIG. 12 is a diagram showing a positional relationship between an object, a virtual camera, and a small display and an image on the small display in the SFVO method.

FIG. 13 is a diagram showing a positional relationship between an object, a virtual camera, and a small display and an image on the small display in the OCMH method.

FIG. 14 is a diagram showing a positional relationship between an object, a virtual camera, and a small display and an image on the small display in the CCMH method.

FIG. 15 is a diagram for explaining an experimental procedure of an evaluation experiment.

FIG. 16 shows four operation modes (FVB, CCMH, SFVO,
It is a figure which shows the average work completion time in OCMH).

FIG. 17 is a diagram quantifying the subject's impressions in the four-operation method.

FIG. 18 is a schematic diagram illustrating a virtual object operation device according to a second embodiment of the present invention.

[Explanation of symbols]

1 Small display (monitor) 3, 101, 103, 105 Manipulator 101a, 103a, 105a Free rotation axis 3a, 3b, 3c, 101b, 101c, 103b, 1
03c, 105b, 105c Active rotation axis 5 Operator 7 Speaker 9a, 9b Screen support 11a, 11b Screen 13 Person (virtual person) 15 Object (virtual object) 17, 19, 21 Meeting participants 23a, 23b Object 24, 107 , 109, 111 Universal joint 25 Background display 27 Position measurement sensor for small display 29 Head position measurement sensor 31 Magnetic position measurement device 33 Ultrasonic motor driver 35 Control computer 37 First image generation computer 39 2 image generation computer 41 virtual camera 43 first object 45 second object

──────────────────────────────────────────────────続 き Continuation of front page (51) Int.Cl. ⁶ identification code FI G06T 17/40 G06F 15/62 350K (58) Field surveyed (Int.Cl. ⁶ , DB name) G09B 9/00 B25J 3 / 00 G06F 17/00 G06T 17/40

Claims (9)

(57) [Claims] 1. A virtual object operation device for operating a virtual object in a virtual world in the real world, wherein the virtual object is displayed by moving a display device on which the virtual object is displayed. , Virtual object operation device. 2. The virtual object operation device according to claim 1, wherein when the display device is moved, a feeling of weight is generated according to the weight of the object represented by the virtual object. 3. The virtual object operation device according to claim 1, wherein when the display device is moved, a reaction force is generated according to an operation of the virtual object. 4. A virtual object operation device for operating a virtual object in a virtual world in the real world, comprising: a display device for displaying the virtual object; and support means for supporting the display device. A virtual object operation device that operates the displayed virtual object by moving the display device. 5. The virtual object operating device according to claim 4, wherein said supporting means includes a heavy feeling generating means for generating a heavy feeling according to the weight of the object represented by said virtual object. 6. The virtual object operating device according to claim 5, wherein said heavy feeling generating means is an ultrasonic motor. 7. The virtual object operation device according to claim 4, wherein the support unit includes a reaction force generation unit that generates a reaction force according to an operation of the virtual object when moving the display device. 8. The support means includes three independent independent support means for supporting the display device, and each of the independent support means generates a reaction force according to an operation of the virtual object. The virtual object operation device according to claim 7, comprising the reaction force generation unit. 9. The virtual object operating device according to claim 7, wherein said reaction force generating means is an ultrasonic motor.