JP3344499B2 - Object operation support device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to relates the object displayed in the three-dimensional virtual space, which is configured on a display object operation support equipment to provide support when operated by a hand of an operator virtual.

[0002]

2. Description of the Related Art The technique of displaying an object in a three-dimensional virtual space formed on a display and operating the object by inputting a command by an operator is applied to a CAD system, a simulator system for driving training, and the like. ing. In this case, for example, a virtual hand is displayed in the three-dimensional virtual space, and the display of the hand is moved by a command input by the operator to perform an operation such as moving or grasping the object to be operated.

[0003] However, it is difficult for an operator to grasp the sense of distance from the position where the hand is displayed on the display to the object with the current technology, and thus it is not easy to move the hand to the position of the object and grasp it. Therefore, it takes time to operate the object, and the burden on the operator is large.

[0004]

As described above, in the prior art, it is difficult to capture a sense of distance from a display position of a hand or the like displayed in a three-dimensional virtual space on a display to an object to be operated. There is a problem in that the operation of the object is not easy, the operation takes a long time, and the burden on the operator is large.

An object of the present invention, the operation of the object is simple, is to provide an object operation support equipment that can reduce the burden of the operator.

[0006]

In order to solve the above-mentioned problems, according to the present invention, an operation position by an operator's hand or the like displayed in a three-dimensional virtual space formed on a display is fixed. The operation is performed by moving the operation target object toward the operation position, and the operation is performed as necessary, and then the operation target object is returned to the original position.

That is, the object operation support apparatus according to the present invention comprises: command input means for inputting a command for operating an object; and position data for inputting position data of an operation position by the operator in a three-dimensional virtual space. Input means, command determining means for determining a command input by the command input means, and, when the command determined by the command determining means is an object return command, the object is returned to a previously registered original position. characterized in that a object return means for returning.

The object operation support device according to the present invention is displayed in a three-dimensional virtual space formed on a display and has the shortest distance to an operation position from among a plurality of objects defined as operable. An object determining means for determining an object as an operation target object is further provided.

Further, in the object operation support device according to the present invention, the object returning means controls the moving speed when the operation target object is returned along the linear path set between the operation position and the operation target object. An object moving speed setting means for setting is further provided.

The object moving speed setting means may set the moving time to a fixed time of about 1 to 2 seconds at which the operator feels that the operation feeling is good, or arbitrarily sets it according to the operator's preference. You may be able to.

[0011]

[0012]

According to the present invention, since an operation target object comes close to an operation position by an operator's hand or the like by a predefined command input, operation can be reliably performed even if the exact distance between the operator and the operation target object is not known. It can be performed. Therefore, the burden on the operator is reduced, and the work efficiency in the three-dimensional virtual space displayed on the display is improved.

[0013]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the overall configuration of a system employing the object operation support device of the present invention.

In FIG. 1, a command input unit 1 is used by an operator to perform operations such as moving an object displayed in a three-dimensional virtual space formed on a display 14 and returning the object to a previously registered original position. For example, a data glove, a keyboard, a voice input device, or a pointing device such as a mouse is used.

The position data input unit 2 inputs position data of a position where an operator operates an object in a three-dimensional virtual space (hereinafter, referred to as an operation position), in other words, a display position of a virtual hand or the like displayed as a graphic. For example, a magnetic sensor or an ultrasonic sensor mounted on an operator's hand is used.

The command judging section 3 judges the command input by the operator from the command input section 1 and outputs an object moving command or an object returning command.
Is input to the object determination unit 6.

When receiving an object movement command or an object return command from the command determination unit 3, the coordinate determination unit 4 determines the coordinates of the operation position in the three-dimensional virtual space from the position data output from the position data input unit 2. The coordinate data of the operation position is output to the object distance calculation unit 7 as the determination result.

The object coordinate storage unit 5 is a memory in which the initial registration coordinate data and the current coordinate data of each object are stored in advance together with a flag indicating whether the object is a movable object. Is read out.

The object determination unit 6 includes an object movement command or an object return command output from the command determination unit 3, initial registration coordinate data of each object read from the object coordinate storage unit 5,
Display 1 based on the current coordinate data and flag
In step 4, an object that is displayed and defined as movable is searched for and determined, and the initial registration coordinate data and the current coordinate data of the object are output to the object distance calculation unit 7.

The object distance calculation unit 7 calculates the coordinate data of the operation position in the three-dimensional virtual space output from the coordinate determination unit 4 and the current coordinate data of the movable object output from the object determination unit 6. The distance between the operation position designated by the operator and each object is calculated, and the object having the shortest distance is determined as the operation target object. The calculation result and the determination result are output to the object speed calculation unit 9. You.

The object moving time storage unit 8 is a memory for storing moving time data indicating a predetermined moving time of the object when the object to be operated is moved, and the contents thereof are read out to the object speed calculating unit 9. It is.

The object speed calculator 9 outputs the operation position indicated by the operator output from the object distance calculator 7, the distance between the nearest object, and the movement time data read from the object movement time storage 8. Is used to calculate the moving speed when the object to be operated is moved, and the calculation result is output to the object moving / return processing unit 10.

The object movement / return processing unit 10 sets a linear path between the object closest to the operation position, that is, the object to be operated, and the operation position, and transfers the object to the operation position by the object speed calculation unit. 9 to perform a process of moving at the moving speed obtained by the step 9 or converting the image to the original position. The processing result is output to the object display data generating unit 12.

The object shape storage unit 11 stores the shape of each object displayed in the three-dimensional virtual space.
2 is read.

The object display data generation unit 12 generates display data of the object based on the processing result of the object movement / return processing unit 10 and the data read from the object shape storage unit 11 and outputs the display data to the display control unit 13. .

The display controller 13 displays the display data generated by the object display data generator 12 on the display 1.
4 is a display control unit that performs control for display on the display unit 4. The display control unit 13 also generates display data of a display object such as a virtual hand displayed at an operation position by the operator.

Next, as a specific application example of the object operation support device of this embodiment, CAD (computer support design)
The system will be described as an example.

As a command input method of the command input unit 1 in FIG. 1, there are a key input by a keyboard, a menu selection by a mouse, and a gesture input by using a data glove as described above. It is assumed that a gesture input is used.
2A and 2B show an example of gesture input using a data glove. FIG. 2A shows a case where an operation target object is moved to an operation position, and FIG. 2B shows a case where an object moved to the operation target is converted to an original position. Each is shown.

The method of inputting the position data of the operation position by the operator in the position data input unit 2 in FIG. 1 is based on a magnetic sensor as shown in FIG.
Further, a CRT display is assumed as the display 14.

FIG. 4 is a display example of a three-dimensional virtual space of CAD created by a computer and configured on the display 14, in which several objects A, B, and C are displayed. These objects A, B, and C are defined in advance as those that can be directly moved by the operator and those that cannot be moved. In this example, the objects A and B can be moved and the object C cannot be moved. Object. The reason why such a definition is made is that, for example, an object to which an operation such as processing is to be added and an object which the user does not want to move from the current position after all the operations such as processing are completed are displayed positions (operation positions) of the operator's hand. This is to prevent the operation from being complicated by moving to an object that the user does not want to move when the object is located at the same distance from the object.

FIG. 5 is a diagram showing the contents of the object coordinate storage unit 5 in FIG. 1 in this case. The data of the initial registration coordinates and the current coordinates of each of the objects A, B, and C and the data of each object are A movement flag indicating whether the object is movable or not is stored.

Now, assume that one of the movable objects A and B is operated. For example, consider a case where an operator grasps an object A with a hand graphically displayed in a three-dimensional virtual space and intends to perform processing. In this case, according to the related art, the operator moves the hand display to the object A, reaches the object A, and tries to grasp the object A. However, it is difficult to grasp the depth and distance in the current display of the three-dimensional virtual space on the display, and this operation has a problem that it is difficult to grasp and pull the object before the original processing operation. is there.

On the other hand, by performing a gesture as shown in FIG. 2A defined in advance with the data glove of the command input unit 1 toward the object A to be operated in accordance with the present invention, FIG. When the object A is moved toward the display position (operation position) of the virtual hand of the operator as shown in (2), the work of grasping the object A becomes very simple.

Further, when it is desired to return the processed object A to the original position registered in advance after moving the object A once to the operation position and drawing the same, the data glove of the command input unit 1 is used to separately define the object A shown in FIG. By performing the gesture as shown in FIG. 6B, by returning as shown in FIG. 6B, the efficiency of the machining operation is further promoted.

When the movement or return of the operation target object A is performed by simple coordinate conversion, the object A
May suddenly disappear from the display screen of the display 14, and confusion such as where it has been moved or returned may not be known. Therefore, as a path for movement or return, a straight path connecting the operation position and the original position of the object A with a straight line is set, and the movement or return of the object A is performed along the straight path at a preset time. , Such confusion can be avoided.

Next, a procedure for moving an object to an operation position will be described with reference to a flowchart shown in FIG.

First, an object movement command is input from the command input unit 1 by a gesture as shown in FIG. 2A, and when the command determination unit 3 determines this, the object determination unit 6 The object coordinate storage unit 5 determines whether there is a movable object among the objects displayed in the virtual space.
(S101). When the command determination unit 3 determines the input of the object movement command, the position data input unit 2 sends the coordinates of the hand in the three-dimensional virtual space from the coordinate determination unit 4, that is, the coordinates H (Xh, Yh, Zh) of the operation position. Is read.

If it is determined in S101 that there is a movable object, the coordinates of each object determined to be movable and the coordinates H (Xh, Yh, Z) of the hand are determined.
h) is calculated by the object distance calculator 7 (S102). In this example, the movable objects are A and B. For example, the coordinates (Xa, Ya, Za) of the object A and the coordinates H (Xh, Yh, Z) of the hand A
The distance Da between h) is calculated by the following equation (1).

Da = {(Xa−Xh) ² + (Ya−Yh) ² + (Za−Zh) ² } ^1/2 (1) Further, the object distance calculation unit 7 calculates the distance D based on the distance D calculated in S102. Then, an object having the minimum D (the object A in this example, the minimum distance is D = Da) is determined, and a straight line L passing through the coordinates of the object A and the coordinates H of the hand is expressed by the following equations (2) to (5). (S103).

X1 = t (Xa−Xh) + Xa (2) Y1 = t (Ya−Yh) + Ya (3) Z1 = t (Za−Zh) + Za (4) where −1 ≦ t ≦ 0 (5) Next, in the object moving speed calculation unit 9, the expression (1)
Da and object movement time storage unit 8 obtained in
The moving speed V is calculated from the moving time T stored in the following equation (6) (S104).

V = Da / T (6) Then, in the object movement / return processing unit 10,
By changing the coefficient t of the equation of the straight line L obtained by the equations (2) to (5) and converting the coordinates of the object A, the object A is displayed in the three-dimensional virtual space on the display 14. A process of moving the hand to the position (operation position) of the coordinates H (Xh, Yh, Zh) of the hand at the speed V obtained by Expression (6) is performed (S105).

By supplying the processing result of the object movement / return processing unit 10 to the display 14 via the object display data generation unit 11 and the display control unit 12, the display as shown in FIG. 6A is performed. That is, the object A moves (becomes drawn) from the initial registration coordinates A (Xa, Ya, Za), which are the original positions indicated by the broken lines, to the coordinates H (Xh, Yh, Zh), which are the operation positions indicated by the solid lines.

After moving the object A to be operated to the operation position as described above, the object A is subjected to some processing operation. Then, if necessary, the object A for which the processing operation has been completed is returned to the original position where it was initially registered.

Next, a procedure for returning the processed object from the operation position to the original position will be described with reference to the flowchart shown in FIG.

First, an object return command is input from the command input unit 1 by a gesture as shown in FIG. 2B, and when the command determination unit 3 determines this, the object determination unit 6 It is determined with reference to the object coordinate storage unit 5 whether or not any of the objects displayed in the virtual space has an object whose current coordinates are different from the initial registration coordinates (S20).
1). When the command determination unit 3 determines the input of the object return command, the position data input unit 2 sends the coordinates of the hand in the three-dimensional virtual space from the coordinate determination unit 4, that is, the coordinates H (Xh, Yh, Zh) of the operation position. Is read.

Here, if it is determined in S201 that there are objects whose current coordinates are different from the initial registration coordinates, the current coordinates of each object and the hand coordinates H (X
h, Yh, Zh) is calculated by the object distance calculator 7 (S202). In this case, the only object whose current coordinates are different from the initial registration coordinates is A, and the current coordinates A '(Xa', Y
The distance Da between a ', Za') and the coordinates H (Xh, Yh, Zh) of the hand is calculated by the following equation (7).

Da = {(Xa′−Xh) ² + (Ya′−Yh) ² + (Za′−Zh) ² } ^1/2 (7) Further, the object distance calculation unit 7 calculates the distance calculated in S202. Based on D, the object having the minimum D (the object A in this example, the minimum distance is D = Da) is determined, and the current coordinates A ′ (Xa ′, Y) of the object A are determined.
a ', Za') and the initial registration coordinates A (Xa, Ya, Za)
And the straight line L 'passing through both coordinates are expressed by the following equation (8)
It is obtained by (12) (S203).

D ′ = {(Xa′−Xa) ² + (Ya′−Ya) ² + (Za′−Za) ² ｝ ^1/2 (8) X1 ′ = t (Xa′−Xa) + Xa ′ ... (9) Yl '= t (Ya'-Ya) + Ya' ... (10) Zl '= t (Za'-Za) + Za' ... (11) where -1≤t≤0 ... (12) In the object moving speed calculation unit 9, the expression (8)
D ′ and object moving time storage unit 8
The moving speed V 'is calculated from the moving time T stored in the following equation (13) (S204).

V ′ = D ′ / T (13) Then, in the object movement / return processing unit 10,
By changing the coefficient t of the equation of the straight line L 'obtained by the equations (9) to (10) and transforming the coordinates of the object A, the object A is displayed in the three-dimensional virtual space on the display 14 at present. Coordinates A '(Xa', Ya ', Z
a) is moved from the position a ') to the position of the initial registration coordinates A (Xa, Ya, Za) at the speed V' determined by the equation (13) and returned (S205).

The processing result of the object movement / return processing unit 10 is converted into the object display data generation unit 1 in the same manner as described above.
By supplying the data to the display 14 via the display control unit 1 and the display control unit 12, a display as shown in FIG. That is, from the position (operation position) of the current coordinates A (Xa ', Ya', Za ') indicated by the broken line of the object A after the machining operation, the initial registration coordinates A (X
a, Ya, Za) (returned).

Next, as another application example of the object operation support device according to the present invention, virtual reality (V
R) An operation when applied to a driving training simulator system using technology will be described.

In this case as well, the command input method in the command input unit 1 is a gesture input method using a data glove as a device, and the position data input method by the operator in the position data input unit 2 is a magnetic sensor. I do. On the other hand, a head mounted display is used as the display 14.

The driving training simulator system using virtual reality is based on an operation space,
The trainee simulates the operation situation and the operation target, and trainees then perform training on the actual spatial arrangement of the site, trouble situations that can occur on the site, and the operation method of the operation object. In the prior art, even in such a simulator system, it is difficult to perform an operation of an object in a three-dimensional virtual space, for example, an operation of grasping a lever, a door knob, or the like. It hinders the achievement of the original purpose of learning.

Therefore, for example, when trying to operate the lever, it is difficult to grasp the lever because the trainee's sense of distance between the virtual hand and the lever graphically displayed in the three-dimensional virtual space on the display cannot be grasped. In such a case, if the lever, which is the object to be operated, is moved to the operation position, that is, the display position of the hand, in the same manner as described above, the operation of grasping the lever can be easily performed. If the lever is returned to the original registration position after the operation of the lever is completed, the relationship between the objects in the three-dimensional virtual space does not collapse.

The present invention can be implemented with various modifications as follows. For example, in the embodiment, the moving time of the object is fixed to a predetermined value. However, if this value is set to a value such as 1 to 2 seconds that gives the operator a good operational feeling, a better operational feeling can be given to the operator. . Further, this time can be optionally set by the operator.

When an object is selected, the display form of the selected object is made different from that of another object (for example, the color is changed or blinked), so that the operator can determine which object is selected. It can also be easy to recognize.

Further, in the embodiment, as a command input method, there are gesture input using a data glove,
Although a method using a magnetic sensor is used as a method for inputting the position data of the operator, the present invention can be implemented by a method using another input device. As for the display, a device other than the CRT and the head mounted display can be used.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an object operation support device according to an embodiment of the present invention.

FIG. 2 is a view for explaining an input method when an object is moved and returned by a command input unit in FIG. 1;

FIG. 3 is a diagram showing an example of a position data input unit in FIG.

FIG. 4 is an exemplary view for explaining an operation state of an object in the embodiment.

FIG. 5 is a diagram showing an example of the contents of an object coordinate storage unit in FIG. 1;

FIG. 6 is a diagram showing a display example when an object is moved and returned on the display in FIG. 1;

FIG. 7 is a flowchart showing the procedure of object movement / return processing in the embodiment.

FIG. 8 is a flowchart showing the procedure of an object return process in the embodiment.

[Explanation of symbols]

REFERENCE SIGNS LIST 1 command input unit 2 position data input unit 3 command determination unit 4 coordinate determination unit 5 object coordinate storage unit 6 object determination unit 7 object distance calculation unit 8 object movement time storage unit 9 object speed calculation Unit 10: Object movement / return processing unit 11: Object shape storage unit 12: Object display data generation unit 13: Display control unit 14: Display

Continuation of the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G06T 11/80 G06F 3/00 G06F 3/03 G06F 3/033 G06F 17/50

Claims (3)

(57) [Claims] An object operation support device for assisting an operator in operating an operation target object displayed in a three-dimensional virtual space configured on a display, wherein a command for operating the object is input. Command input means for inputting position data of an operation position by the operator in the three-dimensional virtual space; command determination means for determining a command input by the command input means; object operation support apparatus being characterized in that a object return means which is determined by the command determination means command to return to the pre-registered original position the object if it is of <br/> object return command . 2. An object which is displayed in a three-dimensional virtual space formed on the display and is defined as movable and which has the shortest distance from the operation position is selected as an object to be operated. claim 1 Symbol mounting object operation support apparatus characterized by comprising an object determining means for determining as an object. Wherein objects for setting the moving speed at the time of Kaesa return the object of the operation target in accordance with the set straight line paths between the object return means of the operation target and the operation position objects claim 1 Symbol mounting object operation support apparatus comprising the moving speed setting means.