JPH06103360A - Three-dimensional image display system - Google Patents

Abstract

PURPOSE:To ensure the support that can assure the similar effect to the shadow of a hand in an object pointing operation by displaying an auxiliary pointer between a detected object and a three-dimensional pointer to support the point ing actions of the latter pointer. CONSTITUTION:A visual point setting device 3 sets the position and the direction of the visual point that designates a range of the images to be outputted in a three-dimensional space which is controlled by a three-dimensional space information control part 2 of a system main body 7. A pointing object deciding part 4 decides an object that exists on an extended line in the direction pointed by a three-dimensional pointer and also is closest to this pointer. Then, an auxiliary pointer azimuth deciding part 5 decides the azimuth of an auxiliary pointer set at the center between the object decided by the part 4 and the three- dimensional pointer using the space information on the object decided by the part 4. An image display device 6 displays the graphic image data in the three- dimensional space in response to the visual point decided by the setter 3.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a three-dimensional space that is graphically displayed on a computer system.
The present invention relates to a three-dimensional image display system that supports a manual positioning operation for a three-dimensional pointer used for pointing a position or an object in space.

[0002]

2. Description of the Related Art Conventionally, in a three-dimensional space that is graphically displayed on an image display device, even when accompanied by binocular disparity information, the user is ambiguous in the stereoscopic effect and the sense of distance. Was difficult.

As a method for supporting these pointing operations, there is a method of displaying the distance between an object and a pointer in a three-dimensional space by a numeral or a graph ("Teleegistence Robot Simulator with Artificial Reality") IEICE Transactions D-II Vol.J75 D-II No2.pp.179-1
89, February 1992).

At a distance at which the object surface is likely to be touched by the hand, the shadow of the hand on the object surface is one of the important cues for the sense of distance in the real world. However, in the CG (computer graphics) representation, a very burdensome process is required to perform a display corresponding to the shadow of this hand, and it is difficult to use it when specifying a position.

Therefore, there is a demand for a support that can achieve the same effect as a shadow of a hand in an object pointing operation in a three-dimensional space by a process that is much lighter than a projection display.

[0006]

As described above, it is possible to provide the same effect as the shadow of the hand in the object pointing operation in the three-dimensional space by the processing which is much lighter than the projection display. What is desired is to provide a 3D image display system that can support the effect similar to the shadow of a hand in an object pointing operation in a 3D space by a process that is much lighter than the projection display. The purpose is to

[0007]

A three-dimensional image display system according to the present invention displays at least an image of an object, a three-dimensional pointer or the like using a three-dimensional space defined in advance, and the plurality of displayed images are displayed. Storage means for storing three-dimensional shape data defined in accordance with a coordinate system defining a three-dimensional space for the object, designating means for designating a three-dimensional position and direction in the three-dimensional space for the three-dimensional pointer, Existence on the extension line indicated by the three-dimensional pointer in the three-dimensional space according to the three-dimensional position and direction designated by the designating means and the three-dimensional shape data for each object stored in the storage means. Detecting means for detecting an object closest to the three-dimensional pointer, and the object detected by the detecting means and the three-dimensional pointer in the three-dimensional space. And a display means for displaying the auxiliary pointers to assist an indication of the three-dimensional pointer between the pointer.

The three-dimensional image display system of the present invention displays at least images of objects and three-dimensional pointers using a three-dimensional space defined in advance. Storage means for storing three-dimensional shape data defined in accordance with a coordinate system that defines, a designation means for designating a three-dimensional position and direction in the three-dimensional space for the three-dimensional pointer, and designation by this designation means. The three-dimensional position and direction and the three-dimensional shape data of each object stored in the storage means exist on the extension line indicated by the three-dimensional pointer in the three-dimensional space, and are the most three-dimensional. Detecting means for detecting an object close to the pointer, and calculating intermediate coordinates in the three-dimensional space between the object detected by the detecting means and the three-dimensional pointer. Calculating means for, and the coordinates of the three-dimensional space calculated by the calculation means, and a display means for displaying the auxiliary pointers to assist an indication of the three-dimensional pointer.

[0009]

The present invention displays an image of at least an object, a three-dimensional pointer or the like using a three-dimensional space defined in advance, and uses a coordinate system that defines a three-dimensional space for the plurality of displayed objects. Therefore, the defined three-dimensional shape data is stored in the storage means, and the 3
The three-dimensional position and direction in the three-dimensional space for the three-dimensional pointer are designated by the designating means, and the designated three-dimensional position and direction and the three-dimensional shape data for each object stored in the storage means are set. Accordingly, the detecting means detects an object existing on the extension line indicated by the three-dimensional pointer in the three-dimensional space and closest to the three-dimensional pointer, and detected by the detecting means in the three-dimensional space. An auxiliary pointer for assisting the pointing of the three-dimensional pointer is displayed between the object and the three-dimensional pointer.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the three-dimensional image display system of the present invention.

As shown in FIG. 1, the three-dimensional image display system includes a three-dimensional data creation unit 1, a three-dimensional pointer pointing device 2, a viewpoint setting device 3, a pointer pointing object determination unit 4, an auxiliary pointer orientation determination unit 5, and It is composed of an image display device 6, a system body 7, and I / O control devices 8 and 9.

The three-dimensional data creating section 1 is composed of a system main body 7
The three-dimensional shape data of the object O defined according to the coordinate system defining the three-dimensional space provided therein is created. For example, an image display device 6 for computer graphics such as CAD (cad)
The number of vertices, the number of planes, the vertex coordinates, and the plane-determined vertex set as space coordinates for each object O ... Are set.

The three-dimensional pointer pointing device 2 has three-dimensional position coordinates (Px, Py, Pz) of the three-dimensional pointer Pa displayed on the image display device 6 and a direction vector (Vx, Vy, in the three-dimensional space). The six numerical data necessary for specifying Vz) are sequentially input. For example, it is composed of a magnetic field generator (not shown) and a device for determining the relative orientation and position of the magnetic field generator.

The viewpoint setting device 3 is provided in the system main body 7.
The position / orientation of the viewpoint that specifies the range of the image to be output in the three-dimensional space managed by the three-dimensional space information management unit 24 is set.

For example, whether the screen is viewed from the front or obliquely is set by inputting coordinates from a keyboard or the like. Alternatively, a magnetic sensor may be used to set the position and orientation of the viewpoint.

The pointer pointing object determination unit 4 determines an object O existing on the extension of the direction pointed by the three-dimensional pointer Pa and closest to the three-dimensional pointer Pa. The pointer pointing object determination unit 4 includes an intersection coordinate calculation unit 11, an objective distance calculation unit 12, and a shortest object determination unit 1.
It consists of three.

The intersection coordinate calculation unit 11 has a three-dimensional pointer P.
The coordinates of the intersection of a straight line parallel to the direction vector indicated by the three-dimensional pointer Pa and each object O ... Are calculated from the current position of a as a starting point.

The objective distance calculation unit 12 has a three-dimensional pointer P.
When the object O exists on the direction vector line of a, the object distance, that is, the three-dimensional distance between the objects O from the three-dimensional pointer Pa is calculated.

The shortest object determination unit 13 uses the three-dimensional pointer P
When a plurality of objects O, ... Are present on the direction vector line of a, the object O (shortest object) having the shortest three-dimensional distance is determined as the three-dimensional pointer pointing object O.

The auxiliary pointer azimuth determining unit 5 and the object O determined by the pointer pointing object determining unit 4 and the three-dimensional pointer P
The azimuth of the auxiliary pointer Pb set at the center position with respect to a is determined using the spatial information of the object O determined by the pointer pointing object determination unit 4.

The image display device 6 graphically displays graphic image data corresponding to the viewpoint determined by the viewpoint setting device 3 in a three-dimensional space defined by the system body 7. Various instructions are given to the object O displayed graphically by the three-dimensional pointer Pa. The three-dimensional pointer Pa has the above 3
The display position is changed by the dimensional pointer pointing device 2. When there is a distance between the three-dimensional pointer Pa and the object O, the auxiliary pointer P is located at the intermediate position.
b is displayed. Auxiliary pointer Pb
Is displayed for the purpose of pointing support, and the auxiliary pointer Pb is used to display the three-dimensional pointer P.
It is an important clue for the sense of distance between a and the pointing object O.
The system body 7 controls the entire three-dimensional image display system.

The system body 7 includes a data conversion device 21,
The periodic signal transmission device 22, the pointer control device 23, the three-dimensional space information management unit 24, the visual field determination unit 25, and the graphic image development unit 26 are configured.

The data conversion device 21 determines the plane determination vertex set for each plane of each object O ... Using the spatial coordinates for each object O. The number of vertices, the number of planes, the vertex coordinates, and the plane-determined vertex set for each ... Are output to the three-dimensional space information management unit 24. The periodic signal transmission device 22 generates a pulse signal at fixed time intervals (for example, 1/60 seconds).

The pointer control device 23 receives the output from the periodic signal transmission device 22 as a start signal, and receives the six numerical data supplied from the three-dimensional pointer designating device 2.
Orientation information (Px, Py, Pz) of the three-dimensional pointer functioning in the three-dimensional space built in the system body 7 (V
x, Vy, Vz).

The three-dimensional space information management unit 24 manages the space information of all the objects O ... (Objects) (including the three-dimensional pointer) existing in the defined three-dimensional space.

The field-of-view determination unit 25 determines the viewpoint position in the three-dimensional space from the data (the position and orientation of the viewpoint that specifies the output range) supplied from the viewpoint setting device 3 and determines the image range to be displayed. is there.

The graphic image expanding unit 26 is set to 3 in accordance with the image range to be displayed which is determined by the visibility determining unit 25.
The three-dimensional space is cut out from the three-dimensional space information management unit 24 and the image data to be output is expanded. The I / O control device 8 includes the three-dimensional pointer pointing device 2 and the system body 7.
The pointer control device 23 of FIG. I /
The O control device 9 connects the viewpoint setting device 3 and the visual field determination unit 25 of the system body 7. An example of the spatial information managed by the three-dimensional spatial information management unit 24 will be described with reference to FIG.

That is, 3 for the three-dimensional pointer Pa
The three-dimensional coordinates (consisting of X, Y, and Z components) and the three-dimensional direction vector (consisting of X, Y, and Z components) are stored, and the three-dimensional coordinates for the auxiliary pointer Pb, the three-dimensional direction vector, and the display / non-display flag ( 1/0), three-dimensional coordinates with respect to the viewpoint, and three-dimensional direction vector are stored, and the number of vertices of the object O ... As a frame of data defined in the three-dimensional space of each object O. The number of planes held by, the position coordinate data of each vertex, and the plane-determined vertex set consisting of a set of three vertices that determine each plane are stored.

For example, for the three-dimensional pointer Pa,
Three-dimensional coordinates (100, 25, 120) and direction vector (0, 1, 0) are stored. Three-dimensional coordinates (100, 50, 120), direction vector (0, 1, 0), and display flag (1) are stored for the auxiliary pointer Pb. Three-dimensional coordinates (2000, 0, 1, relative to the viewpoint
00) and the direction vector (1, 0, 0) are stored.

With respect to the first object O, the number of vertices (8), the number of planes (12), vertex coordinates (point A: Xa, Ya, Za, point B: Xb, Yb, Zb, point C: Xc, Yc, Zc,
...), plane-determined vertex set (plane 1: A, B, C, plane 2:
A, C, D, plane 3: B, C, E, ...) are stored. Also, the number of vertices, the number of planes, the vertex coordinates, and the plane-determined vertex set are stored for the second, third, ...

Data of each object O is managed on a plane-by-plane basis, and even if each plane is composed of a large number of vertices, it is always managed by being divided into a range designated by three points. . 3 corresponds to the first object O described above.
It is a rectangular parallelepiped described as an example of an object located in a dimensional space. Next, the operation for the above configuration will be described.

That is, the three-dimensional pointer pointing device 2 has six
One input value is generated and supplied to the pointer control device 23 in the system main body 7 via the I / O control device 8. I / O
The controller 8 converts the six input values into numerical data.

The pointer control device 23 receives a pulse signal generated by the periodic signal transmission device 22 at a unit time interval (for example, 1/60 seconds) as an activation signal, and the I / O control device 8
6 numerical data supplied from the 3D position coordinate values (Px, Py, Pz) and the direction vector (Vx, Vy, Vz) of the three-dimensional pointer Pa in the three-dimensional space provided in the system body 7. Convert. After that, the three-dimensional coordinates and the direction vector of the “three-dimensional pointer” data accumulated in the three-dimensional space information management unit 24 are updated.

The pointer pointing object determination unit 4 determines the object O pointed by the three-dimensional pointer Pa according to the input position coordinates and direction vector of the three-dimensional pointer Pa. The flow of this process calculation process will be described with reference to the flowcharts of FIGS.

Starting from the current position of the three-dimensional pointer Pa,
The intersection point coordinate calculation unit 11 obtains the intersection point coordinates of a straight line parallel to the direction vector pointed by the three-dimensional pointer Pa and each object O. FIG. 5 shows this process. The flowchart of FIG. 5 includes the lower flowcharts shown in FIGS. 6 and 7. First, the equation of a straight line m that passes through the three-dimensional pointer position and is parallel to the three-dimensional pointer Pa, which is a reference for obtaining the intersection, is obtained as described below. Derivation of the equation of the straight line m General formula of the equation of the straight line m passing through the points (a, b, c) The straight line (x-a) / l = (y-b) / m of the direction vector (l, m, n) = (Z-c) / n
(L, m, n ≠ 0) l = 0, mn ≠ 0 x = a (y−b) / m = (z−c) / n m = 0, ln ≠ 0 y = b (x−a) / l = (z−c) / n n = 0, lm ≠ 0 z = c (x−a) / l = (y−b) / m l = m = 0, n ≠ 0 x = a y = b l = N = 0, m ≠ 0 x = a z = c m = n = 0, l ≠ 0 y = b z = c (l = m = n = 0 does not occur here) Next, all objects O In order to save the trouble of calculating the intersection point with ..., It is determined whether or not each object O is located in front of or behind the three-dimensional pointer Pa. This determination process is performed on the plane P (ax) having the normal vector (a, b, c).
+ By + cz + d), the determination formula of whether the point A (p, q, r) exists in the normal direction or in the opposite direction ap + bq + cr + d> 0 (the point A exists in the normal direction) ap + bq + cr + d = 0 (plane P Use point A above) ap + bq + cr + d <0 (the point A exists in the direction opposite to the normal).

Assuming that the direction vector of the pointer is the normal vector of the above equation, each point of the object O is in the normal direction (= forward direction) with respect to the plane passing through the pointer position and having the direction vector of the pointer as the normal vector. ) To find out.

For the determination, a formula for determining the positional relationship between the point and the plane is used. The object O in which all the vertices included in the object O are located behind the three-dimensional pointer Pa is determined not to be specified by the three-dimensional pointer Pa, and is excluded from the instruction candidates. Next, for each object O located in front of the three-dimensional pointer Pa, the intersection with the straight line m extending from the three-dimensional pointer Pa is obtained. This calculation process is shown in FIG.

In general, a plane can be uniquely determined by three different points. As shown in FIG. 2, the plane forming the surface of each object O ... Is managed by a set of three points, and the equation P representing the plane can be obtained from the coordinates.

The intersection coordinates are obtained by solving simultaneous equations for the intersections of the equations representing the respective planes and the straight line m. But,
Since the plane on the object surface is only a small part of the region represented by equation P, boundary conditions must be applied to verify that the obtained solution exists on the surface of the object O.
The verification process is shown in FIG. For the verification, an expression that expresses the internal area of the triangle is used using a vector that expresses two different sides of the triangle. This judgment formula is as described below. △ In ABC, from point B to point R on side AC
The vector to is expressed by BR (vector) = ma (vector) + (1-m) c (vector) (0 ≦ m ≦ 1) (1). Here, if an arbitrary point R ′ in ΔABC is used, BR ′ = kBR (0 ≦ k ≦ 1) (2)

Thus, if the intersection of the plane P and the straight line m, which is determined by the points A, B, and C, is regarded as the point R ', three simultaneous equations can be obtained for the variables m and k2.
The values of m and k can be specified. Here, the intersection R'is ΔABC
The conditions to be within are: 0 ≤ m ≤ 1 and 0 according to equations (1) and (2)
≦ k ≦ 1. Only the coordinates that satisfy the above determination formula are adopted as the intersection coordinates.

Since this verification is performed for each plane divided by triangles, a plane constituted by four points (or more) is divided into triangles as in the example of the rectangular parallelepiped shown in FIG. Manage. In the example of FIG. 3, the surface ABCD is
It is treated as two surfaces ABC and ACD.

If there is no object O located on the direction vector line of the three-dimensional pointer Pa, there is no need to display the auxiliary pointer Pb, so the display / non-display flag of the auxiliary pointer Pb of the three-dimensional space information management unit 24. Is not displayed (0)
To be When the object O exists on the direction vector line of the three-dimensional pointer Pa, the object distance calculation unit 12 calculates the three-dimensional distance between (three-dimensional pointer Pa and object O).
When there are a plurality of objects O on the direction vector line, the shortest object determination unit 13 determines the object O having the shortest three-dimensional distance as the pointing object O of the three-dimensional pointer Pa. When the pointer pointing object determination unit 4 determines the pointing object O, the auxiliary pointer orientation determination unit 5 determines the position coordinates and direction vector component of the auxiliary pointer Pb.

The position coordinate of the auxiliary pointer Pb is intermediate between the intersection obtained by the intersection coordinate calculation unit 11 for the pointing object O and the position of the three-dimensional pointer Pa, and the direction vector component of the auxiliary pointer Pb is that of the three-dimensional pointer Pa. It is equal to the direction vector component. The azimuth information value of the auxiliary pointer Pb stored in the three-dimensional space information management unit 24 is immediately updated with the newly obtained azimuth information of the auxiliary pointer Pb.

By the viewpoint setting device 3, the system main body 7
When outputting the three-dimensional space provided therein to the image display device 6, the orientation of the viewpoint from which the screen is cut out is determined. The viewpoint setting device 3 outputs a setting trigger signal for instructing the position and direction of the viewpoint and updating of the viewpoint setting value to the visibility determining unit 25 in the system body 7 via the I / O control device 9.

The field-of-view determination unit 25 holds the viewpoint information used for determining the range of the output screen in the form of the coordinate of the lowered coordinate of the viewpoint and the downward direction vector. When a setting trigger signal is received from the viewpoint setting device 3 via the I / O control device 9, the held viewpoint information is updated to the simultaneously received viewpoint information.

The graphic image development unit 26 develops graphic image data to be displayed on the image display device 6 based on the information of the three-dimensional space information management unit 24 and the visual field determination unit 25, and the developed graphic image data is It is supplied to the image display device 6.

Based on the graphic image data, the image display device 6 displays a three-dimensional space image composed of the objects O, ... In the system body 7 and the three-dimensional pointer Pa and the auxiliary pointer Pb as shown in FIG. The graphic is displayed.

As described above, the three-dimensional pointer and the three
Since the three-dimensional pointer is assisted by displaying the auxiliary pointer in the middle of the object pointed by the three-dimensional pointer, it is used for pointing the position or object in the space in the three-dimensional space displayed graphically.
It is possible to support a manual positioning operation for the dimensional pointer.

That is, the sense of distance to the object in the three-dimensional space graphically displayed at the image position is more intuitive than the shadow of the hand that occurs when an object is touched as compared to the display such as numerical values and graphs. It is possible to improve the pointing accuracy by facilitating the pointing operation, facilitating the pointing work, and particularly effectively functioning near the object.

In the CG (computer graphics) representation, the projection display of the shadow of the hand requires a huge amount of calculation and time due to the influence of the shape of the light source and the projection surface. By the processing that is much lighter than the above, the same effect as the shadow of the hand in the object pointing operation in the three-dimensional space can be obtained.

Therefore, in the three-dimensional space displayed graphically on the image display device, the auxiliary pointer is displayed at the midpoint between the intersection between the designated solid and the three-dimensional pointer when the three-dimensional pointer is extended. By doing so, the effect similar to the shadow of the hand, which is an important clue to the sense of distance in the real world, is realized by a process much lighter than the projection calculation of the hand shadow, and the pointing work is facilitated. The accuracy can be improved.

Further, the shape of the auxiliary pointer displayed is congruent with the original three-dimensional pointer, and the sense of distance obtained by comparing the sizes of the two is better than when other auxiliary objects are presented. .

Further, the auxiliary pointer is the intersection point 3 with the designated solid when the original 3D pointer is extended and
Since it is displayed at the midpoint of the original position of the three-dimensional pointer, the position of the auxiliary pointer and the original three-dimensional pointer are adjusted so that the auxiliary pointer and the original three-dimensional pointer are aligned with each other.
It becomes easy to specify the position of the dimension pointer.

Further, by facilitating the pointing work in the vicinity, the effect more than the shadow of the hand in the real world can be expected because it is not affected by the position of the light source. Moreover, CG
This can be realized by far less processing than the projection calculation in 1., and can contribute to the ease and accuracy of pointing work in the three-dimensional space displayed on the image display device.

[0055]

As described above in detail, according to the present invention, it is possible to support the same effect as the shadow of the hand in the object pointing operation in the three-dimensional space by the processing which is much lighter than the projection display. A three-dimensional image display system that can be performed can be provided.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of a three-dimensional image display system according to an embodiment of the present invention.

FIG. 2 is a diagram showing an example of spatial information managed by a three-dimensional spatial information management unit in FIG.

FIG. 3 is a diagram showing an example of an object located in a three-dimensional space corresponding to a first object.

FIG. 4 is a flowchart illustrating a display process of an auxiliary pointer.

FIG. 5 is a flowchart for explaining processing for obtaining an intersection between a direction vector of a three-dimensional pointer and a first object.

FIG. 6 is a flowchart for explaining processing for determining whether each object is located on the entire surface of a three-dimensional pointer.

FIG. 7 is a flowchart for explaining a process of checking whether or not the three-dimensional pointer is located on the direction vector line.

FIG. 8 is a flowchart for explaining a process of determining whether the obtained intersection is located on the corresponding surface of the object.

[Explanation of symbols]

1 ... 3D data creation unit, 2 ... 3D pointer pointing device, 3 ... viewpoint setting device, 4 ... pointer pointing object determination unit,
5 ... Auxiliary pointer orientation determining unit, 6 ... Image display device, 7 ...
System main body, 8, 9 ... I / O control device, Pa ... Three-dimensional pointer, Pb ... Auxiliary pointer, O ...

Claims (2)

[Claims] 1. A three-dimensional image display system for displaying at least an image of an object, a three-dimensional pointer, or the like using a three-dimensional space defined in advance, and coordinates for defining a three-dimensional space for the plurality of displayed objects. Storage means for storing three-dimensional shape data defined according to the system, designating means for designating a three-dimensional position and direction in the three-dimensional space for the three-dimensional pointer, and three designated by the designating means. Depending on the three-dimensional position and direction and the three-dimensional shape data for each object stored in the storage means, the three-dimensional pointer exists in the extension line indicated by the three-dimensional pointer in the three-dimensional space and is the most three-dimensional pointer. A detecting means for detecting a near object, and the three-dimensional point between the object detected by the detecting means and the three-dimensional pointer in the three-dimensional space. 3-dimensional image display system characterized by comprising display means, the displaying auxiliary pointer to assist in the instruction. 2. A three-dimensional image display system for displaying at least an image of an object, a three-dimensional pointer or the like using a three-dimensional space defined in advance, and coordinates defining a three-dimensional space for the plurality of displayed objects. Storage means for storing three-dimensional shape data defined according to the system, designating means for designating a three-dimensional position and direction in the three-dimensional space for the three-dimensional pointer, and three designated by the designating means. Depending on the three-dimensional position and direction and the three-dimensional shape data for each object stored in the storage means, the three-dimensional pointer exists in the extension line indicated by the three-dimensional pointer in the three-dimensional space and is the most three-dimensional pointer. Detecting means for detecting a close object, and a calculation for calculating an intermediate coordinate in the three-dimensional space between the object detected by the detecting means and the three-dimensional pointer. And means, on the coordinates of the three-dimensional space calculated by the calculation means,
A three-dimensional image display system comprising: a display unit that displays an auxiliary pointer that assists the instruction of the three-dimensional pointer.