JPH03296176A - High-speed picture generating/displaying method - Google Patents

Abstract

PURPOSE:To generate and display the pictures of high quality at a high speed by setting a hierarchy degree with the distance between the origin or a visual point of a display reference coordinate system and an optional point of an object shown in an object coordinate system used as at least one parameter. CONSTITUTION:An object of display has a constant size. However the subject is displayed in a larger size as shown in an object coordinate system h-obj1 on a screen S1 when it is in a near place and displayed in a smaller size as shown in an object coordinate system h-obj3 when it is in a far place respectively. In this case, the modeling hierarchy is set fine in a near place and then set rough in a far place respectively. Therefore for an observer both of the systems h-obj1 and h-obj3 are felt satisfactorily precise. Furthermore the calculation speed of the perspective projection transformation of both systems obj2 and obj3 is much higher than the obj1. Then the calculation quantity needed for display of an entire screen S1 is reduced. Thus the pictures of high ambience can be obtained at a high speed.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] This invention relates to computer graphics for displaying realistic images on a screen at high speed, that is, a high-speed image generation and display method.

[Prior art] Generating and displaying a three-dimensional image by a computer,
So-called computer graphics technology is being used in a variety of fields, including simulations of scientific and technical calculations. Currently, no technology has been found that satisfies this requirement. In other words, if you try to generate a high-quality image, a huge amount of calculation time is required to generate the image.On the other hand, if you aim for high-speed, real-time display such as displaying several tens of images per second, you will need coarse calculations. As a result, the quality of the generated image deteriorates.

On the other hand, the field of application of computer graphics is also expanding to the field of image communication called intelligent coded communication. In this method, the transmitting side uses image recognition to extract three-dimensional structural information of the transmission target, parameterizes its characteristics, and transmits the information. On the receiving side, three-dimensional structure information (database) is prepared in advance, and the three-dimensional database is converted and displayed at high speed based on the transmitted feature parameters. In these communication methods, the receiving side communicates with the sender's 3
Since it is possible to freely generate and display images from any viewpoint based on dimensional structure information, it has many advantages such as the following.

■ Motion vision can be realized by detecting the movement of the receiver's viewpoint and displaying an image that corresponds to this movement. Here, kinesthetic vision refers to a person's potential ability to perceive space from changes in the image reflected on the retina of the eyeball when moving the head. Trying to raise awareness through movement is a common experience. That is, by artificially realizing motion vision using computer graphics, it is expected that the three-dimensional effect of images on the screen will be improved, making it possible to display images with a rich sense of realism.

■ Binocular stereopsis can be achieved by using the receiver's eyes as two viewpoints and generating and displaying images from each viewpoint. Since a person's eyes are located approximately 6 cm apart, different images are projected onto the retina each time the eye is viewed, even without moving the head. The correspondence between these two different images is called parallax information, and people use this to obtain a three-dimensional sense of space. By creating an image with this parallax information using computer graphics, the three-dimensional effect of the image on the screen is further improved.

■ In addition, it is also possible to easily generate images that show the person on the other end (sender) having a conversation while making eye contact.

However, there are currently some difficulties in implementing such a communication method. The first is image recognition and feature parameter extraction on the transmitting side, and the second is an image generation and display method on the receiving side that displays a wide field of view at high speed and with high resolution.

In communication, real-time processing is essential, and there is a higher demand for faster processing than in conventional computer graphics.

The reason why high-speed, high-precision display is difficult will be explained in detail below.

FIG. 15 is a diagram for explaining a typical conventional three-dimensional image generation and display method. In FIG. 15, the screen S1 is represented by a display reference coordinate system X(w)-Y(w)Z(w) having the center of the screen S1 as its origin. Note that this coordinate system has a function of describing the positional relationship of the following coordinate systems, and the origin does not have to be the center of the screen S1. Viewpoint coordinate system X (e) −Y (
e) −Z (e) is the display reference coordinate system X (w) −
The position and rotation angle of the coordinate axes are known from Y (w) −Z (w). The display target object obj has a three-dimensional structure, and the structure point Pi (obj) of this target is defined by the target coordinate system X (ob D −Y (ob D −Z (ob D
Assume that it is expressed as .

In this situation, when the structural point Pi is viewed from the viewpoint Oe, consider the position on the screen S1 where the structural point Pi is displayed. Display reference coordinate system X (w)
-Y (w) -Z (w), target coordinate system
(ob j) -Y (ob D -Z(ob D
is known, so the object coordinate system X(ob D −Y
(ob D −Z (structural point P represented by ob D
i(pbj) is the display reference coordinate system X(w)-Y(w)
−Z (w), and P i (w) = M1・Pi(o
bj).

Here, Ml is a transformation matrix for translation and rotation. Also, the viewpoint coordinate system X (e) −Y(e) −Z
(e) also displays reference coordinate system X (w) −Y(w) −Z
(w), so if this transformation matrix is M2, then the display reference coordinate system X
(w) -Y (w) -Z (w) Structural point P
i(w) is the viewpoint coordinate system X (e)Y (e) −Z
Seen from (e), P i (e) −M2-1・P i (w)−=M
1 ・M2-I Pi (obj). In this way, the display reference coordinate system
(w) -Y (w) -Z (w) and target coordinate system X
(ob j) -Y (ob D -Z (ob D
and display reference coordinate system X (w) −Y (w) −Z
(w) and viewpoint coordinate system X (e) −Y (e) −Z
If the relationship with (e) is known, the structure point Pi(obj) shown in the object coordinate system can be freely expressed in the viewpoint coordinate system.

Here, in order to find the point on the screen of the structure point Pi(e) indicated by the viewpoint coordinate system, consider X (np) - Y (np) - Z (np), which is called a regular perspective coordinate system. This coordinate system has its origin 0 (np) on the Z (e) axis of the viewpoint coordinate system, and areas A, B, and C, called visual field pyramids, are hatched in the viewpoint coordinate system.

D, A (cO), B (oo), C (-)
, D (=)) is the rectangular parallelepiped A shown by the - dotted chain line,
B, C, D, A' (■), B' (oO),
There is a corresponding relationship between C' (,■) and D' (glossy). It is assumed that the Z (np) axis direction of the rectangular parallelepiped is normalized to 1. That is, the point Z (np) = 0 is Z (e)
Corresponding to axis h, the point Z (np) −1 is Z (e)
= Corresponds to ■.

Under these conditions, the rod-shaped object obj shown in FIG. 10 is -
As shown by the dotted chain line, the cross section becomes smaller on the larger Z(np) axis. The transformation between this viewpoint coordinate system and the normal perspective coordinate system can be expressed by a 4×4 matrix M3. Therefore, P t (n p) can be expressed as P i (np) = M3 · P i (e). Here, P i (n p) is A,
When projected parallel to planes B, C, and D, ξi is shown by the dotted line.
(s) is obtained.

In FIG. 15, for ease of explanation, the regular perspective coordinate system
(np)-Y (np)-Z (np) is assumed to have an origin and an axial direction that overlap the display reference coordinate system X (w) -Y (w) -Z (w). Therefore, the Z (e) axis of the viewpoint coordinate system is the Z (e) axis of the display reference coordinate system.
(w) It is in the same direction as the axis.

In this case, rectangular parallelepipeds A, B, C, and D correspond to the size of the screen. Therefore, ξi (s) is the projected image on the screen seen from viewpoint 0 (e). In this way, from the structure point Pi(obj) to ξi
(s) requires three matrix operations and a parallel projection.

Next, a case where the viewpoint deviates from the Z (e) axis of the display reference coordinate system will be described. In Figure 15, 0' (
e) is the moved viewpoint position. A new viewpoint coordinate system X' (e) Y' (e) -Z with this moved point as the origin and the 0' (e) - 0 (w) line as the 2 (e) axis
' Consider (e). Display reference coordinate system X (w) −
If the position of the new viewpoint coordinate system as seen from Y (w) -Z (w) is known, then X (e) -Y(e)-Z (e
) to X' (e) -Y' (e) Z' (e
) is possible via matrix M4. Therefore, the structure point Pi seen from the new viewpoint coordinate system is P i
(e) ' = M4 ・P i (e) M4 ・M
It is expressed as l.M2-1.Pi (obj).

FIG. 16 is a view seen from above the Y(w) axis in FIG. 15.

Multiplying Pi(e)' by the normal perspective projection transformation matrix M3 described above yields Pi' (np), and parallel projection of this yields ξi" (s). Here,
New viewpoint coordinate system X' (e) seen from the display reference coordinate system
Since the position of -Y' (e) -Z' (e) is known,
It is easy to find the point where the line connecting ξi" (s) and O' (e) intersects the screen. This point ξi
'(s), that is, the projected image on the screen as seen from the viewpoint O'. In this way, the structure point Pi(obj)
From ξi'(s), four matrix operations, parallel projection, etc. are required.

As mentioned above, the display reference coordinate system X (w) −Y(w)
−Z (w), the object coordinate system X(obD −Y
(ob D -Z (ob
A display object whose coordinates are described can be perspectively projected onto the screen from any viewpoint. However, as described above, it is necessary to perform many matrix operations on all constituent points to be displayed, resulting in an enormous amount of calculation time. Matrix calculations can be made considerably faster by using hardware, but real-time processing becomes difficult when the number of constituent points exceeds several thousand. Further, a lot of unnecessary processing is performed depending on the position of the display target, which limits the processing speed. This is explained below.

Figure 17 shows the coordinate system shown in Figure 15 viewed on the Y (e) axis and in the Z (e) axis direction, and Figure 18 shows the display target shown in Figure 17. be.

In FIG. 17, the screen S1 consists of a large number of pixels gi. The display target is 3 as shown in Figure 18.
It has a dimensional structure, with a structural point Pi and a structural surface L created by that point.
It is composed of i, etc. The case where the display object is nearby in the display reference coordinate system is shown in FIG. 17 (A), and the case where it is far away is shown in FIG. 17 (B), and these are designated as objl and obj2, respectively. By the method explained in FIGS. 15 and 16, the display target h-
.

bjl and h-obj2 are projected. A signal can be sent to the corresponding pixel on the screen coordinates of each component point and displayed. Since obj2 is located far away, the distance between each constituent point is smaller than that of h-objl.

If this interval is sufficiently smaller than the pixel interval, there is no need to calculate all the constituent points within that pixel. In addition, the human eye has visual acuity characteristics, and it is meaningless to calculate each component point on the screen 81 at an interval that is less than the resolution of the eye.

Existing algorithms calculate all constituent points of the target regardless of the position of the display target and without considering the user's visual acuity characteristics, which slows down the display speed and makes it difficult to display complex targets in real time. It becomes difficult to do so.

[Problems to be Solved by the Invention] The above-mentioned problems are particularly problematic when realizing stereoscopic computer graphics. Stereoscopic computer graphics is a system that uses the user's right and left eyes as viewpoints to separately calculate the perspective images on the screen when looking at a 3D display object, and uses time sharing to calculate the perspective images for the user's left and right eyes. This is a method of creating a display that gives the user a three-dimensional impression by projecting it onto a screen that corresponds to the eyes.

In stereoscopic display, depth intervals are clearly recognized by the user, so it becomes necessary to model normally large objects (such as buildings) large and small objects (such as insects) small. In non-stereoscopic computer graphics, it is possible to make objects appear larger by placing them closer to the viewpoint without having to model them larger. However, in stereoscopic vision, the depth interval is given by parallax information, so even if a small object is placed in the foreground, it is not recognized as a large object, and the small object appears to be in the foreground.

When an object is modeled to its original size, a large object becomes
If we assume that such an object will come close to the viewpoint and spread out across the screen, it will be necessary to model it down to the smallest detail. On the other hand, when the object is far away from the viewpoint, the same amount of calculation is required as when the object is nearby, even though the entire object is displayed on a part of the screen, as shown in Figure 17 (B). There is an absurdity in requiring it.

The need for high-speed computer graphics has been described above, but even if computer graphics is real-time, there are disadvantages depending on the application. In immersive communication, a virtual space is artificially created using computer graphics, and users can interact with the virtual space in various ways as if they were actually there. What is required here is not only high speed, but also the ability to change the screen image in synchronization with the observer's movements.

However, in computer graphics, there is always time left to generate images, so even if it is possible to generate and display several tens of images per second, there is a delay in generating each image relative to the movement of the observer. , it creates a sense of discomfort and a sense of unity with the virtual space cannot be obtained.

Therefore, the main purpose of this invention is to provide real-time and highly accurate computer graphics display necessary for intelligent encoded communication and stereoscopic computer graphics.
An object of the present invention is to provide a high-speed image generation and display method that can be realized by significantly reducing the amount of calculation.

Another object of the present invention is a high-speed image generation and display method that reduces the time delay between image generation and display in computer graphics that realize motion vision, and can realize a natural display that does not give an unnatural feeling. The goal is to provide the following.

[Means for Solving the Problems] The present invention is a high-speed image generation and display method for projecting and displaying an object having a three-dimensional structure on a two-dimensional screen, in which the constituent planes of the object have the size of the area in the object coordinate system. at least 1
When projecting the component planes of an object as seen from an arbitrary viewpoint onto a two-dimensional screen, it is necessary to The hierarchy level is set using the distance to a point as at least one parameter.

More preferably, the arbitrary viewpoint is the pupil or iris position of the observer, and the hierarchy level is set using a distance from the viewpoint to an arbitrary point on the object as a parameter.

More preferably, the position of the viewpoint at the next image display time is predicted from the trajectory of the viewpoint, and a projected image viewed from the predicted viewpoint is generated in advance.

Still more preferably, the position of the pupil or iris is represented in a display reference coordinate system, the position of the object is approximated by the origin of the object coordinate system, and the origin is represented in the display reference coordinate system, and the position of the pupil or iris is The distance from the position to the origin of the target coordinate system is expressed in the display reference coordinate system.

1 Effect] 6 The invention according to the first claim hierarchically models the display target using the size of the area, that is, the scale as one parameter, and when the target is near the origin of the display reference coordinate system or the viewpoint, By displaying even a fine modeling hierarchy and displaying a coarse modeling hierarchy when an object is far from the origin of the display reference coordinate system or the viewpoint, the amount of calculation is greatly reduced and speed is increased, especially for distant objects.

Note that in monocular computer graphics, since the displayed object is never in front of the screen, the parameter for selecting the level of hierarchy may be the distance from the origin of the display reference coordinate system. In this case, calculation is easier than when the viewpoint is the origin.

On the other hand, in the case of binocular stereoscopic computer graphics, objects may be displayed so as to jump out in front of the screen, so the distance from the viewpoint may be used as a parameter for selecting the level of hierarchy. In this case, it becomes necessary to calculate the distance each time the viewpoint moves.

The invention according to claim 2 is to generate a highly accurate motion parent image by using the center of the pupil or iris, which is located near the optical viewpoint of the eye, as a viewpoint. Furthermore, speeding up is achieved by reducing the total amount for objects located far from the viewpoint.

The invention according to claim 3 eliminates the unnatural feeling of display due to the delay in image generation time, which is a problem when realizing motion vision in the invention according to claim 2, and obtains a more natural image.

The invention according to claim 4 is the invention according to claim 2, in which the position of the pupil or iris is represented by a display reference coordinate system, the position of the target is approximated by the origin of the target coordinate system, and the origin is displayed. It is expressed in a reference coordinate system, and the distance from the position of the pupil or iris to the origin of the object coordinate system is expressed in the display reference coordinate system.

[Embodiments of the invention] Fig. 1 is a diagram showing the principle of this invention, and Fig. 2 is a diagram showing the principle of the invention.
FIG. 6 is a diagram showing an example of hierarchical modeling of the display target shown in the figure.

First, the principle of this invention will be explained with reference to FIG. Vectors VOI and VO2 are respectively at the origin 0 of the display reference coordinate system X (w) -Y (w) -Z (w)
It is a vector from (w) to the origin of the object coordinate system (o b j 1. o b j 2) of the display objects objl and obj2. Also, compared to the conventional example shown in FIG.
The difference is in how the structural data to be displayed is held. Second
The figure shows an example of hierarchical modeling of display objects. In the example shown in FIG. 2, the display target is a plurality of structural points (k l I
+ k 2 + + k 3 J... Explosion j=1...
) and a structural surface (Ll) created by a set of these structural points.

is defined as a surface surrounded by four structural points). Each structure point is in the object coordinate system X (ob
D -Y (obD -Z (represented by obD. The hierarchy setting in Figure 2 is such that the distance between each structure point or the size of the structure surface is such that the number of structure points is decreased each time the distance between each structure point or the size of the structure surface exceeds a predetermined size. There are three levels of hierarchy in total.This data structure represented by structural points and line segments connecting the points is called a wireframe structure.

In the example shown in Figure 1, as shown in Figure 2, the hierarchy level of the layered display object is set to the origin 0 of the display reference coordinate system.
The distance from (w) to the origin of the target coordinate system is adaptively selected and displayed as one parameter.

Figure 3 shows this situation, and shows an example in which the distance from the origin 0 (w) of the display coordinate system to the origin of the target coordinate system is adaptively selected and displayed as one parameter. be. In FIG. 3, areas Th, , Th2 . Th
3 is an area where the distance from the origin 0(w) of the display target coordinate system is a predetermined value.

The object (for example objl) that includes the origin of the target coordinate system inside the area Th, uses the modeling hierarchy shown in FIG. 2(b), and the object that includes the origin of the target coordinate system inside the area Th3 uses the second Using figure (C), h-obj2. h
-Display as obj3.

The displayed objects have the same size as shown in Figure 2, but if they are nearby, they will be displayed large as h-objl on screen S1, and if they are far away, they will be displayed small as h-obj3. be done. This modeling hierarchy at 0 is fine when it is nearby.
When the object is far away, it is selected coarsely. Therefore, to the observer, both h-objl and h-obj3 seem to be sufficiently detailed. The calculation speed of the perspective projection transformation of obj2 and obj3 is much faster than that of objl. Therefore, the amount of calculation required to display the entire screen S1 is significantly smaller than before. This effect increases particularly as the background display object becomes more complex. In real space, there are far more objects far away than near the observer. Therefore, images with a high sense of realism can be generated at high speed.

In the examples shown in FIGS. 1 and 3 above, the distance from the origin 0(w) of the display reference coordinate system to the origin of the target coordinate system was used as a parameter. For example, if the target is large, the target coordinates Since the distance from the system origin to the target structural surface becomes large, the distance between one structural surface and another structural surface may become large, and one may be located in the front while the other is located far away.

In such cases, it may be more reasonable to use the distance to a point other than the origin of the target coordinate system as a parameter.

FIG. 4 is a diagram showing an example in which the display target is formed by a combination of a plurality of large structures, and an individual target coordinate system can be set for each part of the structure. In FIG. 4, the origin or viewpoint position of the display reference coordinate system is represented by K, the main object coordinate system is represented by CorM-obj, this position vector is represented by 0, and the main object coordinate system is CorM-obj. -M-
Part or main object coordinate system Cor-M- of obj.

The subobject coordinate system of object obj2 connected to bj is Cor
-3-obj. Main object coordinate system Cor-
The position of the sub-object coordinate system Cor-3-obj when viewed from M-obj is given by the vector Vol. In this example, the position of the object obj2 is represented by ■00Vol=V2. It is assumed that the fine and non-fine modeling layers are divided according to the distance Th from the viewpoint position K. Origin 0 of target obj2
(.

Since bj2) is inside the distance T)t, a model with close spacing between structure points is selected. The origin of the target objl is 0 (o
bjl) is outside the distance T h, so a coarse model is chosen. Note that the broken line in the target ○bjl is a candidate point (line) that was not selected.

FIG. 5 is a schematic block diagram of one embodiment of the present invention.

In the embodiment shown in FIG. 5, the viewpoint is the eyeball, which is moving, and for the sake of simplicity, it is assumed that the display object is stationary in the display reference coordinate system. In addition, the fifth
The part indicated by the thick solid line in the figure is a novel means not found in the prior art.

Referring to FIG. 5, viewpoint detection means 10 detects the viewpoint of the eyeball in the display reference coordinate system. The target coordinate system setting means 20 specifies the origin position and rotation angle (angle) of the display target in the display reference coordinate system. The distance calculation means 30 calculates the distance from the viewpoint to the origin of the target coordinate system according to the detection output of the viewpoint detection means 10 and the origin position and rotation angle specified by the target coordinate system setting means 20. In the three-dimensional hierarchical database 80, the constituent surfaces of the object to be displayed are hierarchically described using the size of the area as one parameter. The component point/component surface generating means 40 receives as input the origin position of the target coordinate system to be displayed and the above-mentioned distance, and generates display reference coordinates while referring to the component surface data indicated by the three-dimensional hierarchical database 80. A three-dimensional model is generated in the system. Note that the portion indicated by the dotted line in FIG. 5 constitutes a processing means 100 that selects the hierarchy level using distance as a parameter.

The perspective projection conversion means 50 receives the viewpoint position and the above-mentioned three-dimensional model as input, and perspectively projects the display object onto the two-dimensional screen. The display image generation means 60 creates a pattern such as coloring on the target constituent surface while referring to the constituent plane data obtained by specifying an object and one layer of constituent planes in the three-dimensional hierarchical database 80. Attach.

A lenticular screen or the like is used as the display means 70 to display the target object. In the embodiment shown in FIG. 5, when generating constituent points/constituent planes, the fineness of the constituent planes is weighted by the above-mentioned distance, but this operation is performed by the constituent points/constituent plane generating means 40. This is done by extracting the constituent planes of a predetermined hierarchy from the three-dimensional and four-dimensional hierarchical database 80 while comparing the distance with a predetermined reference value Th.

FIG. 6 is a flowchart for explaining the specific operation of one embodiment of the present invention.

Next, referring to FIG. 6, a detailed operation of an embodiment of the present invention will be described. Now, as shown in Figures 1 and 3 above, place the object object in the display reference coordinate system.
jl, obj2. It is assumed that obj3 generates an image on the screen that appears to be at a predetermined position. The position of the viewpoint is the pupil position, and therefore the viewpoint is assumed to be moving.

Furthermore, for ease of explanation, the object is stationary in the display reference coordinate system, and the three-dimensional structure data of the object is described in the object coordinate system, as shown in Figure 2, and the constituent planes are of the area. It is stratified using size as a parameter.

Further, in step SPl (abbreviated as SP in the figure), the object coordinate system setting means 20 sets the position vector of the object coordinate system origin V obl-1== (xobl-1+
yob, -1+ Zobl-1) and the rotation angle of the target coordinate system Robl-1-(αobj-1+ β.bl-1+
γobl−1) is set and provided to the processing means 100. In step SP2, the viewpoint position Voe-(Xoe, Yoe
.

Zoe) is provided to the processing means 100. Processing means 10
0 reads the viewpoint position vector Voe at that time given from the viewpoint detection means 10 in step SP3, and obtains the position vector ■ of the origin of the target coordinate system. 1. −1 and the viewpoint position vector Voe, the distance d I between the two points is
Sobl-1-l Vobl■oe1 is calculated.

In step SP4, the processing means 100 compares the calculated distance d I S obl-1 with a reference value Th2.

When the processing means 100 determines that the distance d I S obl-1 is larger than the reference value Th2, it proceeds to step SP5.
In this step, a model with a low hierarchy level shown in FIG. 2(c) is selected from the three-dimensional hierarchical database 80, such as the object 0bj3 shown in FIG. And processing means 100
is the position vector of the origin of the target coordinate system V ob i −1
A three-dimensional model is generated at the position of the rotation angle Rob J −l of the target coordinate system.

In step SP4, the processing means 100 calculates the distance dis between the two points. , I-1 is smaller than the reference value Th2, the distance d between the two points is determined in step SP7.
i S obil is compared with the reference value Thl. The processing means 100 calculates the distance dis between two points. , 11 is the reference value Th
If it is determined that it is larger than l, in step SP8, the data as shown in FIG. 2(b) is obtained from the three-dimensional hierarchical database 80.
A model with the hierarchical level shown in is selected, and in step SP9, a three-dimensional model is generated in the same manner as in step SP6 described above. Furthermore, the processing means 100 determines the distance d between the two points.
If iS abj-1 is smaller than the reference value Thl, in step 5 PIO, the three-dimensional hierarchical database 8
0 to a model with a high hierarchy shown in FIG. 2(a) is selected, and a three-dimensional model is generated in step 5P11.

In this way, SF3. When a three-dimensional model is generated using either SF3.5 PII, perspective projection conversion means 50
is the perspective projection transformed image of the object h7 obj−+ 1 hobl 2 + 11ob+−
Generate 3. In step 5P13, the display image generation means reads the pattern data of each component surface stored in the three-dimensional hierarchical database 80 and applies a pattern to the target component surface of the two-dimensional screen. The target image generated in this way is displayed on the display means 70 in step 5P14.

FIG. 7 is a diagram showing another example in which the distance of a point other than the origin in the target coordinate system is used as a parameter, and FIG. 8 is a diagram showing a hierarchically modeled display target.

In FIG. 7, the display target obj is hierarchically modeled as shown in FIG.
For the constituent surfaces aI+a2...al of the hierarchy shown in , the center position of the surface is Val in the target coordinate system.

It is assumed that the values are expressed as Va2...Va+. In FIG. 5, to create each constituent surface a+, the distance to the constituent surface is determined as the sum of the position vector ■0 of the origin of the object coordinate system and the center position Va, and the distance VO+Va is determined by the distance T
The hierarchy of the 8 constituent planes is selected by comparing with hl and Th2. 8(a) on the configuration surface a1.
The display target shown in FIG. 8 is selected, and the display target shown in FIG.
The display object shown in b) is selected, and the display object shown in FIG. 8(C) is selected on the configuration surface a3. In this way, the distance to any point in the target coordinate system can be used as a parameter.

In the example shown in FIG. 1, the display reference coordinate system X (w
) −Y (w) −Z (w) The distance from the origin 0 (w) to any point in the target coordinate system is used as a parameter, but the origin of the viewpoint (0 (e) or O' (e
) etc.) may be used as a starting point. In this case, if the viewpoint moves even if the displayed object does not move, there is a need to change the hierarchy each time, but considering that the visual acuity characteristics of the person who recognizes the details of the object greatly depends on the distance from the viewpoint. , it can be said that a more rational hierarchy selection can be made. In addition, in binocular stereoscopic computer graphics, etc., the display target may be displayed in front of the screen S1, but in such a case, as shown in Figure 1, the distance from the origin 0 (W) If is a parameter, even though the object is near viewpoint 0 (e), it is immediately behind (rear) screen S1
This would also result in an irrational situation in which a coarser hierarchy would be selected over objects in the . Therefore, although the calculation is complicated, a method using the distance from the viewpoint to the origin of the target coordinate system as a parameter is also effective.

In this invention, using the size of the area as one parameter means that the number of data is calculated every time a measure of the area exceeds a predetermined size, such as the distance between structure points or the volume of one structure area. It means decreasing stratification. Further, parameters other than the area size may be used as other parameters. For example, if the display target is a human image, the cheeks,
Compared to the nose, ears, etc., the eyes and mouth are important parts that determine a person's facial expressions. Therefore, it is reasonable to model these parts more precisely than other parts.

In this way, in addition to the size of the area, the importance of the meaning of the displayed portion can be added to the parameters.

Furthermore, as shown in FIG. 2, the structure points may be points that are actually displayed, or may be feature data that defines the structure of a partial area to be displayed.

For example, when the partial region is a free-form surface such as the cheek of a face, a descriptive formula for the free-form surface may be used to model this region, and control parameters for the curved surface expressed by the formula may be used as hierarchical data. - For example, if the partial region shown by dots in Fig. 2 is a curved surface, Be
Using the Zier surface formula, this control point may be used as hierarchical data.

The Bezier curved surface can be expressed as a product surface of Bezier curves expressed by the following equation. The Bezier curve is expressed by the following equation.

R(t:n) = (1-t+tE)'Po
: (0≦t≦1) Here, E is called a shift operator and is EP.

Represents P1+1. Pl is called a control point, and its position is given in the form of coordinates. A Bezier curve is in the form of a linear combination of control points.

FIG. 9 is a diagram showing quadratic and cubic Bezier curved surfaces. The example shown in FIG. 9(a) is a case of 2nd order and 13th order (n=2.3), and is expressed by the following equation.

R(t:3)-(1- to 10tE)'p.

When 1=0, the Bezier curve starts from the control point of Po, and when t=1, it starts from the control point of Po (for n-2, it starts from the control point of Po).
2. When n=3, P3) is reached. The curve smoothly connects the area near the control point. Increasing the number of control points corresponds to increasing the degree, which increases the expressiveness of the curve.

A Bizier surface is expressed as a product of Bezier curves as shown in the following equation.

S (u: m, v: n) = <1-u+u
E) "(1-V+vF)' Po.

(0≦U≦1) (0≦V≦1) E and F are shift operators, and E P II-P
It acts as l+l +5, FPII=PI, II1.

Figures 9(c) and (d) show the quadratic and cubic Bezie
It shows an r-curved surface. For 3rd order, 16 from POO to P33
By preparing these control points, a free-form surface that smoothly connects them can be generated.

2 In this way, by adaptively determining a partial region and selecting enough control points to describe this region, a surface with a fine and smooth surface will be obtained. On the other hand, if the partial area is made large and the number of control points is decreased, the surface will be rough. An example of hierarchizing using the number of control points in a partial region as a parameter has been shown above.

Furthermore, in the Bezier surface equation described above, the structure points are determined by the control points and how to take the values of u and v.

Therefore, a predetermined control point may be determined and the interval between u and v may be used as a hierarchical parameter. That is, in FIG. 9(d), even if the number of control points is constant at 16, U,
Depending on how the V interval Δ is determined, the constituent surfaces can be coarse or dense. If the interval Δ is made smaller, the constituent planes will be further divided than in FIG. 9(d), and many small constituent planes will be generated. In this way, it is also possible to hierarchize the size of the area using Δ as a parameter.

Next, as a method of displaying on the screen, as shown in Figure 1, in addition to the simple method of displaying points and lines obtained by perspectively projecting the wireframe, there is also a method of displaying color data on the structural surface. , to be colored when displayed. ■For each structural surface, a pattern called a texture map is prepared as data,
This picture is displayed on the screen after being rotated, moved, reduced, enlarged, etc. depending on the direction and position of the structural surface.

■Assuming a light source at a predetermined location in the display reference coordinate system,
It is possible to calculate and display the shadow of each constituent surface caused by the light as seen from the viewpoint for each constituent surface.

As an example, a case will be described in which a life-sized plaster cast of a person is modeled using a Bezier curved surface. Approximately 5,000 control points were required to model this plaster image in such a way that every detail could be seen. That is, as shown in FIG. 9(d), these 5000 control points are divided into 4 x 4 control point groups, and the partial area determined by these 4 x 4 control points is divided into u, v. By selecting the value of , it is subdivided into 10 parts. In this way, a structured surface with 500,000 structured points is formed. By creating a display surface that is divided to this extent, an image that can be recognized in fine detail can be obtained.

On the other hand, as long as it is possible to determine whether it is a person or not, the number of control points may be approximately several tens to 100, and furthermore, U.

■ can also be divided into several parts. In other words, it can be expressed using about 100 structural points. In this way, even for objects as small as people, the amount of data varies by a factor of 100 to several thousand times depending on the display environment and necessity. In other words, the layering can be selected in various ways regarding the size of the area.

FIG. 10 is a block diagram for embodying the principle shown in FIGS. 7 to 9 described above. The block diagram shown in FIG. 10 is the same as FIG. 5 described above except for the following points. That is, compared to the processing means 100 shown in FIG. 5, the processing means 101 is provided with a means 31 for generating a higher hierarchy of the target constituent plane, and a distance forming means 32 is provided with a distance forming means 32 for generating a higher hierarchical level of the target constituent plane. Calculate distance. For example, the generating means 31 generates the object ob j 1゜o shown in FIG.
bj2. Generate the main surface of obj3. What is the main aspect here? This is a configuration surface of a rough model as shown in FIG. 8(C) or (d). In Figure 8, the position of each component surface is expressed by a vector to the center (base point) of the component surface in the target coordinate system, and the angle of the component surface is expressed by the normal direction of that surface as the position vector. expressed as an angle. The constituent plane obj-i-j expressed in the object coordinate system is located at the position V ob + in the object coordinate system.
−+ and the angle Rob + −′ can be determined by simple coordinate transformation.

FIG. 11 is a flow diagram for explaining the operation of the embodiment shown in FIG. 10. The flowchart shown in FIG. 11 is the same as the flowchart shown in FIG. 6, with a new step 5P21 added and step 5P22 instead of step SP3. In step 5P21, in step SPI, the target coordinate system setting means 20 sets the target coordinate system position V ob l -1 and the angle R8,1-
If 1 is set, the object coordinate system position ■ for each object obj-i. .

1-1, a three-dimensional model with a hierarchical level (c) is generated with an angle R6b i -1, and the base position V ob 1- + -1 of each constituent surface j of the object obj-i is calculated. In step SP2, the viewpoint detection means 10 determines the viewpoint position ■
When Oe is detected, in step 5P22, the distance calculation means 32 of the processing means 101 calculates the distance d I S obl Vobl from the viewpoint to the base point of each constituent surface of each object.
-1-1-Calculate Vo e l. Then, in steps SP4 to SP5PII, the distance is the reference value Th, (k=
1.2). The other operations are the same as in FIG.

FIG. 12 is a diagram showing another embodiment of the present invention. 1st
In FIG. 2, display objects objl and obj2 are the same as those shown in FIG. Each constituent point and constituent surface data is
The target coordinate system of the object X(obj+) Y(obj
+) Z (obj+) and display reference coordinate system X (w)
If the relationship Y (w) −Z (w) is known,
It can be expressed in the display reference coordinate system. two perspectives 0
(e) R, O (e) L correspond to the principal points of the left and right eye optical systems of each observer. Further, the subscript (−〇) of ○(e)R is the current viewpoint, (+1) is the viewpoint to be moved next, and (−1) and (−2) are the previous viewpoints, respectively.

Figure 12 shows a case where approximately 10 to 30 images are generated and displayed every second, and therefore (+I) (0)
The points (-1) and (-2) are spaced at intervals of approximately 100 to 33 m5ec. Position vector ■ (ob jl), V (o
b j2) is the position vector of the origin of the symmetrical coordinate system of the display target as seen from the viewpoint.

The length of this position vector, that is, the distance, can be easily determined.

Here, the method of selecting the modeling hierarchy is determined by the distance, as in the case shown in FIG.

When the viewpoint is at the position O(e) R(0), the models of display targets objl and obj2 are V(objl)(0),
V (ob j2) (0) depending on the length of the sixth
The display targets shown in Figures (c) and (d) are selected, and h-objl-R(0), h- are displayed on screen S1.
It is projected as ob j2-R(0).

In this embodiment, the viewpoint is that of the eyeball.

Since the viewpoint position is close to the center position of the pupil, if the pupil can be captured by two cameras, it is easy to find this position in the display reference coordinate system by stereo image measurement or the like. For an efficient method of extracting pupils, please refer to
“Image photographing device” by the inventors of the present application (Patent application No. 1-1
81387).

The position of the pupil can be measured by using the "line of sight detection method" (Japanese Patent Application No. 1-296900).

Next, a display that predicts the movement of the viewpoint will be described. The rotational movement of the eyeballs is closely related to visual acuity.

When the target moves slowly with the head fixed, the line of sight moves smoothly in the same direction in conjunction with the movement of the target. Conversely, if you move your head while gazing at a fixed visual target, your line of sight will move smoothly in the opposite direction. In this way, when the line of sight moves smoothly, the visual acuity of a person does not change much compared to when observing a stationary optotype. Furthermore, at this time, the movement of the pupil as seen in the display reference coordinate system is also smooth. Therefore, the position of the pupil,
It is possible to predict the next movement of the pupil from the trajectory such as velocity. Therefore, by generating in advance a perspective image to be displayed using such a predicted pupil position as a viewpoint, it is possible to display the image in synchronization with the movement of the pupil, that is, without any delay time.

9 In Figure 12, O(e) R(-2), 0(
e)R(-1) and O(e)R(-0) are the positions of the pupils that change smoothly. From this position data, the viewpoint position 0 after 100 to 33 m5eC where the next image is displayed
(e) It is possible to predict R(+1). 1st
In Figure 2, the viewpoint approaches the screen, i.e., the display object, with time, and therefore V(objl
)(+1).

The length of V (ob j2) (+1) is shorter.

Therefore, as for the model hierarchy, FIG. 8(b) is selected for the display target objl, and FIG. 8(C) is selected for the display target obj2. Therefore, O(e)
As a perspective projection image onto the screen S1 seen from R(+1), h-ob j 1-R(+
1), h-ob j2R(+1) is prepared.

This image is h-objl-R(0), h-ob
Compared to j2-R(0), it is more detailed because it is one layer higher. Since this image is presented after 100 to 30 m5ec, the observer does not feel any discomfort due to the display delay.

0 Here, if the object of interest of the observer changes and the line of sight or face suddenly moves, the predicted position of the pupil will change, so this method seems unreasonable at first glance. However, there is no problem for the following reasons. That is, as mentioned above, visual acuity is closely related to the movement of the line of sight, and when the line of sight suddenly changes, a person's visual acuity deteriorates and requires more than 200 m5ec to recover. Therefore, even if the predicted viewpoint is incorrect, the observer's visual acuity will be significantly reduced with respect to the generated and displayed image, so the viewer will not feel any discomfort. It is sufficient to measure the correct pupil position and generate and display a perspective projection image from that point until vision recovers.

Note that in this embodiment, the two images are generated in real time following the movement of the eyeballs. Therefore, the viewer can move his head and view the display object obj from various directions, up, down, right and left. Furthermore, since the two images have parallax information, the object can be perceived three-dimensionally.

Additionally, in this embodiment, a binocular stereoscopic display device can be used to display the generated images. A specific example of this device is a ``stereoscopic display device'' (Japanese Patent Application No. 2-18051) by the inventors of the present invention.

The configuration of this device is that a semicircular lens sheet called a lenticular lens is provided on the display surface, and on the back of each lens, pixels for the right eye and pixels for the left eye are arranged on both sides of the focal point. . That is, the two pixels are arranged as a set to face one kamaboko lens. The light emitted from the two pixels is spatially separated by the lens and separately enters the observer's left and right eyes.

In this way, stereoscopic viewing becomes possible without having to wear any special equipment.

FIG. 13 is a block diagram for embodying the embodiment shown in FIG. 12. This embodiment generates and displays images in synchronization with the movement of the viewpoint based on the principle shown in FIG. For this reason, this embodiment newly includes predictive viewpoint detection means 11 compared to the embodiment shown in FIG.
A database 1-2 of viewpoint movement, a distance calculating means 33 between the origin of the target coordinate system and the predicted viewpoint, a means 41 for generating component points/component planes of the object in one screen light, and a secondary distance calculation means 41 at the display time of one screen light. Perspective projection conversion means 5 for generating a projection screen
1 is provided. The predicted viewpoint detection means 11 estimates the viewpoint, that is, the position of the pupil at the next image display time, according to the detection output of the viewpoint detection means 10, while referring to the database 12 regarding viewpoint behavior. The distance calculation means 33 between the origin of the target coordinate system and the predicted viewpoint calculates the distance between the position of the origin of the target coordinate system and the predicted viewpoint.

The constituent point/constituent plane generating means 41 generates a three-dimensional model at the position of the target coordinate system specified by the target coordinate system setting means 20 in the same manner as the generating means 40 . Then, the generating means 41 compares the distance predicted by the distance calculating means 33 with the reference value Th, and generates a three-dimensional hierarchical database 8.
The constituent surface data of an appropriate hierarchy is taken in from 0, and a three-dimensional model at the display time of the next screen is generated. The perspective projection conversion means 51 generates a two-dimensional projection screen at the display time of the next screen. The other operations are the same as those shown in FIG. 35 described above, so the explanation will be omitted.

FIG. 14 is a diagram showing an example of application of the present invention.

The example shown in FIG. 14 allows a computer graphics image to be observed with a rich sense of realism. The viewpoint corresponds to the center position of the pupil or iris. A viewpoint detection device (not shown) performs viewpoint detection 1 on this viewpoint or line of sight, and a detection device (not shown) performs detection 2 of the position and shape of the hand. These data are given to the intention understanding processing unit 3 and analyzed, and the object that the observer is gazing at and the task that the observer is about to perform next are estimated. Based on this information, the 3D model world 6 is updated 5 while referring to the 3D shape database 4, and the 3D model world 6 is generated at high speed.
Image generation 7 is performed using real-time stereoscopic computer graphics, projected onto a screen 8, and displayed on the screen in real time.

[Effects of the Invention] As described above, according to the present invention, the amount of calculation can be significantly reduced for objects that are far from the origin or viewpoint of the display reference coordinate system. Conversely, this extra time can be used to calculate the details of nearby objects. These enable high-speed, high-quality image generation and display. Furthermore, this invention is more effective as the background becomes more complex.

For example, in applications such as immersive conferences where a conference room is generated using computer graphics and images of people are generated inside the conference room, the speed is increased several hundred times because the complex background is roughly modeled. can be expected. Moreover, the display uses the pupil as a viewpoint and predicts this movement to generate an image that is synchronized with the movement of the eyeballs, so there is less discomfort due to delays in image display, and a high-quality interlocking display that improves the sense of unity with the image. Computer graphics image display becomes possible. Furthermore, it becomes possible to display binocular stereoscopic computer graphics at high speed and with high quality. Further, according to the present invention, the application effect is particularly great for the computer graphics processing portion of intelligent encoded communication with a rich sense of reality, and stereoscopic computer graphics. It can also be used in many other computer graphics fields that require high speed and high quality.

[Brief explanation of drawings]

FIG. 1 is a diagram for explaining the invention in detail. FIG. 2 is a diagram showing hierarchical modeling of the display object shown in FIG. FIG. 3 is a diagram showing an example in which a hierarchy is adaptively selected and displayed using the distance from the origin 0 (w) of the display coordinate system to the origin of the target coordinate system as one parameter. FIG. 4 is a diagram showing an example in which the display target is formed by a combination of a plurality of large structures, and an individual target coordinate system can be set for each partial structure. FIG. 5 is a schematic block diagram of one embodiment of the present invention. FIG. 6 is a flow diagram for explaining the operation of the embodiment shown in FIG. FIG. 7 is a diagram showing another example in which the distance of a point other than the origin in the target coordinate system is used as a parameter. FIG. 8 is a diagram showing hierarchically modeled display objects. FIG. 9 is a diagram showing quadratic and cubic Bezier curved surfaces. FIG. 10 is a schematic block diagram showing another embodiment of the invention. FIG. 11 is a flow diagram for explaining the operation of the embodiment shown in FIG. 10. FIG. 12 is a diagram showing another example of the present invention. FIG. 13 is a block diagram for implementing the example shown in FIG. 12. FIG. 14 is a diagram showing an example of application of the present invention. FIG. 15 is a diagram showing a typical conventional three-dimensional image generation and display method. FIG. 16 is a diagram of the coordinate system shown in FIG. 15 viewed from the Y(w) axis. Figure 17 shows the coordinate system shown in Figure 15 on the Y (e) axis and on the Z (
e) Viewed in the axial direction. FIG. 18 is a diagram showing the display target shown in FIG. 17. In the figure, objl is a display target known in the display reference coordinate system, Pi (obj) is a structural point of the display target indicated in the target coordinate system, Sl is a screen, hobj is a perspective projection image, O(e) is a viewpoint, ep represents the screen gaze point, eO represents the spatial gaze point, and Th represents the hierarchy selection criterion.

Claims (4)

[Claims] (1) A high-speed image generation and display method for projecting and displaying an object having a three-dimensional structure on a two-dimensional screen, wherein the constituent plane of the object has an area size of at least 1 in the object coordinate system.
When projecting the constituent surfaces of the object when viewed from an arbitrary viewpoint onto the two-dimensional screen, the object expressed in the object coordinate system from the display reference coordinate system origin or viewpoint is A high-speed image generation and display method, characterized in that a hierarchy level is set using a distance to an arbitrary point as at least one parameter. (2) The arbitrary viewpoint is a pupil or an iris of the observer, and the hierarchical level is set using a distance from the viewpoint to an arbitrary point on the object as at least one parameter. 2. The high-speed image generation and display method according to item 1. (3) Further, the position of the viewpoint at a future image display time is predicted based on the trajectory of the viewpoint, and a projected image viewed from the predicted viewpoint is generated and prepared in advance. Alternatively, the high-speed image generation and display method according to item 2. (4) the position of the pupil or iris is represented in the display reference coordinate system, the position of the object is approximated by the origin of the object coordinate system, the origin is represented in the display reference coordinate system, and the pupil or 3. The high-speed image generation and display method according to claim 2, wherein the distance from the position of the iris to the origin of the object coordinate system is expressed in the display reference coordinate system.