JPH05282466A - Method and device for image processing - Google Patents

Abstract

PURPOSE:To perform accurate interpolation even when variation in the angle in an interpolation section is large and to display an image without irregularity by performing an interpolating process based on an angle of incidence and an angle difference. CONSTITUTION:Model data consist of pairs of coordinate data on the apexes of a polygon which approximately represents a body and data showing tangential plane normal vectors at the points; and the pairs of data are given as to all the vertexes and various constants are also given. As for each of the apexes, coordinate conversion and the calculation of the azimuth angles and a are performed, and pieces of information on a viewpoint position theta and alphalight source position for them are inputted. Then the Z depth of each pixel is found by using the depth-directional distance (Z depth) of the apex on a display screen and angle data theta and alpha on three mutually nearby apexes are used as to each pixel which can be displayed to calculate azimuth angle data thetaand alpha respectively by an interpolating method. Consequently, the accurate interpolation is performed even if the azimuth angle difference between adjacent apexes is large.

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to an image processing method and apparatus in the field of computer graphics.

[0002]

[Background Art and its Problems] In computer graphics, by processing model data representing an object stored in a memory of a computer, data for drawing a shaded image instead of a line drawing is obtained, Various methods have been proposed for displaying more realistic images. However, in any of these proposed methods, the emission brightness (or R, G, or R) of minute points (pixels or pixels) forming the display screen is
(B data) has to be calculated for all pixels, and there is a problem that an extremely long time is wasted due to this calculation processing.

This problem will be described in detail by taking a Phong drawing method, which is excellent in expressing the object model, as an example.

FIG. 1 shows a part of the object model. The surface of a three-dimensional object is approximately represented as a set of triangular facets, that is, a polyhedron. Two triangular facets ABC and ACD are shown in FIG.

The model data of such an object is composed of the position coordinates of the vertices A, B, C, D, etc. of the polyhedron and the tangential plane normal vector. Each vertex A, B, C, each normal vector of _{D ↑ n A, ↑ n B} , ↑ n C, ↑ n D
(In the drawings, a vector is represented by a horizontal arrow → above the reference symbol in the drawing, and in the specification by a vertical arrow ↑ in front of the reference symbol).

First, the tangent plane normal vector at each pixel on the display screen is obtained by the interpolation method using the tangent plane normal vector at the apex.

Consider a scan line SL, and find a tangential plane normal vector at a pixel on this scan line SL.

The normal vectors ↑ n _A , ↑ n _B and ↑ n _C at the vertices A, B and C are set as follows.

↑ n _A = (x _A , y _A , z _A ) Equation 1 ↑ n _B = (x _B , y _B , z _B ) Equation 2 ↑ n _C = (x _C , y _C , z _C )… Equation 3

The points where the line segments AB and AC of the triangle ABC and the straight line SL intersect are defined as Q and R, respectively. The normal vectors at these points (pixels) Q and R are ↑ n _Q and ↑ n _R , respectively.

[0011] _{↑ n Q = (x Q,} y Q, z Q) ... Equation _{4 ↑ n R = (x R} , y R, z R) ... formula 5

It can be easily understood that the x, y and z components of these normal vectors ↑ n _Q and ↑ n _R are given by the following equations according to the linear interpolation method.

[0013] _{x Q = x A + (AQ} / AB) (x B -x A) = [1- (AQ / AB)] x A + (AQ / AB) x B ... Equation 6 y _Q = y _A + _{(AQ / AB) (y B} -y A) = [1- (AQ / AB)] y A + (AQ / AB) y B ... equation _{7 z Q = z A + (} AQ / AB) (z B - _{z A) = [1- (AQ} / AB)] z A + (AQ / AB) z B ... equation _{8 x R = x A + (} AR / AC) (x C -x A) = [1- (AR _{/ AC)] x A + (} AR / AC) x C ... equation _{9 y R = y A + (} AR / AC) (y C -y A) = [1- (AR / AC)] y A + (AR / AC) y _C ... equation _{10 z R = z A + (} AR / AC) (z C -z A) = [1- (AR / AC)] z A + (AR / AC) z C ... equation 11

Here, AB represents the length of the line segment connecting the points A and B. The same applies to AQ, AC, and AR.

Since the normal vectors ↑ n _Q and ↑ n _{R at} the points Q and R are obtained in this manner, the normal vector at the pixel (point) on the line SL between the points QR is similarly obtained. Is the calculated normal vector ↑ n _Q , ↑ n
Calculate by interpolation using _R.

Further, similarly, the normal vector of the pixel on another scanning line is obtained. The normal vector is obtained in the same manner for the pixels on the small plane ACD, and for the pixels representing all the other small planes. As a result, normal vectors are obtained for all points of the polygon representing the object.

Next, the brightness of each pixel is calculated.

As shown in FIG. 2, the normal vector of a pixel P is ↑ n _P. Light from a virtual light source that illuminates an object enters pixel P. Reflection and diffusion of light occur at the pixel P. The shaded image of the object when the reflected light or diffused light (which will be diffused light in general) from the pixel P is viewed (observed) from a certain position is an image to be displayed. Will be calculated.

A vector indicating the direction of the light source viewed from the pixel P is ↑ L, and a vector indicating the direction of the reflected light is ↑ R,
A vector indicating the direction of the position to be observed is defined as ↑ S. The incident angle of light (equal to the reflection angle) is θ, and the angular difference between the regular reflection direction and the observation direction is α.

The brightness (observation brightness) I _P of the point P when viewed from the observation position is given by the following equation.

I _P = k _a I _a + [I _l / (d + K)] (k _d cos θ + k _s cos ⁿ α) Equation 12

Here, I _a : ambient light luminance I _l : light source luminance k _d : diffuse reflection coefficient k _s : specular reflection coefficient k _a : ambient light reflection coefficient d: observation distance (between observation point and point P) Distance) K: Shortest distance parameter n: Order of reflection directionality (representing gloss)

Cos θ and cos α in Expression 12 are given by the following expressions using vectors.

Cos θ = (↑ n · ↑ L) / | ↑ n || ↑ L | ... Equation 13 cosα = (↑ R · ↑ S) / | ↑ R || ↑ S |

Here, the middle point / represents the vector inner product.

Using equations 13 and 14, equation 12 can be written as follows.

I _P = k _a I _a + [I _l / (d + K)] [k _d {(↑ n ・ ↑ L) / (| ↑ n || ↑ L |)} + k _s {(↑ R ・ ↑ S) ⁿ / (| ↑ R || ↑ S |) ⁿ }] ... Equation 15

In computer graphics systems, since model data is often held in the XYZ coordinate system, equation 15 is mainly used, and cos θ and cos ⁿ α are used.
I try to avoid the calculation of.

In any case, the calculation of Expression 12 or Expression 15 is required for each pixel in order to obtain the luminance data of each pixel, the amount of calculation is enormous, and it takes an extremely long time to obtain the shadow image data. Takes.

[0030]

DISCLOSURE OF THE INVENTION An object of the present invention is to provide hardware capable of processing luminance calculation at high speed.

Another object of the present invention is to obtain high quality image data.

According to the present invention, an object model is approximately represented by a polyhedron composed of a large number of small planes, model data composed of position coordinates and normal vectors at each vertex of this polyhedron is given, and this model data is given. In the image processing method for calculating the brightness of many pixels on the display screen using the calculated brightness and drawing the shadow image of the object according to the calculated brightness,
Information about the observation position to see the object and the position of the virtual light source that illuminates the object is given. Using these given information and model data, the incident angle θ of the light from the light source at each vertex and the direction of specular reflection light are given. The angle difference α between the observation direction and the observation direction is calculated, and the incident angle and the angle difference at each pixel for representing the small plane are obtained by the interpolation method using the incident angle and the angle difference at the three adjacent vertices. The brightness of the pixel is calculated using the incident angle and the angle difference in the pixel.

According to the present invention, the interpolation processing for the incident angle and the angle difference is performed in place of the conventional normal vector interpolation described above, so that accurate interpolation can be performed even when the angle change in the interpolation section is large. Therefore, accurate incident angle and angle difference data at each pixel can be obtained. Therefore, even in a model in which a curved surface with a small radius of curvature is represented by a set of small planes, it is possible to display images evenly.

Preferably, the processing for calculating the brightness of each pixel by using the incident angle and the angle difference in each pixel is performed using a dedicated hardware circuit.

The present invention provides a dedicated hardware circuit therefor. The dedicated hardware according to the present invention for calculating the brightness from the incident angle and the angle difference in each pixel is provided with the first adder circuit and the first adder circuit for adding the given observation distance data d and the shortest distance parameter K. Output is an input, a first look-up table that generates an output that represents the reciprocal of the input, a second look-up table that receives the given incident angle data θ and that produces an output that represents cos θ, the given angle difference data α
And a third look-up table for generating an output representing cos ⁿ α by inputting the reflection direction order n, and a first look-up table for computing the product of given light source luminance data I _l and the output of the first look-up table. , A second multiplication circuit for calculating the product of given diffuse reflection coefficient data k _d and the output of the second look-up table, given specular reflection coefficient data k _s and the output of the third look-up table And a second addition circuit for adding the output of the second multiplication circuit and the output of the third multiplication circuit, the output of the first multiplication circuit and the second addition circuit And a third adder circuit for adding the output data of the fourth multiplier circuit to the data k _a × I _a related to the given ambient light amount.

In a preferred embodiment of this invention,
Given shortest distance parameter K, reflection directionality n,
A register for temporarily storing the light source luminance I _l , the diffuse reflection coefficient k _d , the specular reflection coefficient k _s, and the data k _a × I _a relating to the amount of ambient light is further provided.

This dedicated hardware can be constructed so that the entire circuit is divided into a plurality of stages and the arithmetic processing is executed by pipeline processing in synchronization with the clock signal supplied.

As described above, since the circuit for calculating the brightness in each pixel is constructed by the dedicated hardware, the processing speed is remarkably improved.

[0039]

DESCRIPTION OF THE PREFERRED EMBODIMENT FIG. 3 shows the hardware architecture of an electrical circuit that performs the pixel-by-pixel intensity calculation represented by Equation 12. This architecture is configured so that calculation processing proceeds by pipeline operation in synchronization with a clock signal, and stages 1 to 3 are performed.
It consists of three stages. First, calculation processing is performed in stage 1, and when the processing is completed, the calculation result is transferred to stage 2, and calculation processing in stage 2 is performed. Next, the calculation processing result in stage 2 is transferred to stage 3, and the calculation processing in stage 3 is executed. From stage 3, the final result is the observed brightness I _P at pixel P.

The stage 1 has a register 18 for the shortest distance parameter K and the register 19 for the reflection direction order n, and the stage 2 has a register for the light source luminance I _l.
27, a register 28 for the diffuse reflection coefficient k _d and a register 29 for the specular reflection coefficient k _s are provided on the stage 3 with a product of the ambient light reflection coefficient k _a and the ambient light brightness I _a of k _a × I _a . Registers 34 are provided for each of them. These K, n, I _l , k _d , k _s and k _a × I _a are constants and are transferred from the CPU of the computer through the data bus to the corresponding registers 18, 1 before the start of the calculation process.
Pre-stored in 9, 27, 28, 29 and 34 respectively.

In stage 1, the computer CP
Data representing the observation distance d from U to the point P and the angles θ and α are given. The distance data d is added to the adder circuit 17
In step S11, the result is added to the parameter K stored in the register 18, and the addition result d + K is given to the look-up table (LUT) 11. The LUT 11 is a memory in which data representing the reciprocal 1 / x of the input data x is stored in advance. The input data x may be configured to address the storage location of data representing the reciprocal 1 / x. Addition result d + K is LUT11
1 / (d + K) is read from the LUT 11 and temporarily stored in the latch circuit 14.

Incident angle θ of the incident light from the light source with respect to the point P
Is supplied to a look-up table (LUT) 12. The LUT 12 is a memory, and the input data θ
The data representing cos θ is stored in advance in correspondence with. In the LUT 12, data representing the input data θ may be used as an address, and the data cos θ may be stored in a storage location designated by the address. LUT12
The data cos θ read from is temporarily stored in the latch circuit 15.

The data representing the angle difference α between the specular reflection direction and the observation direction for point P is a look-up table (LU
T) given to 13. The LUT 13 is a memory, and data representing cos ⁿ α corresponding to the angle difference α is stored in advance for each reflection direction order n. Data representing the order n is stored in the register 19. In the LUT 13, page switching is performed according to the order n, and the data stored in the register 19 and representing cos ⁿ α for the order n can be accessed. Also in the LUT 13, preferably, the data c of the page opened by the degree n
Os ⁿ α is accessed by using the data representing the angle difference α as an address. Data cos ⁿ read from LUT13
α is temporarily stored in the latch circuit 16.

Data 1 / (d + K), cos θ and co stored in the latch circuits 14, 15 and 16, respectively.
s ⁿ α is transferred to the stage 2 at the timing of the next clock signal and given to the multiplication circuits 21, 22 and 23, respectively.

The multiplication circuit 21 multiplies the constant I _l stored in the register 27 and the input 1 / (d + K) by the product I _l / (d
+ K) is calculated. The multiplication result is temporarily stored in the latch circuit 25.

The multiplication circuit 22 calculates the product of the constant k _d stored in the register 28 and the input data cos θ. The multiplication result k _d cos θ is immediately given to the adder circuit 24. Similarly, the multiplication circuit 23 calculates the product of the constant k _s stored in the register 29 and the input data cos ⁿ α. The multiplication result k _s cos ⁿ α is immediately given to the adder circuit 24.

The adder circuit 24 inputs the multiplication result k _d cos
Add θ and k _s cos ⁿ α. The addition result is temporarily stored in the latch circuit 26.

In this way, while the multiplication and the addition for the previous data are performed in the stage 2, in the stage 1, from the LUTs 11, 12, 13 based on the input data d, θ, α for the next pixel. It goes without saying that the data reading and latching are performed.

By the next clock signal, the latch circuit 2
The data stored in 5 and 26 are transferred to the next stage, stage 3. At this time, the latch circuit of stage 1
It goes without saying that the data temporarily stored in 14 to 16 are transferred to the stage 2.

The data transferred from the latch circuits 25 and 26 to the stage 3 are given to the multiplication circuit 31,
The second term on the right side of is calculated. The result of this operation is the adder circuit 32.
To enter. Since the first term on the right-hand side of Expression 12 is given to the adder circuit 32 from the register 34, the observation luminance data I _P represented by Expression 12 is obtained by adding these two input data. This data _IP is latched by the latch circuit 33.

The luminance data I _P of the latch circuit 33 is output at the timing of the next clock signal. At this time, it goes without saying that the calculated data of stage 1 is sent to stage 2, and the data of stage 2 is sent to stage 3.

In this way, by sequentially inputting the input data d, θ, and α, the brightness I _{P of} each pixel is sequentially obtained for each clock cycle, and a quick brightness calculation is executed.

The constant data of the registers 18, 19, 27, 28, 29 and 34 described above may be fixed until the calculation of the brightness data of all pixels for one object model is completed, or an appropriate constant. May be changed at an appropriate timing. For example, the reflection direction order n, the diffuse reflection coefficient k _d , and the specular reflection coefficient k _s may be different for each small plane forming the polyhedron, so these values are calculated every time the luminance data is calculated for all pixels on the small plane. May be changed.

FIG. 4 shows the overall processing procedure in the computer graphics system. This processing is executed by the CPU of the computer.

First, model data of an object is created or input (step 41). As described above, the model data consists of a pair of coordinate data of the vertices of a polyhedron that approximately represents an object (coordinates in three-dimensional virtual space = world coordinates) and data representing a tangential plane normal vector at that point. , This paired data is given for all vertices.
Also, the above-mentioned various constants I _a , I _l , k _a , d, K, k
_d , k _s , n are given.

Next, for each vertex (step 44), coordinate conversion (step 42) and calculation of azimuth angles θ and α (step 43) are performed. For this purpose, the viewpoint (observation) position,
Information such as the light source position is input.

The coordinate transformation is to transform the world coordinates (three-dimensional) of each vertex into display coordinates (two-dimensional), and the viewpoint position information is used for this transformation. This coordinate conversion process is a known process and is not particularly related to the present invention, so a detailed description thereof will be omitted.

The calculation of the azimuth angles θ and α is performed using Equations 13 and 14.

Normal vector ↑ n _P , light source direction vector ↑
L, observation direction vector ↑ S and reflection direction vector ↑ R
Put as follows.

[0060] the normal _{vector: ↑ n P = (n x} , n y, n z) the light source direction _{vector: ↑ L = (L x,} L y, L z) viewing direction _{vector: ↑ S = (S x,} S _y , S _z ) Reflection direction vector: ↑ R = (R _x , R _y , R _z )

On the other hand, (↑ n · ↑ L) = n _x L _x + n _y L _y + n _z L _z Equation 16 | ↑ n | = (n _x ² + _ny ² + _nz ² ) ^1/2 Equation 17 | ↑ L | = (L _x ² + L _y ² + L _z ² ) ^{1/2 Since} equation 18 is substituted, cos θ = (n _x L _x + n _y L _y + n _z L _z ) / [(N _x ² + _ny ² + _nz ² ) (L _x ² + L _y ² + L _z ² )] ^1/2 ... Formula 19 θ = cos ^-1 [(n _x L _x + ny _y L _y + _{nz z} ) / [(N _x ² + _ny ² + _nz ² ) (L _x ² + L _y ² + L _z ² )] ^1/2 ... Equation 20

Similarly, the angle difference α is also obtained by the following equation.

Α = cos ⁻¹ [(R _x S _x + R _y S _y + R _z S _z ) / [(R _x ² + R _y ² + R _z ² ) (S _x ² + S _y ² + S _z ² )] ^{1 / 2} ... Equation 21

Next, the processes of steps 45 to 48 are performed for all the pixels on the display screen (step 49).

The Z depth of each pixel is obtained by the linear interpolation method using the distance (Z depth) in the depth direction of the vertices on the display screen (step 45), and hidden by the image with shallow Z depth and cannot be displayed. Is checked (step 46). Since such hidden points are not displayed, it is not necessary to calculate the luminance, and therefore the process does not proceed to steps 47 and 48 below.

For each displayable pixel, the azimuth angle data θ and α are calculated by the interpolation method using the angle data θ and α of the three vertices close to the pixel (step 47). The interpolation method of the data θ and α of each pixel is exactly the same as the interpolation method of the normal vector described with reference to FIG. In Equation 6, x _Q is θ _Q or α _Q , x _A is θ _A
Alternatively, α _A and x _B may be replaced with θ _B or α _B. In equation 9, x _R is θ _R or α _R , x _A
Can be replaced with θ _A or α _A , and x _C can be replaced with θ _C or α _C.

FIG. 5 shows the difference between the two interpolation methods.

A vector created from the vectors ↑ V1 and ↑ V2 by the interpolation method using the above-described conventional orthogonal coordinate component is ↑ V4 shown by a chain line. According to this interpolation method, the vector ↑ V4 becomes shorter than the vectors ↑ V1 and ↑ V2, and not only the preservation of the vector length cannot be established, but also the angle change becomes large in the vector ↑ V4 near the center. This becomes remarkable especially when the angle difference between the vectors ↑ V1 and ↑ V2 is large.

The vector ↑ V3 shown by the solid line is the vector ↑ V3.
It is created by angle interpolation from V1 and ↑ V2.
According to the angle interpolation, the length of the created vector ↑ V3 is also saved, and the mutual angular intervals can be set equally.

As described above, by interpolating the azimuth angles θ and α in each pixel, accurate interpolation can be performed even when the difference (change) in azimuth angles between adjacent vertices is large. For example, a model in which a curved surface with a small radius of curvature is represented by a set of small planes can be displayed on a screen with less unevenness.

In this way, when the angles θ and α are obtained for each pixel, the hardware circuit shown in FIG. 3 is provided with the observation distance data d together with these data θ and α so that the luminance data I of each pixel is obtained. _P is obtained (step 48).

When there are a plurality of object models, the above-mentioned steps 42 to 49 are repeated for each object (step 50). At this time, the brightness calculation is not performed for the portion of the rear object that is hidden by the front object, as determined in step 46.

The object model is displayed on the display screen of the display device using the brightness data for each pixel thus obtained (step 51). If the data for the object model is given for the three primary colors R, G, B, the luminance data for the three primary colors are obtained, and the object is displayed in color.

[Brief description of drawings]

FIG. 1 shows how normal vectors are interpolated in the image generation processing by the Phong method.

FIG. 2 is a vector diagram for explaining calculation of pixel luminance in the image generation processing by the Phong method.

FIG. 3 is a block diagram showing dedicated hardware for calculating luminance from an incident angle and an angular difference in each pixel according to the embodiment of the present invention.

FIG. 4 is a flow chart showing an image processing procedure according to the embodiment of the present invention.

FIG. 5 is a vector diagram showing a difference between the interpolation of the direction vector by the orthogonal coordinate component and the interpolation of the direction vector by the angle.

[Explanation of symbols]

 11, 12, 13 Look-up table 14, 15, 16, 25, 26, 33 Latch circuit 17, 24, 32 Adder circuit 21, 22, 23, 31 Multiplier circuit 18, 19, 27, 28, 29, 34 register

Claims (5)

[Claims] 1. An object model is approximately represented by a polyhedron composed of a large number of small planes, model data composed of position coordinates and normal vectors at each vertex of this polyhedron is given, and this model data is used. In the image processing method for obtaining the brightness of many pixels on the display screen and drawing the shadow image of the object according to the calculated brightness, information about the observation position to see the object and the position of the virtual light source illuminating the object is given. Incident angle θ from the light source at each vertex using the given information and model data of
And the angle difference α between the direction of the specularly reflected light and the observation direction is calculated, and the incident angle and the angle difference at each pixel for representing the facet are interpolated by using the incident angles and the angle differences of the three adjacent vertices. An image processing method in which the brightness of each pixel is calculated using the angle of incidence and the angle difference in each pixel that is calculated. 2. The image processing method according to claim 1, wherein the process of calculating the brightness of the pixel by using the incident angle and the angle difference in each pixel is performed by using a circuit with dedicated hardware. 3. A first adder circuit (1) for adding given observation distance data (d) and shortest distance parameter (K).
7), the first look-up table (11) that receives the output of the first adder circuit as an input and generates an output that represents the reciprocal of the input, receives the incident angle data (θ) as input, and c
A second look-up table (12) for generating an output representing osθ, a third look-up table for inputting given angular difference data (α) and reflection directionality (n), and generating an output representing cos ⁿ α An up table (13), a first multiplication circuit (21) for calculating a product of given light source luminance data (I _l ) and an output of the first lookup table, and given diffuse reflection coefficient data (k _d ). A second multiplication circuit (22) for calculating the product of the output of the second lookup table, the given specular reflection coefficient data (k _s ) and the third
A third multiplication circuit (23) for calculating the product of the output of the lookup table and the second addition circuit (24) for adding the output of the second multiplication circuit and the output of the third multiplication circuit, A fourth multiplication circuit (31) that calculates the product of the output of the first multiplication circuit and the output of the second addition circuit, and the data (k _a × I _a ) related to the given amount of ambient light and the fourth multiplication circuit An image processing device comprising a third addition circuit (32) for adding the output and. 4. A given minimum distance parameter (K),
A register (18) for temporarily storing the reflection direction order (n), the light source luminance (I _l ), the diffuse reflection coefficient (k _d ), the specular reflection coefficient (k _s ), and the data (k _a × I _a ) related to the ambient light amount. , 19, 27, 28, 29, 34),
The image processing apparatus according to claim 3. 5. The image processing apparatus according to claim 3, wherein the image processing apparatus is divided into a plurality of stages and executes an arithmetic operation by pipeline processing in synchronization with a supplied clock signal.