JP2011028641A - Image processing device and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing device capable of efficiently executing anti-aliasing without storing image information magnified in each direction to a memory. <P>SOLUTION: The image processing device includes: an interpolation unit which determines a plurality of depths and a plurality of distances for an interpolation position by executing processing to each of a plurality of pixels adjacent to the interpolation position based on the depth of a certain pixel adjacent to the interpolation position and the distance to an end of a pattern including the pixel to determine the depth and the distance to the end of the pattern in the interpolation position for each pixel; and a pixel color determination unit which determines a color in the interpolation position based on colors of the plurality of pixels, and the depths and distances in the interpolation position determined by the interpolation unit. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present disclosure generally relates to an image processing apparatus and an image processing method, and more particularly to an image processing apparatus and an image processing method for performing anti-aliasing processing in three-dimensional graphics.

  In the three-dimensional graphics processing, antialiasing processing is used as a technique for removing a jagged feeling (jaggy) around a figure. Common methods for anti-aliasing include full scene anti-aliasing and multi-sample anti-aliasing. Full scene anti-aliasing is a method of drawing an enlarged image that is n times the size and width of a desired image and reducing the enlarged image to 1 / n. In this method, the processing efficiency is low because the drawing process of the enlarged image of n × n times is performed. Multi-sample anti-aliasing is a method that reduces the processing amount for this full scene anti-aliasing.

  In the multi-sample anti-aliasing, only the depth information and the inside / outside determination data are multiplied n times vertically and horizontally, and the same amount of color information is used for n × n pixels, thereby reducing the amount of calculation. However, even when this multi-sample anti-aliasing is used, n × n times of memory access is required for graphics drawing, and further memory access is required for processing to reduce to 1 / n. As a result, a large bus bandwidth is required.

JP 2001-251500 A JP 2005-38415 A Japanese Patent No. 2682191

  In view of the above, there is a demand for an image processing apparatus that can efficiently perform anti-aliasing without storing image information expanded vertically and horizontally in a memory.

  The image processing apparatus performs a process of obtaining a depth at the interpolation position and a distance to the edge of the graphic based on a depth of a pixel adjacent to the interpolation position and a distance to the edge of the graphic including the pixel. An interpolation unit that calculates a plurality of depths and a plurality of distances at the interpolation position by executing each of the plurality of pixels adjacent to the position; and the interpolation unit that calculates the colors of the plurality of pixels and the interpolation unit And a pixel color determining unit configured to determine a color at the interpolation position based on the plurality of depths at the position and the plurality of distances.

  In the image processing method, a process for obtaining a depth at the interpolation position and a distance to the edge of the graphic based on a depth of a pixel adjacent to the interpolation position and a distance to the edge of the graphic including the pixel is performed by the interpolation. By executing for each of a plurality of pixels adjacent to the position, a plurality of depths and a plurality of distances at the interpolation position are obtained, and the plurality of depths, the plurality of distances, and the colors of the plurality of pixels And determining the interpolation color at the interpolation position, and mixing the interpolation color determined for a plurality of interpolation positions adjacent to the position of one target pixel and the color of the one target pixel, It includes each step for obtaining a new color of one pixel of interest.

  According to at least one embodiment of the present disclosure, the interpolation color at the interpolation position is determined based on the colors of the plurality of pixels and the plurality of depths and the plurality of distances at the interpolation position obtained by the interpolation unit. Accordingly, it is possible to efficiently determine the pixel color at the interpolation position and execute the anti-aliasing process using this pixel color without storing the image information enlarged vertically and horizontally in the memory.

It is a figure which shows an example of the whole structure of an image processing system. It is a figure which shows typically the triangle used as the object of graphic drawing. It is a figure which shows typically the distance to the edge of the figure in which the pixel is included about the pixel of interest. It is a figure for demonstrating the increment of the x direction of the distance to the edge of the figure about a pixel. It is a figure for demonstrating the increment of the y direction of the distance to the edge of the figure about a pixel. It is a figure which shows an example of the data structure of the data stored in the frame buffer of FIG. It is a figure for demonstrating the process of a depth and distance interpolation part. It is a figure for demonstrating the process of a depth and distance comparison part and a pixel color determination part. It is a figure which shows another example for demonstrating the process of a depth / distance interpolation part, a depth / distance comparison part, and a pixel color determination part. It is a figure which shows another example for demonstrating the process of a depth / distance comparison part and a pixel color determination part. It is a figure for demonstrating the process of a pixel blend part. It is a figure for demonstrating the meaning of the distance value in the interpolation position shown in FIG. It is a figure for demonstrating the meaning of the distance value in the interpolation position shown in FIG. It is a figure for demonstrating the meaning of the distance value in the interpolation position shown in FIG. It is a flowchart which shows the flow of the anti-aliasing process by an anti-aliasing circuit. It is a flowchart which shows the flow of the interpolation color determination process performed in FIG. It is a figure which shows an example of the more concrete circuit structure of an anti-aliasing circuit. It is a figure for demonstrating the read-write process of pixel information holding RAM.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating an example of the overall configuration of an image processing system. The image processing system in FIG. 1 includes a graphics processing circuit 10, an antialiasing circuit 11, a frame buffer 12, and a display device 13. The graphics processing circuit 10 includes a vertex buffer 21, a coordinate transformation / lighting processing unit 22, a vertex post-processing unit 23, a rasterization processing unit 24, a pixel processing unit 25, a pixel post-processing unit 26, a memory access unit 27, a texture unit 28, and A distance information calculation unit 29 is included. The anti-aliasing circuit 11 includes a depth / distance interpolation unit 31, a depth / distance comparison unit 32, a pixel color determination unit 33, and a pixel blend unit 34.

  The graphics processing circuit 10 calculates a two-dimensional image that should be seen when a three-dimensional object given as three-dimensional data is viewed from a predetermined viewpoint, and writes the data for each pixel in the frame buffer 12 to thereby generate two-dimensional image data. Is generated. The two-dimensional image data stored in the frame buffer 12 is processed by the antialiasing circuit 11. The two-dimensional image data processed by the anti-aliasing circuit 11 and written in the frame buffer 12 is then read out as drawing data and displayed on the display device 13.

  In general, a three-dimensional object is expressed as an aggregate of many polygons. At this time, a triangle is generally used as the simplest polygon. The three-dimensional data of the three-dimensional object to be drawn is given as vertex data including vertex coordinate data indicating the coordinates of each vertex of the triangle and vertex color data indicating the color of each vertex. This vertex data is stored in the vertex buffer 21 of the graphics processing circuit 10. Based on the three-dimensional data of the three-dimensional object read from the vertex buffer 21, the coordinate conversion / lighting processing unit 22 performs geometric calculation processing such as enlargement / rotation according to the position / posture of the object. The coordinate conversion / lighting processing unit 22 further performs coordinate conversion in units of polygons (triangles), and also performs arithmetic processing such as light source calculation. Data of a plurality of triangles constituting a two-dimensional image of an object viewed from a desired viewpoint is calculated by these geometric calculation processing, coordinate conversion, light source processing, and the like. The vertex post-processing unit 23 determines whether or not the calculated vertex coordinates protrude from the display screen frame, and when the vertex coordinates protrude, the vertex is divided so as to fit within the display screen. The vertex post-processing unit 23 further calculates an increment value necessary for the drawing process from the vertex data. Here, the increment value is, for example, the X coordinate along the side of the triangle and the inclination (change) of each image parameter.

  The rasterization processing unit 24 and the pixel processing unit 25 generate two-dimensional drawing data by rasterization processing that fills each pixel of each surface constituting the object. In the rasterizing process, the image parameter of each pixel of each span included in the triangle is obtained by interpolation or the like based on the image parameter for each vertex of each triangle and the above increment value. The image parameters include, for example, the brightness of each RGB color, a Z value representing a distance in the depth direction, a texture coordinate value for texture display, an alpha value for alpha blending, and the like. The pixel post-processing unit 26 accesses the frame buffer 12 via the texture unit 28 using the texture coordinates of each pixel generated by the rasterization process, and reads the texture data. The pixel post-processing unit 26 performs blend processing of the read texture data and color data.

  The distance information calculation unit 29 calculates the distance to the end of the figure (triangle) including the pixel for each pixel, and further calculates the distance corresponding to the position change of one pixel in each direction in the X direction and the Y direction. Calculate the increment value. The memory access unit 27 writes the pixel color, depth (depth), depth gradient (depth increment value), distance, and distance gradient (distance increment value) to the frame buffer 12 for each pixel in the span to be written. . At this time, the memory access unit 27 accesses the memory address of each pixel of the span to be written, and reads the image data from the frame buffer 12. The pixel data read from the frame buffer 12 is combined with the generated pixel data of each span. At this time, for example, hidden line processing is performed based on the Z value, or alpha blending is performed based on the alpha value.

  Pixel color data and depth data can be obtained by a normal rasterization process. The depth inclination data indicates the increment value of the depth corresponding to the position change of one pixel in each direction of the x direction and the y direction, and is used when obtaining the depth of the pixel on each span in the rasterizing process. It is. Accordingly, the depth inclination data may be output from the pixel processing unit 25 that executes the processing of each pixel in the rasterizing process. The distance data and the slope data of the distance may be calculated by the distance information calculation unit 29 as described above. Calculation of distance data and distance inclination data by the distance information calculation unit 29 will be described below.

  FIG. 2 is a diagram schematically showing a triangle to be subjected to graphic drawing. The triangle is composed of three straight lines E0 (x, y), E1 (x, y), and E2 (x, y). The straight line E0 (x, y) can be expressed as E0 (x, y) = a0x + b0y + c0 = 0. The straight line E1 (x, y) can be expressed as E1 (x, y) = a1x + b1y + c1 = 0. The straight line E2 (x, y) can be expressed as E2 (x, y) = a2x + b2y + c2 = 0. Using these straight line equations, the inner region of the triangle is a point (x, y) that satisfies E0 (x, y)> 0 and E1 (x, y)> 0 and E2 (x, y)> 0. Can be defined as a set of

FIG. 3 is a diagram schematically illustrating the distance from the pixel of interest to the end of the graphic including the pixel. The distance d0 (x, y) from the point 36 of the coordinate (x, y) to the side 37 represented by E0 (x, y) is
d0 (x, y) = E0 (x, y) / D0
It becomes. Here, D0 = (a0 ² + b0 ² ) ^1/2 . Similarly, the distance d1 (x, y) to the side 38 represented by E1 (x, y) is
d1 (x, y) = E1 (x, y) / D1
It becomes. Here, D1 = (a1 ² + b1 ² ) ^1/2 . Furthermore, the distance d2 (x, y) to the side 39 represented by E2 (x, y) is
d2 (x, y) = E2 (x, y) / D2
It becomes. Here, D2 = (a2 ² + b2 ² ) ^1/2 . D0 (x, y), d1 (x, y), and d2 (x, y) obtained in this way are distances to the end of the figure for the pixel 36.

FIG. 4 is a diagram for explaining the increment in the x direction of the distance d0 (x, y) to the edge of the figure for the pixel 36. FIG. In FIG. 4, the distance from the pixel 36 to the end of the figure is d0 (x, y), and the distance at the pixel 40 at the position moved +1 pixel from the pixel 36 in the x-axis direction is d0 (x + 1, y). is there. At this time, the increment value d (d0) dx corresponding to the position change of one pixel in the x direction is obtained as d0 (x + 1, y) −d0 (x, y). The distance d0 (x + 1, y) at the pixel 40 at the coordinates (x + 1, y) is
d0 (x + 1, y) = E0 (x + 1, y) / D0
= {E0 (x, y) + a0} / D0
= D0 (x, y) + a0 / D0
It becomes. Therefore, the increment value in the x direction of the distance d0 (x, y) is a0 / D0.

FIG. 5 is a diagram for explaining the increment in the y direction of the distance d0 (x, y) to the edge of the figure for the pixel 36. In FIG. In FIG. 5, the distance from the pixel 36 to the end of the figure is d0 (x, y), and the distance at the pixel 41 at the position moved by +1 pixel from the pixel 36 in the y-axis direction is d0 (x, y + 1). is there. At this time, the increment value d (d0) dy corresponding to the position change of one pixel in the y direction is obtained as d0 (x, y + 1) -d0 (x, y). The distance d0 (x, y + 1) at the pixel 41 at the coordinates (x, y + 1) is
d1 (x, y + 1) = d0 (x, y) + b0 / D0
It becomes. Therefore, the increment value in the y direction of the distance d0 (x, y) is b0 / D0.

  Similarly, the increment value in the x direction of the distance d1 (x, y) is a1 / D1, and the increment value in the y direction is b1 / D1. Further, the increment value in the x direction of the distance d2 (x, y) is a2 / D2, and the increment value in the y direction is b2 / D2.

  FIG. 6 is a diagram showing an example of the data structure of data stored in the frame buffer 12 of FIG. In the example of FIG. 6, 32-bit data is stored in the frame buffer 12 as pixel color data. Similarly, 32-bit data is stored in the frame buffer 12 as depth (depth value) data. For depth inclination (increment of depth value), 8-bit data is stored as incremental data in the x direction, and 8-bit data is stored as incremental data in the y direction. As for the distance increment, 8-bit data is stored as incremental data in the x direction and 8-bit data is stored as incremental data in the y direction for each of the distances d0, d1, and d2. Further, 10-bit data is stored for each of the distances d0, d1, and d2. As described above, the pixel color and the depth are set to 4-byte accuracy, the depth and distance increment values are set to 1-byte accuracy, and the distance is set to 10-bit accuracy, thereby reducing the total number of bits per pixel. It can be 160 bits. In practice, sufficiently high quality anti-aliasing processing can be realized even when the accuracy of the increment value data and the distance data of the depth and distance is reduced.

  As described above, even if the configuration is 160 bits per pixel, the 160-bit data may be written and read once for each pixel. On the other hand, in the multi-sample anti-aliasing of 2 × 2 times, each pixel is written and read out with respect to data of 4 times the amount of 32-bit pixel color data and depth data. Therefore, in the case of 2 × 2 multisample anti-aliasing, it is necessary to read / write data of 64 × 4 and 256 bits per pixel. As described above, by executing the anti-aliasing process described below using data as shown in FIG. 6, the number of bits per pixel and the memory access amount can be greatly reduced.

  In the anti-aliasing circuit 11 of FIG. 1, the depth / distance interpolation unit 31 determines the depth at the interpolation position and the figure based on the depth of a pixel adjacent to the interpolation position and the distance to the edge of the figure including the pixel. The process which calculates | requires the distance to an edge is performed. The depth / distance interpolation unit 31 obtains a plurality of depths and a plurality of distances at the interpolation position by executing the above-described processing on each of a plurality of pixels adjacent to the interpolation position. The depth / distance comparison unit 32 compares a plurality of depths and a plurality of distances at the interpolation position obtained by the depth / distance interpolation unit 31 between a plurality of pixels. The pixel color determination unit 33 determines the color at the interpolation position based on the color of the plurality of pixels and the plurality of depths and the plurality of distances at the interpolation position obtained by the depth / distance interpolation unit 31. At this time, the pixel color determination unit 33 determines the color at the interpolation position based on the comparison result of the plurality of depths by the depth / distance comparison unit 32, the comparison result of the plurality of distances, and the colors of the plurality of pixels. Good. In addition, the pixel blending unit 34 mixes the color determined by the pixel color determining unit 33 for a plurality of interpolation positions adjacent to the position of one target pixel and the color of the one target pixel, thereby combining the one target pixel. Find the new color of the pixel. The new color (color luminance data) thus obtained becomes pixel data after alias reduction.

  FIG. 7 is a diagram for explaining processing of the depth / distance interpolation unit 31. Pixels A to D are shown in FIG. These pixels A to D are pixels written to the frame buffer 12 by the texture unit 28 of the graphics processing circuit 10. As shown in (a), an interpolation position 51 is assumed between the pixels A to D. Next, as shown in (b), the depth / distance interpolation unit 31 is based on the distance of the pixel A at the coordinates (x, y) (the distance from the pixel A to the end of the figure including the pixel A). The distance at the interpolation position adjacent to the pixel A is obtained. Specifically, the smallest value among the distances d0 (x, y), d1 (x, y), and d2 (x, y) of the pixel A is the distance value of the pixel A, and d0 (x + 0.5, y ), D1 (x + 0.5, y), and d2 (x + 0.5, y) are set to the smallest value as the distance value of the interpolation position (x + 0.5, y). Here, d0 (x + 0.5, y), d1 (x + 0.5, y), and d2 (x + 0.5, y) can be obtained as follows.

d0 (x + 0.5, y) = d0 (x, y) + a0 / 2D0
d1 (x + 0.5, y) = d1 (x, y) + a1 / 2D1
d2 (x + 0.5, y) = d2 (x, y) + a2 / 2D2
In the example shown in FIG. 7B, the distance value of the interpolation position (x + 0.5, y) obtained from the pixel A is 0.5. Similarly, the smallest value among d0 (x, y + 0.5), d1 (x, y + 0.5), and d2 (x, y + 0.5) is set as the distance value of the interpolation position (x, y + 0.5). . In the example shown in FIG. 7B, the distance value of the interpolation position (x, y + 0.5) obtained from the pixel A is 0.3. Furthermore, the smallest value among d0 (x + 0.5, y + 0.5), d1 (x + 0.5, y + 0.5), and d2 (x + 0.5, y + 0.5) is set to the interpolation position (x + 0.5, y + 0.5). ) Distance value. In the example shown in FIG. 7B, the distance value of the interpolation position (x + 0.5, y + 0.5) obtained from the pixel A is 0.7.

  Further, as shown in FIG. 7C, the depth / distance interpolation unit 31 is based on the distance of the pixel B at the coordinates (x, y + 1) (the distance from the pixel B to the end of the graphic including the pixel B). Thus, the distance at the interpolation position adjacent to the pixel B is obtained. In this example, the distance value of the interpolation position (x + 0.5, y + 1) obtained from the pixel B is 0.1. The distance value of the interpolation position (x, y + 0.5) obtained from the pixel B is −0.1. Further, the distance value of the interpolation position (x + 0.5, y + 0.5) obtained from the pixel B is −0.2.

  FIG. 12 is a diagram for explaining the meaning of the distance value at the interpolation position between the pixel A and the pixel B shown in FIG. FIG. 12 shows the pixels A to D shown in FIG. The distance obtained from the pixel A is 0.3 at the interpolation position 51 between the pixel A and the pixel B shown in FIG. This means that the distance from the interpolation position 51 to the end of the triangle 61 that is a graphic including the pixel A is 0.3. Further, at the interpolation position 51 between the pixel A and the pixel B shown in FIG. 12, the distance obtained from the pixel B is −0.1. This means that the distance from the interpolation position 51 to the end of the triangle 62 that is a graphic including the pixel B is −0.1. Here, when the distance is a positive value, it indicates that the interpolation position is located in the corresponding figure, and when the distance is a negative value, the interpolation position is located outside the corresponding figure. It shows that. That is, the interpolation position 51 is inside the triangle 61 but outside the triangle 62. In this case, the interpolated color (interpolated color) at the interpolation position 51 is preferably the color of the pixel inside the triangle 61.

  FIG. 8 is a diagram for explaining processing of the depth / distance comparison unit 32 and the pixel color determination unit 33. As shown in FIG. 8A, the distance obtained from the pixel A at the interpolation position (shown as a virtual pixel E) between the pixel A and the pixel B is 0.3, and the distance obtained from the pixel B Is -0.1. The depth / distance comparison unit 32 compares the plurality of distances (0.3 and −0.1). In this case, since the smaller distance (−0.1) is a negative value, as described with reference to FIG. 12, when determining the interpolation color of the virtual pixel E at the interpolation position, the pixel corresponding to this negative distance. The pixel color of B is not considered. Accordingly, the pixel color determination unit 33 uses the same color as the pixel color of the pixel A corresponding to the positive distance as the color of the virtual pixel E as shown in FIG.

  FIG. 9 is a diagram illustrating another example for explaining processing of the depth / distance interpolation unit 31, the depth / distance comparison unit 32, and the pixel color determination unit 33. In the example of FIG. 9, as shown in FIG. 9A, the distance obtained from the pixel A at the interpolation position (shown as a virtual pixel E) between the pixel A and the pixel B is 0.3, and the pixel B The distance obtained from is 0.2.

  FIG. 13 is a diagram for explaining the meaning of the distance value at the interpolation position between the pixel A and the pixel B shown in FIG. FIG. 13 shows the pixels A to D shown in FIG. In the interpolation position 51 between the pixel A and the pixel B shown in FIG. 13, the distance obtained from the pixel A is 0.3. This means that the distance from the interpolation position 51 to the end of the triangle 61 that is a graphic including the pixel A is 0.3. Further, the distance obtained from the pixel B at the interpolation position 51 between the pixel A and the pixel B shown in FIG. 13 is 0.2. This means that the distance from the interpolation position 51 to the end of the triangle 63 that is a graphic including the pixel B is 0.2. Here, when the distance is a positive value, it indicates that the interpolation position is located in the corresponding figure, and therefore the interpolation position 51 is inside both triangles 61 and 63. Thus, when a plurality of distances have positive values, that is, when the interpolation position is included in a plurality of figures, the interpolation color of the interpolation position is determined based on the depth value of the interpolation position. Specifically, since the triangle corresponding to the smaller depth value of the interpolation position should be in front, the interpolation color (interpolation color) of the interpolation position 51 is the pixel inside the triangle in front of this. It is preferable to use a color.

  As shown in FIG. 9B, the depth / distance interpolation unit 31 obtains the depth at the interpolation position adjacent to the pixel A based on the depth value of the pixel A at the coordinates (x, y). In this example, the depth value of the interpolation position (x + 0.5, y) obtained from the pixel A is 24. The depth value of the interpolation position (x, y + 0.5) obtained from the pixel A is 23. Further, the depth value of the interpolation position (x + 0.5, y + 0.5) obtained from the pixel A is 27.

  Further, as shown in FIG. 9C, the depth / distance interpolation unit 31 obtains the depth at the interpolation position adjacent to the pixel B based on the depth value of the pixel B at the coordinates (x, y + 1). In this example, the depth value of the interpolation position (x + 0.5, y + 1) obtained from the pixel B is 1. The depth value of the interpolation position (x, y + 0.5) obtained from the pixel B is 8. Further, the depth value of the interpolation position (x + 0.5, y + 0.5) obtained from the pixel B is 4.

  As shown in (d), at the interpolation position (shown as virtual pixel E) between pixel A and pixel B, the depth value obtained from pixel A is 23, and the depth value obtained from pixel B is 8 It is. The depth / distance comparison unit 32 compares the plurality of depth values (23 and 8). In this case, the pixel corresponding to the smaller depth value (8) is the pixel B. Therefore, as described with reference to FIG. 12, the pixel color of the pixel A is not taken into account when determining the interpolation color of the virtual pixel E at the interpolation position. As shown in (e), the pixel color determination unit 33 uses the same color as the pixel color of the pixel B corresponding to the smaller depth value as the color of the virtual pixel E. As described above, the pixel color determining unit 33 obtains the smallest one of the plurality of depths corresponding to the non-negative values among the plurality of distances, and interpolates the color of the pixel corresponding to the smallest depth. The color in.

  FIG. 10 is a diagram illustrating still another example for explaining the processing of the depth / distance comparison unit 32 and the pixel color determination unit 33. In the example of FIG. 10, as shown in FIG. 10A, the distance obtained from the pixel A at the interpolation position (shown as a virtual pixel E) between the pixel A and the pixel B is −0.3, and the pixel The distance obtained from B is -0.2.

  FIG. 14 is a diagram for explaining the meaning of the distance value at the interpolation position between the pixel A and the pixel B shown in FIG. FIG. 14 shows the pixels A to D shown in FIG. In the interpolation position 51 between the pixel A and the pixel B shown in FIG. 14, the distance obtained from the pixel A is −0.3. This means that the distance from the interpolation position 51 to the end of the triangle 64 that is a graphic including the pixel A is −0.3, and the interpolation position 51 is outside the triangle 64. Further, in the interpolation position 51 between the pixel A and the pixel B shown in FIG. 14, the distance obtained from the pixel B is −0.2. This means that the distance from the interpolation position 51 to the end of the triangle 65 that is a graphic including the pixel B is −0.2, and the interpolation position 51 is outside the triangle 65. In this way, when the plurality of distances all have negative values, that is, when the interpolation position is outside of all the figures, the colors of the plurality of pixels are synthesized according to the ratio of the plurality of distances to obtain the interpolation position. Determine the interpolation color.

  As shown in FIG. 10A, the distance obtained from the pixel A at the interpolation position (shown as a virtual pixel E) between the pixel A and the pixel B is −0.3, and is obtained from the pixel B. The distance is -0.2. The depth / distance comparison unit 32 compares the plurality of distances (−0.3 and −0.2) to obtain the ratio of the distances. That is, 0.3 / (0.3 + 0.2) = 0.6 and 0.2 / (0.3 + 0.2) = 0.4 are obtained. As shown in (b), the pixel color determining unit 33 weights the virtual pixel E as the color of the virtual pixel E using the ratios 0.4 and 0.6, and determines the pixel color of the pixel A and the pixel color of the pixel B. Mix. (C) shows a state in which the color of the virtual pixel E is obtained by mixing the pixel color of the pixel A and the pixel color of the pixel B.

  FIG. 11 is a diagram for explaining the processing of the pixel blend unit 34. The interpolation color of the virtual pixel E can be obtained by the procedure described with reference to FIGS. Similarly, the interpolation color of the virtual pixel F and the interpolation color of the virtual pixel G can be obtained. Note that how to obtain the interpolation color of the virtual pixel G will be described later. FIG. 11A shows a state in which an interpolation color (interpolation color) is obtained for each of the virtual pixels E to G. The pixel blending unit 34 mixes the color determined by the pixel color determining unit 33 and the color of one target pixel A for a plurality of interpolation positions (virtual pixels E to G) adjacent to the position of one target pixel A. Thus, a new color (color after anti-aliasing) of one target pixel A is obtained. For example, the pixel blending unit 34 calculates the average of the color of the pixel A, the color of the virtual pixel E, the color of the virtual pixel F, and the color of the virtual pixel G, thereby obtaining the pixel value of the pixel A after antialiasing. That is, assuming that each pixel color is A, E, F, and G, (A + E + F + G) / 4 is calculated. FIG. 11B shows a state in which the pixel value A ′ after antialiasing processing of the pixel A is obtained. Similarly, the pixel blending unit 34 obtains a pixel value after anti-aliasing for each pixel stored in the frame buffer 12.

  FIG. 15 is a flowchart showing the flow of anti-aliasing processing by the anti-aliasing circuit 11. In step S1, the coordinate value y is initialized to 0, and while incrementing y by 1, the following steps are repeated in a loop until the y value of the screen height is reached. In step S2, the coordinate value x is initialized to 0, and while incrementing x by 1, the following steps are repeated in a loop until the x value of the screen width is reached. That is, each pixel of the image data stored in the frame buffer 12 is read out as a pixel of interest by the loop of steps S1 and S2, and antialiasing processing is executed in order. First, in step S3, the pixel of interest is the pixel A, and the pixel value of the virtual pixel E is obtained based on the pixels A and B as described above. In step S4, the pixel value of the virtual pixel F is obtained based on the pixel A and the pixel C. In step S5, the pixel value of the virtual pixel G is obtained based on the pixels A to D. In step S7, for example, an average value of the pixels A, E, F, and G is obtained to obtain a pixel value of the pixel A after the antialiasing process. When the loop started in steps S1 and S2 ends, the anti-aliasing process ends.

  FIG. 16 is a flowchart showing the flow of interpolation color determination processing executed in steps S4 to S6 of FIG. In step S1, it is determined whether or not the plurality of distance values obtained for the target interpolation position are all negative values. If there is even one positive value, the process proceeds to step S2. In step S2, pixels corresponding to a distance value that is a negative value are excluded. For example, as shown in FIG. 8A, at the interpolation position (virtual pixel E) between the pixel A and the pixel B, the distance obtained from the pixel A is 0.3, and the distance obtained from the pixel B is If it is −0.1, the pixel B corresponding to the negative distance −0.1 is excluded from the following processing. For example, at the interpolation position (virtual pixel G) between the pixels A to D, the distance obtained from the pixel A is 0.3, the distance obtained from the pixel B is -0.1, and the distance obtained from the pixel C is 0. 2. Assume that the distance obtained from the pixel D is -0.7. In this case, the pixel B corresponding to the negative distance −0.1 and the pixel D corresponding to the negative distance −0.7 are excluded from the following processing.

  In step S3, the smallest depth value is obtained from the depth values at the interpolation position obtained from the pixels other than the excluded pixels, and the pixel corresponding to the minimum depth value is selected. For example, when the distance obtained from the pixel A is 0.3 and the distance obtained from the pixel B is −0.1 at the interpolation position (virtual pixel E) between the pixel A and the pixel B, the negative distance The pixel B corresponding to −0.1 is excluded, and the remaining pixel A corresponding to the minimum depth value 0.3 is selected. For example, at the interpolation position (virtual pixel G) between the pixels A to D, the distance obtained from the pixel A is 0.3, the distance obtained from the pixel B is -0.1, and the distance obtained from the pixel C is 0. 2. Assume that the distance obtained from the pixel D is -0.7. In this case, the pixel B corresponding to the negative distance −0.1 and the pixel D corresponding to the negative distance −0.7 are excluded, and the remaining pixel C corresponding to the minimum depth value 0.2 is selected. .

  In step S4, the color of the pixel selected with the minimum depth value is set as the color at the interpolation position. That is, in the example of the interpolation position (virtual pixel E) between the pixel A and the pixel B, the pixel color of the pixel A is set as the interpolation color at the interpolation position. In the example of the interpolation position (virtual pixel G) between the pixels A to D, the pixel color of the pixel C is set as the interpolation color at the interpolation position.

  If it is determined in step S1 that all the distances are negative values, the process proceeds to step S5, and the colors of the plurality of pixels are mixed according to the ratio of the plurality of distances. For example, when the distance value obtained from the pixel A is dA and the distance value obtained from the pixel B is dB, the interpolation color of the virtual pixel E between the pixel A and the pixel B is, for example, (dB × A + dA × B) / (DA + dB). When the virtual pixel G located at the center of the pixels A, B, C, and D is interpolated, the distance values obtained from the pixels A, B, C, and D are set as dA, dB, dC, and dD, respectively. For example, the interpolation color

としてよい。以上で補間色決定処理が終了する。

As good as This completes the interpolation color determination process.

  FIG. 17 is a diagram illustrating an example of a more specific circuit configuration of the anti-aliasing circuit 11. The anti-aliasing circuit 11 further includes a pixel information holding RAM 71 that is a line buffer. For example, as shown in FIGS. 7 to 11, in order to perform the anti-aliasing processing of the pixel A, not only the data shown in FIG. 6 for the pixel A but also the same data for the pixels C to D are read from the frame buffer 12. There is a need. When the anti-aliasing process is performed for each pixel, if the data of three neighboring pixels is read in addition to the data of the pixel, the memory access amount increases. Therefore, the pixel information holding RAM 71 is configured to hold pixel data of three lines (horizontal line of the image) or more.

  FIG. 18 is a diagram for explaining read / write processing of the pixel information holding RAM 71. The pixel information holding RAM 71 includes at least line memories 91 to 93. Each line memory of the line memories 91 to 93 can store data for one horizontal line of the two-dimensional image to be anti-aliased. First, as shown in the upper part of FIG. 18, two lines of an image are read into the line memories 91 and 92, and antialiasing processing is performed on each pixel of the line memory 91 using the data for the two lines. For example, in the case of the example illustrated in FIGS. 7 to 11, the pixel A and the pixel C are part of the pixel column stored in the line memory 91, and the pixel B and the pixel D are pixels stored in the line memory 92. Part of the column. By executing the above-described anti-aliasing process, the pixel value after the anti-aliasing process is obtained for each pixel of the pixel column stored in the line memory 91 including the pixel A and the pixel C. The pixel value after anti-aliasing obtained by this anti-aliasing is written from the pixel blend unit 34 to the frame buffer 12. In parallel with the execution of the anti-aliasing process for the line memory 91, the pixel data of the next horizontal line is read from the frame buffer 12 into the line memory 93.

  Next, as shown in the lower part of FIG. 18, anti-aliasing processing is performed on each pixel of the line memory 92 using data for two lines of the line memories 92 and 93. For example, in the case of the example illustrated in FIGS. 7 to 11, the pixel value after anti-aliasing processing is obtained for each pixel of the pixel column stored in the line memory 92 including the pixel B and the pixel D. The pixel value after anti-aliasing obtained by this anti-aliasing is written from the pixel blend unit 34 to the frame buffer 12. In parallel with the execution of the anti-aliasing processing for the line memory 92, the pixel data of the next horizontal line is read from the frame buffer 12 into the line memory 91. Since the pixel column data in the line memory 91 used for the previous anti-aliasing process is already unnecessary, there is no problem in overwriting the line memory 91 with the pixel data read from the frame buffer 12.

  As described above, the small-capacity and high-speed pixel information holding RAM 71 is provided in the anti-aliasing circuit 11, and the data of each pixel is read from the frame buffer 12 to the pixel information holding RAM 71 only once. Browse the data. With this configuration, pixel data can be read from the frame buffer 12 only once for each pixel, and the memory access amount can be minimized.

  Returning to FIG. 17, the depth / distance interpolation unit 31 includes addition circuits 72 and 73. The adding circuit 72 obtains the depth value of the interpolation position by adding the x-direction increment or the y-direction increment of the depth value to the depth value of the pixel of interest. The adding circuit 73 obtains the distance value of the interpolation position by adding the x-direction increment or the y-direction increment of the distance value to the distance value of the target pixel. In the description using FIGS. 7 to 11 described above, for the sake of convenience of explanation, the distance value is obtained and then the depth value is obtained. However, in practice, the distance value calculation and the depth value are obtained as shown in FIG. Calculations may be executed in parallel.

  The depth / distance comparison unit 32 includes a depth value holding buffer 74, a distance value holding buffer 75, a depth value comparison circuit 76, and a distance value comparison circuit 77. The depth value holding buffer 74 holds a plurality of depth values obtained from a plurality of pixels adjacent to the target interpolation position. The distance value holding buffer 75 holds a plurality of distance values obtained from a plurality of pixels adjacent to the target interpolation position. The depth value comparison circuit 76 compares a plurality of depth values stored in the depth value holding buffer 74 with each other, and supplies the comparison result to the pixel color determination unit 33. This comparison result may be information indicating, for example, the depth values arranged in order of the depth value from the smallest depth value among the plurality of depth values, and the arrangement of the corresponding pixels in the depth value. The distance value comparison circuit 77 compares a plurality of distance values stored in the distance value holding buffer 75 and supplies the comparison result to the pixel color determination unit 33. This comparison result is information for identifying a pixel having a negative distance value, information for identifying a pixel having a positive distance value, and information indicating a ratio (weight) of distance values having a positive value. It's okay.

  The pixel color determination unit 33 includes a pixel selection / blending unit 78 and a pixel color holding buffer 79. The pixel selection / blending unit 78 is based on the depth value comparison result from the depth value comparison circuit 76, the distance value comparison result from the distance value comparison circuit 77, and the colors of the plurality of pixels read from the pixel information holding RAM 71. The color at the target interpolation position is determined. When the distance value comparison result indicates that the plurality of distances are all negative values, the colors of the plurality of pixels are synthesized according to the ratio of the distance values indicated by the distance value comparison result. When the distance value comparison result indicates that a plurality of distances have a positive value, the pixel having the smallest depth value indicated by the depth value comparison result among the positive distance value pixels specified by the distance value comparison result And the color of the specified pixel is set as the color at the target interpolation position. The interpolation color at the target interpolation position obtained by the pixel selection / blending unit 78 is stored in the pixel color holding buffer 79.

  The depth / distance interpolation unit 31, the depth / distance comparison unit 32, and the pixel color determination unit 33 have a plurality of interpolation positions (for example, the virtual pixel E, FIG. 11) adjacent to one pixel of interest (for example, the pixel A in FIG. 11). For F, G), the same processing as described above is repeated to obtain interpolation colors at a plurality of interpolation positions. The plurality of interpolation colors obtained for the plurality of interpolation positions are stored in the pixel color holding buffer 79. The pixel color of the pixel of interest is also stored in the pixel color holding buffer 79. The pixel blend unit 34 obtains a pixel color after antialiasing processing of the target pixel by mixing the interpolation colors of the plurality of interpolation positions stored in the pixel color holding buffer 79 and the pixel color of the target pixel, and the frame buffer 12 To store. The pixel color after the anti-aliasing process stored in the frame buffer 12 may be written so as to overwrite the pixel color data of the existing target pixel. Alternatively, it may be written as other image data.

  As mentioned above, although this invention was demonstrated based on the Example, this invention is not limited to the said Example, A various deformation | transformation is possible within the range as described in a claim.

The present invention includes the following contents.
(Appendix 1)
A plurality of processes adjacent to the interpolation position that calculate the depth at the interpolation position and the distance to the edge of the graphic based on the depth of a pixel adjacent to the interpolation position and the distance to the edge of the graphic including the pixel. An interpolation unit that calculates a plurality of depths and a plurality of distances at the interpolation position by executing each of the pixels;
A pixel color determination unit that determines colors at the interpolation position based on the colors of the plurality of pixels and the plurality of depths and the plurality of distances at the interpolation position obtained by the interpolation unit. An image processing apparatus.
(Appendix 2)
A pixel for obtaining a new color of the one pixel of interest by mixing the color determined by the pixel color determination unit and the color of the pixel of interest for a plurality of interpolation positions adjacent to the position of one pixel of interest The image processing apparatus according to appendix 1, further comprising a color mixing unit.
(Appendix 3)
The pixel color determination unit combines the colors of the plurality of pixels according to a ratio of the plurality of distances when the plurality of distances are all negative values, and the plurality of distances are all negative values. Otherwise, the smallest one of the plurality of depths corresponding to a non-negative value among the plurality of distances is obtained, and the pixel corresponding to the smallest depth among the plurality of pixels. The image processing apparatus according to appendix 1 or 2, wherein the color is the color at the interpolation position.
(Appendix 4)
The comparison unit further includes a comparison unit that compares the plurality of depths and the plurality of distances at the interpolation position obtained by the interpolation unit between the plurality of pixels, and the pixel color determination unit includes a plurality of depths by the comparison unit. The image processing apparatus according to appendix 1, wherein a color at the interpolation position is determined based on a comparison result, a comparison result of a plurality of distances, and colors of the plurality of pixels.
(Appendix 5)
A plurality of processes adjacent to the interpolation position that calculate the depth at the interpolation position and the distance to the edge of the graphic based on the depth of a pixel adjacent to the interpolation position and the distance to the edge of the graphic including the pixel. To obtain a plurality of depths and a plurality of distances at the interpolation position,
Determining an interpolation color at the interpolation position based on the plurality of depths, the plurality of distances, and the colors of the plurality of pixels;
Including each step of obtaining a new color of the one pixel of interest by mixing the interpolation color determined for a plurality of interpolation positions adjacent to the position of one pixel of interest and the color of the one pixel of interest An image processing method characterized by the above.
(Appendix 6)
The step of determining the interpolation color includes combining the colors of the plurality of pixels according to a ratio of the plurality of distances when the plurality of distances are all negative values, and the plurality of distances are all negative values. Otherwise, the smallest depth among the plurality of distances corresponding to a non-negative value among the plurality of distances is obtained, and the smallest depth among the plurality of pixels is handled. 6. The image processing method according to appendix 5, wherein a color of a pixel to be processed is a color at the interpolation position.
(Appendix 7)
Memory,
An image drawing unit that calculates a pixel color, a depth, and a distance to an edge of a graphic including the pixel for each pixel, and stores the calculated image in the memory;
A process of reading the depth and the distance from the memory and obtaining a depth and a distance at the interpolation position based on the depth and the distance of a pixel adjacent to the interpolation position. An interpolation unit that obtains a plurality of depths and a plurality of distances at the interpolation position,
Pixel color determination for determining the color at the interpolation position based on the pixel color read from the memory for the plurality of pixels and the plurality of depths and the plurality of distances at the interpolation position obtained by the interpolation unit And
A pixel for obtaining a new color of the one pixel of interest by mixing the color determined by the pixel color determination unit and the color of the pixel of interest for a plurality of interpolation positions adjacent to the position of one pixel of interest A color mixing section;
An image system comprising: an image display unit that displays an image composed of pixels having a pixel color obtained by the pixel color mixing unit.

DESCRIPTION OF SYMBOLS 10 Graphics processing circuit 11 Anti-aliasing circuit 12 Frame buffer 13 Display apparatus 21 Vertex buffer 22 Coordinate conversion / lighting processing unit 23 Vertex post-processing unit 24 Rasterization processing unit 25 Pixel processing unit 26 Pixel post-processing unit 27 Memory access unit 28 Texture unit 29 Distance Information calculation unit 31 Depth / distance interpolation unit 32 Depth / distance comparison unit 33 Pixel color determination unit 34 Pixel blending unit

Claims (5)

A plurality of processes adjacent to the interpolation position that calculate the depth at the interpolation position and the distance to the edge of the graphic based on the depth of a pixel adjacent to the interpolation position and the distance to the edge of the graphic including the pixel. An interpolation unit that calculates a plurality of depths and a plurality of distances at the interpolation position by executing each of the pixels;
A pixel color determination unit that determines colors at the interpolation position based on the colors of the plurality of pixels and the plurality of depths and the plurality of distances at the interpolation position obtained by the interpolation unit. An image processing apparatus.   A pixel for obtaining a new color of the one pixel of interest by mixing the color determined by the pixel color determination unit and the color of the pixel of interest for a plurality of interpolation positions adjacent to the position of one pixel of interest The image processing apparatus according to claim 1, further comprising a color mixing unit.   The pixel color determining unit combines the colors of the plurality of pixels according to a ratio of the plurality of distances when the plurality of distances are all negative values, and the plurality of distances are all negative values. Otherwise, the smallest one of the plurality of depths corresponding to a non-negative value among the plurality of distances is obtained, and the pixel corresponding to the smallest depth among the plurality of pixels. The image processing apparatus according to claim 1, wherein the color is a color at the interpolation position.   The comparison unit further includes a comparison unit that compares the plurality of depths and the plurality of distances at the interpolation position obtained by the interpolation unit between the plurality of pixels, and the pixel color determination unit includes a plurality of depths by the comparison unit. The image processing apparatus according to claim 1, wherein a color at the interpolation position is determined based on a comparison result, a comparison result of a plurality of distances, and colors of the plurality of pixels. A plurality of processes adjacent to the interpolation position that calculate the depth at the interpolation position and the distance to the edge of the graphic based on the depth of a pixel adjacent to the interpolation position and the distance to the edge of the graphic including the pixel. To obtain a plurality of depths and a plurality of distances at the interpolation position,
Determining an interpolation color at the interpolation position based on the plurality of depths, the plurality of distances, and the colors of the plurality of pixels;
Including each step of obtaining a new color of the one pixel of interest by mixing the interpolation color determined for a plurality of interpolation positions adjacent to the position of one pixel of interest and the color of the one pixel of interest An image processing method characterized by the above.

Images