JP4808167B2 - Image processing method and 3D drawing circuit using the processing method - Google Patents

Description

  The present invention relates to an image processing method for predicting an outside of an edge of image data from an inside image and expanding and displaying the image data, and a 3D drawing circuit using the image processing method.

Conventionally, when there is a missing area in a part of the image displayed on the display, an image processing method to restore the image of the missing area (interpolation processing of the internal image with the previous and subsequent images) However, when an image is expanded outside the image area, there is no image processing method for estimating and expanding the outside of the image area.
When we wanted to display outside the range of the image, it was realized by simple coordinate replacement.
FIG. 8 shows an example of an image processed by a conventional image processing method.
For example, it is assumed that pixel information of a picture in a range as shown in L ₁ × L ₂ as shown in FIG. When the image is extended to the left outer side of the range of L ₁ × L ₂ , if the method is called wrapping, the right end image of the original image is connected to the left outer side of the range of L ₁ × L ₂ . For this reason, the continuity of the extended portion of the image is impaired, and a smooth extended image cannot be obtained (b).

Further, when a technique called clamping is employed, when the image is extended to the left outside of the range of L ₁ × L ₂ , the image processing is such that the density of the image at the left edge of the image is extended to the left side as shown in (c). It becomes an image with such an extended part and is unnatural. Furthermore, if a technique called a mirror is used, in the original image with a slanted pattern, the image viewed from the back side is connected to the left outer side of the range of L ₁ × L ₂ , and similarly an unnatural image is obtained (d). .
Such processing has a drawback that the edge portion is conspicuous even in the case of a visually very small image portion.

A video data processing apparatus (Patent Document 1) has been proposed as a process when white images are generated between images when images are rearranged.
This is a proposal of a process for filling a white image portion generated between each screen when the image data is displayed on ¼ of the display screen when four multi-screen images are displayed. A process of repeatedly adding the same pixel to the white image portion starting from the pixel of the image in contact with the image is performed. For this reason, a rectangular pattern extends in the direction of the white image from the wheel of the bullet train in contact with the edge of the screen. This is the same as the processing of FIG. 8C shown in the conventional example, and the smoothness of the image is impaired when the area of the white image portion is large.
JP 2005-142678 A

It is an object of the present invention to have original image data for displaying a predetermined range on a screen, and predict an area outside the image range by predicting an area outside the image range with respect to an area outside the original image data (image extension portion). An object of the present invention is to provide an image processing method capable of displaying an image that continues smoothly up to an extended portion.
Another object of the present invention is to provide a 3D drawing circuit which can display an image near the boundary of the polygon texture image pasting using this image processing method more smoothly than in the prior art.

In order to achieve the object, claim 1 of the present invention has original image data for displaying a predetermined range, and replenishes a pixel outside the predetermined range based on the original image data. An image processing method for detecting an edge of a picture formed by the original image data by an edge detection means and extending the detected picture edge to an area outside the predetermined range by an edge extension means. The pixel supplement surface calculation means calculates a surface that is outside the predetermined range and is surrounded by the extended picture edge, and the pixel information calculation means is an original image that is the same surface as the surrounded surface. The pixel information to be supplemented is obtained from the pixel information of the data, and the region outside the predetermined range is drawn with the pixel information to be supplemented.
Claim 2 of the present invention is a 3D drawing circuit for replenishing pixels when an area in which a texture image is missing is generated near a boundary separating the polygon surface when the texture image is pasted on the polygon surface. A texture image contour detection unit for detecting a contour of a texture image, and a split surface for calculating a split surface formed by extending the edges of the texture image detected by the texture image contour detection unit to the missing region. By referring to the pixel information of the texture image that is the same plane as the division plane, the division plane pixel information calculation section that calculates pixel information to be supplemented to the division plane, and the pixel information of the division plane And an update unit that updates pixel information in addition to the original image data.
According to a third aspect of the present invention, in the second aspect, the pixel information includes plane coordinate data and color data.

According to the above configuration, a smooth image can be displayed on the display screen outside the image data. In particular, when applied to texture edges applied to the surface of a 3D model, a 3D image can be obtained more smoothly than in a conventional image processing method.
Further, when the entire image is restored from a photograph in which only a part of an object is shown, the work process can be reduced and the manual work can be simplified by using the method of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1A is a block diagram showing an embodiment of a drawing processing circuit configured by an image processing method according to the present invention.
Original image data 2 a is stored in the memory unit 2. The drawing processing circuit 3 uses the original image data 2a to form image data that is extended to the outside of the original image data 2a.
The drawing processing circuit 3 includes an edge detection circuit 3a, an edge extension circuit 3b, a pixel supplement plane calculation circuit 3c, and a pixel information calculation circuit 3d. An image of an extended portion outside the original image data 2a from the pixel information calculation circuit 3d. Data is output, and the image data of the expanded portion is also displayed on a display unit (not shown).

FIG. 1B is a diagram for explaining the steps of the image processing method according to the present invention.
The original image data 2a is composed of a gradation band image 5 that gradually becomes thinner toward the left outer boundary line and a band image 6 that is inclined downward with respect to the left outer boundary line ( a). When generating an extended image in a predetermined range on the left outer side of the original image, the original image edge detection circuit 3a for detecting the edge of the original image performs processing for detecting only the edge portion, and the band shown in FIG. Shape lines 5a and 6a are detected.
Next, the edge extension circuit 3b extends the band shape lines 5a and 6a to the left outer side to form the extension lines 5b and 6b in the extended portion (c).
Further, the surface surrounded by the extended picture edge of the area outside the original image data 2a is calculated by the image supplemental surface calculation circuit 3c, and the original image data that becomes the same surface as the surrounded surface is calculated by the image information calculation circuit 3d. The pixel information to be supplemented is calculated based on the pixel information of the original image data. The pixel information includes plane coordinate data and color data, so that a smooth image connected to the original image data 2a can be drawn on the extended portion.

FIG. 2 is a diagram showing an example of an image processed by the image processing method according to the present invention.
The original image 1 has a large number of thick bands and a large number of thin bands arranged obliquely at a predetermined interval as shown in FIG. The line 1a extending the edge of the line is formed as shown in (b), and by referring to the image data on the same surface as the part surrounded by the line, a pixel rendered as shown in (c) can be obtained. it can.

FIG. 3 is a block diagram showing an embodiment of a circuit of a 3D game device to which the image processing method according to the present invention is applied.
The 3D game device 16 has an input operation device 10 for inputting a player operation with a pad, a keyboard, a mouse, etc. and sending it to the CPU 12, a sound circuit 15 for outputting BGM of the game, a speaker 11 for outputting sound, and a display based on pixel information. 14 stores a graphic circuit 13 for displaying an image, a game control program, a HDD 18 for storing a 3D game program and necessary data, and a 3D game program and image data read from the HDD 18 when the game is started. A RAM 17 and a CPU 12 that are also used as areas are provided.

The CPU 12 has functions of a geometry control unit 12b and a rendering unit 12c for a 3D game, in addition to a game control unit 12a for executing a game using a program and data read from the HDD 18 and temporarily stored in the RAM 17.
The geometry processing unit 12b is a control unit that processes polygon three-dimensional coordinate data at high speed, and performs processing on movement, rotation, and the like of the polygon coordinate data sent from the game control unit 12a to perform clipping, field of view conversion, and the like. Perform the process. The rendering unit 12c is a control unit that processes polygon rendering at high speed, and performs rendering processing on polygon data sent from the game control unit 12a and the geometry processing unit 12b, and performs hidden surface processing, shading processing, and the like. The drawing processing unit 19 according to the present invention is a part of a rendering process, and includes functions of a texture image contour detection unit 19a, a divided surface calculation unit 19b, a divided screen pixel information calculation unit 19c, and a pixel information update unit 19d.

  When the rendering unit 12c performs vertex processing on polygon data sent from the game control unit 12a and the geometry processing unit 12b, and refers to the outside of the range of the texture image in the process of pasting the texture image on the created polygon surface That is, when the Boligon surface and the texture image do not match and it is determined that the image should be replenished outside the range of the texture image, the texture image contour detection unit 19a determines the boundary between the shades of the texture image. Thus, the contour of the image is detected. The dividing surface calculation unit 19b extends the contour line of the image to a missing region outside the texture image, and calculates a dividing surface formed between the extended edge and the boundary of the adjacent polygon and the outside line of the texture image. The division plane pixel information calculation unit 19c calculates pixel information smoothly connected to the division plane by referring to the pixel information of the texture image that is the same plane as the division plane. The pixel information update unit 19d updates pixel information to be pasted by adding pixel information for the divided surface to the original texture image data. The pixel information update unit 19d also performs update of pixel information for background processing and processing by transparency.

FIG. 4 is a flowchart showing the flow of the image processing method according to the present invention.
The drawing process is a series of graphic processes (S001). For example, information such as vertices and textures related to a picture is read from the RAM 17, and the CPU 12 prepares for graphic processing (S002).
Next, vertex processing is performed to determine which polygon is to be drawn where, the drawing place is determined, and pixel processing is started (S003, S004). That is, the process proceeds to a process of drawing a screen.
The coordinates of the texture image to be referred to are associated with each pixel, and the area of the texture image to be pasted is searched for each pixel. In this process, when the texture image is larger than the pixel portion to which the image is to be pasted, the pixel portion to be pasted is completely within the texture image range, so the outside of the texture image range is not referred to. On the other hand, when the texture is small relative to the pixel portion to be pasted, the outside of the texture, that is, the peripheral portion is also referred to. If it is a reference within the range of the texture image, it is only necessary to bring the existing color data of the texture, so that the process proceeds to the conventional pixel information update (S005, S010).
On the other hand, in the case of reference outside the range of the texture image, that is, when it must be pasted to the pixel portion outside the texture, processing for detecting the contour of the texture image is performed (S005, S006).

This is to detect where and what to find the approximate outline of the picture.
The surface is divided by the extended contour by extending the obtained contour (S007).
It is examined to which part of each polygon surface the divided contour corresponds. Then, an operation for obtaining a divided surface formed by the divided contour, the boundary line with the adjacent polygon, and the boundary line with the texture screen is performed (S008).
Next, the process proceeds to a process of assigning pixel data to the obtained divided surface by referring to the internal data (for example, black dots, white dots, etc.) of the texture image of the same surface with respect to the obtained divided surface. For example, if the internal data of the texture image on the same surface as the divided surface is gradation data whose blue color gradually becomes lighter, it should be pasted from the connecting portion of the divided surface connected to the internal data of the texture image toward the extended portion of the divided surface Blue gradation data is calculated (S009). At this time, the blue color affixed to the connection portion of the dividing surface starts from the same density as the outermost blue density of the internal data of the texture image, and becomes gradation data that becomes gradually lighter.

In this way, pixel information such as color and pattern is allocated to the divided surface by calculating the pixel information for the divided surface. At this time, the background and transparency are also processed, and the pixel information on the polygon surface is updated including the calculation results of these processes (S010).
Next, the game control unit 12a determines whether or not all such pixel processing has been completed for each polygon, and if not, returns to S005 and continues the processing.
When the pixel process is completed, the process proceeds to a drawing process for displaying an image on the display 2 (S012).

FIG. 5 is a diagram for explaining a specific image example in the process of 3D rendering processing according to the present invention.
In (a), the polygon B adjacent to the polygon A is formed by the vertex processing, the texture image A is pasted on the polygon A, the texture image B is pasted on the polygon B, and the texture image B pasted on the polygon B is the polygon B. The polygon A shows a state where a texture image defect 8a is generated. The texture image A has a picture of a blue band 9 that gradually becomes thinner toward the boundary line that contacts the adjacent polygon B, and the texture image B becomes thinner gradually from the boundary line that contacts the adjacent polygon A. Have an image of the blue band 10.

  In such a case, the edge portion of the band 9 is extended toward the outside of the texture image, that is, toward the polygon B, and the extension line, the boundary line between the polygons A and B, and the outline line of the texture image A The division surface 9a to be created is obtained by calculation. Pixels to be pasted on the dividing plane 9a are obtained by referring to pixel information of the same area 9c (part surrounded by a two-dot chain line) as the dividing plane inside the outline of the texture image A. Since the pixels are arranged in the region 9c so that the blue band gradually becomes thinner toward the outside, from this information, the blue band gradually increases toward the outside in the same manner as this. An operation is performed in which the band supplement pixel 9b is arranged so that the pixel of the portion in contact with the region 9c becomes the same as the shade of the color of the texture image of the portion so as to be thin. By processing in this way, a band connected to the band 9 with the same color and gradation is formed.

FIG. 6 is a flowchart for explaining the flow of another 3D image processing.
In the flowchart of FIG. 4, the processing steps of detecting the contour of the texture image (S006) and dividing the surface by the edge (S007) are performed after referring to the outside of the texture range. The procedure of the image processing method is different in that the processing steps of detecting the contour (S103) and dividing the surface by the edge (S104) are performed before the vertex processing (S105). The contents of the other process steps are the same as the contents of the process steps corresponding to each of the flowcharts of FIG.

FIG. 7 is a flowchart for explaining the flow of still another 3D image processing.
This embodiment is different from FIG. 4 in that a complicated process of branching is not performed in pixel processing.
The drawing process is a series of graphic processes (S201). Information such as vertices and textures related to a certain picture is read from the RAM 17 and the CPU 12 prepares for graphic processing (S202).
Next, the contour detection processing of the texture image is performed (S203), and the surface is divided by this contour (S204).

Thereafter, the outer round of the texture image is first obtained by pre-calculation. This does not process as shown in Fig. 4 in which the texture image is searched every time the range of the texture image is referenced, but all the calculation of the edge extension and pixel interpolation processing is performed for all four sides of the texture image. I'll do it. As a result, the branch processing in the case of pixel processing shown in FIG. 4 is not performed. By calculating in this way, the data that is likely to be referred to in the pixel processing is calculated in advance and has information, so this information is used.
Subsequent to S204, pre-calculation of pixels outside the texture image range is started (S205), and a divided plane including the pixel being calculated is calculated (S206). Then, pixel information of the pixel being calculated is calculated from the pixel information inside the texture (S207).
In this way, it is determined whether or not the pre-calculation of all pixels outside the texture image range has been completed (208).

If the pre-calculation of all pixels outside the texture image range has not been completed, the process returns to S206, and the processes of S206 and S207 are continued. When the pre-calculation is completed, the vertex processing is performed to determine the place to draw and the pixel processing is started (S209, S210). That is, the process proceeds to a process of drawing a screen.
In the pixel processing, the pixel information on the polygon surface is updated with the pixel information calculated in advance. Next, it is determined whether or not the update process has been performed for all pixels (S212). If the update process has not ended, the process returns to S211 and the pixel information update process is continued. When the update process is completed, the process proceeds to a drawing process for displaying an image on the display 2 (S213).

  It is used for an image processing device, a game device for playing a 3D game, and the like.

It is a block diagram which shows embodiment of the drawing processing circuit comprised by the image processing method by this invention. It is a figure for demonstrating the process of the image processing method by this invention. It is a figure which shows the example of the image processed with the image processing method by this invention. It is a block diagram which shows embodiment of the circuit of the 3D game device to which the image processing method by this invention is applied. 4 is a flowchart for explaining a flow of 3D image processing in FIG. 3. It is a figure for demonstrating the example of a specific image about the process of 3D drawing process by this invention. It is a flowchart for demonstrating the flow of another 3D image process. It is a flowchart for demonstrating the flow of another 3D image process. It is a figure which shows the example of an image processed with the conventional image processing method.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Original image 2 Memory part 2a Original image data 3 Drawing processing circuit 5, 6 band image 10 Input operation apparatus
12 CPU
13 Graphic circuit 14 Display
15 sound circuit 16 game device 17 RAM
18 HDD

Claims (3)

An image processing method for replenishing pixels based on the original image data for an area outside the predetermined range, having original image data for displaying a predetermined range,
Detecting an edge of a picture formed by the original image data by an edge detection means;
The edge of the detected picture is extended to an area outside the predetermined range by an edge extension means,
Calculating a surface that is outside the predetermined range and surrounded by the extended edge of the picture by means of a pixel supplement surface calculation means;
Obtain pixel information to be replenished from the pixel information of the original image data that is the same surface with respect to the surface surrounded by the pixel information calculation means,
An image processing method, wherein a region outside the predetermined range is drawn with pixel information to be supplemented. A 3D drawing circuit for replenishing pixels when an area in which a texture image is lost occurs near a boundary that divides the polygon surface when the texture image is pasted on the polygon surface;
A texture image contour detector for detecting the contour of the texture image;
A split surface calculation unit that extends the edge of the texture image detected by the texture image contour detection unit to the area to be lost and calculates a split surface formed by the extended edge;
A division plane pixel information calculation unit that calculates pixel information to be supplemented to the division plane by referring to pixel information of a texture image that is the same plane as the division plane;
An update unit that updates pixel information by adding pixel information of the dividing plane to the original image data;
A 3D drawing circuit comprising:   3. The 3D drawing circuit according to claim 2, wherein the pixel information includes plane coordinate data and color data.

Images