CN109155074B

CN109155074B - System and method for seamlessly rendering points

Info

Publication number: CN109155074B
Application number: CN201780032052.9A
Authority: CN
Inventors: 格拉哈姆·法伊夫
Original assignee: Sony Corp; Sony Pictures Entertainment Inc
Current assignee: Sony Corp; Sony Pictures Entertainment Inc
Priority date: 2016-03-28
Filing date: 2017-03-28
Publication date: 2023-09-08
Anticipated expiration: 2037-03-28
Also published as: US10062191B2; EP3437072B1; WO2017172842A1; EP3437072A4; EP3437072A1; CN109155074A; US20170278285A1

Abstract

Various systems and methods disclosed herein relate to rendering a point-based graphic on a computing device, wherein a space between points is filled with color, thereby generating a seamless or pore-free surface. According to one method, one or more rasterizing channels are performed on an image space. One or more fill channels are performed on pixels in an image space, wherein spaces between pixels in the image space are filled with colors, thereby forming a continuous surface in a new image plane. After filling the channels, one or more mixing channels are performed on the image space, wherein the colors of a set of pixels are mixed together. The new image space is rendered from the image space in the image buffer.

Description

System and method for seamlessly rendering points

Copyright description

A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the patent and trademark office patent files or records, but otherwise reserves all copyright rights whatsoever.

Cross reference

The present application claims the benefit of U.S. provisional patent application US62/314,334 filed 28 at 3/2016, the entire contents of which are incorporated herein by reference.

Technical Field

The present application relates generally to computer graphics, and more particularly to a system and method for seamlessly rendering points.

Background

In computer graphics, point-based graphics have been used for model acquisition, processing, animation, and rendering. One particular challenge in point-based rendering is how to build a continuous surface without holes as the point of view changes. Various point-based rendering techniques have been developed to address this problem. Although these techniques can be used to fill holes in an image, the resulting solution is always lacking in practice or results in poor image quality.

Disclosure of Invention

Various systems and methods disclosed herein are directed to rendering a dot-based graphic on a computing device, wherein spaces between dots are filled with colors, thereby creating a seamless or pore-free surface appearance. According to one method, one or more rasterizing channels are performed on an image space. Based on the first camera image plane, color attributes and color effects of pixels in the image space are stored in an image buffer. One or more fill channels are performed for the pixels in the image space, wherein the space between the pixels in the image space is filled with colors, thereby forming a continuous surface in the new image plane. After filling the channels, one or more mixing channels are performed on the image space, wherein the colors of a set of pixels are mixed together. Rendering a new image space from the image space in the image buffer, wherein the new image plane is based on a second camera image plane, wherein the first and second camera image planes are different.

Drawings

FIG. 1 is a flow chart of an exemplary method for rendering points.

FIG. 2 is a schematic diagram of one embodiment of an image buffer.

Detailed Description

Briefly summarized, the various systems and methods disclosed herein are directed to rendering a point-based graphic on a computing device, wherein spaces between points are filled with colors, thereby creating a seamless or pore-free surface appearance. The system and method disclosed by the application improves the rendering performance of a large number of points through gap filling, and simultaneously provides consistent performance for rendering of various points with different sizes.

As shown in fig. 1, an exemplary method for rendering a point-based graphic includes the steps of: executing one or more rasterization channels (110); executing one or more filling channels (120); and executing one or more mixing channels (130).

In one embodiment, the system includes a Graphics Processing Unit (GPU) running software code. The software can store in the image buffer, during one or more of the rasterization channels 110, point attributes at locations determined by projecting 3D point locations in the image plane coordinates of the camera model, including point colors and point geometry parameters. The GPU vertex shader and GPU fragment shader may be used to project and store point colors and geometric parameters. The geometric parameters may include the 2D center of the point in the image plane coordinates, the image space radius in the case of a spherical point primitive, and other parameters used to define an ellipse in the case of an ellipsoidal point primitive.

In an alternative approach, the rasterizing step may store UV texture coordinates without storing color, or additionally store color as well. The geometric parameters stored in the image buffer may comprise the original 3D parameters of the 3D point primitive or parameters derived based on the projection of the point primitive on the camera image plane. In a variant embodiment, the point attributes including color and geometry parameters may be stored as a linear array, and the indices of the array may be stored in the image buffer instead of the point attributes themselves. In another variant embodiment, in the rasterizing step, 48-bit half-floating-point red, green, blue values are stored in an image buffer.

After rasterizing the channels, the GPU executes software for executing one or more fill channels 120 in image space. In one embodiment, a GPU fragment shader is used to fill channels. Each fill channel accesses each pixel site 210 in the image buffer 220 shown in fig. 2, calculates updated geometric parameters at each pixel site, and stores the results in the second image buffer. The first and second image buffers may exchange roles after each of the fill channels. Updating the geometric parameters at a pixel location is accomplished by accessing the geometric parameters of the pixel location and its 8 adjacent pixel locations on grid 230, which may not contain a geometric quantity (i.e., zero radius), or may contain the same geometric parameters as the central pixel location or different geometric parameters. The grid pitch may be one grid point per pixel, one grid point per two pixels, or one grid point per N pixels, N being some integer. The grid spacing N in the first fill channel is to the power of 2, which is greater than the image space radius of the largest point primitive that is desired to be rendered, and the grid spacing is halved after each fill channel, such that the geometric parameter propagates a longer distance in the image buffer in the first fill channel, and progressively shorter distances in subsequent fill channels, until only a single pixel distance propagates in the last fill channel. The geometric parameters on the central pixel site and its 8 neighboring pixel sites (as shown in fig. 2) are tested for ray intersection using rays passing through the central pixel site based on the camera model. The geometry of the pixel location after updating is the geometry of the intersection closest to the position of the camera model (if any) or the point with the smallest depth value. The intersection can be calculated approximately; for example, a spherical dot element can be considered as a cone, a disk, or a truncated paraboloid in close proximity to a sphere. As a result of the completion of filling the channel, each pixel location in the image buffer approximately contains the geometric parameters of the dot primitive that are visible at the pixel location closest to the camera model or having the smallest depth value.

In an alternative approach, the fill channel may propagate an index of a linear array of point properties, rather than the propagation properties themselves. In another variant embodiment, the fill channel may not propagate any point attributes, or propagate some or all of the point attributes, except for propagating image buffer coordinates corresponding to pixel locations containing the remaining non-propagated point attributes. In another variant embodiment, the image buffer coordinates are the same or identical to the point center attribute.

In the fill channel, different adjacent regions other than the conventional grid containing the center pixel site and its 8 adjacent pixel sites may be accessed, and different grid spacing schemes other than the power of 2 may be employed. In a variant embodiment, a larger grid may be employed consisting of a central pixel site of 5*5 grid and 24 adjacent pixel sites thereof, along with a grid spacing scheme of the power of 4. In another variant embodiment, the fill channels may be switched between the horizontal grid fill channels of 1*3 and the vertical grid fill channels of 3*1, and employ a grid spacing that is halved only after every two channels to the power of 2. Similar variant embodiments with even larger grids may also be employed.

After the last fill channel, the GPU executes the mix channel 130. In one embodiment, a GPU fragment shader is used for the mixing channel. The blend channel accesses each pixel site 210 in the image buffer 220. The geometric parameters stored at the pixel site during the last fill channel 120 may be accessed and the point center (if any) may be used to access the point color information or other point attributes stored at the second pixel site in the image buffer that includes the point center. In this way, the dot color and other dot properties do not need to be propagated through multiple filling channels, thereby improving the efficiency of the method. The dot color and dot properties of 8 immediately adjacent pixel locations of the pixel location can be accessed in a similar manner. The final rendered color may be calculated as a weighted mix of up to 9 accessed colors associated with up to 9 dot centers. The weight of a color in a blend may be inversely proportional to a monotonic function of the distance from the center of the relevant point to the center of the pixel location, such as the square of the distance plus a small constant.

In an alternative approach, the mixing channel may smoothly interpolate color or UV texture coordinates between the dot centers. Based on the mixed UV coordinates, the color may be accessed from one or more texture maps. In a variant embodiment, a color effect is applied to the color during the mixing channel instead of the rasterizing channel.

In another approach, all channels employ GPUs that use vertex shaders and fragment shaders. In rasterizing the channel, colors are input as 48-bit half-floating point red, green, blue values, which are then processed for color effects such as gamma, brightness, contrast, or color look-up tables. The color data is then stored in the image buffer as 24-bit red, green, blue values. The fill channels use a grid spacing of 2 to the power of 2, starting with the user specified maximum power of 2, and halving after each channel until the last channel uses a grid spacing of 1. In the fill channel, the propagated geometric parameter is simply the 2D point center corresponding to the image buffer coordinates. Other geometric parameters are accessed from pixel locations corresponding to the dot centers, if desired. In a mixed channel, the monotonic weight function is the square of the distance from the associated point center to the pixel site center plus a small constant.

These various methods and embodiments may be implemented on an exemplary system including at least one host processor coupled to a communication bus. The system includes a main memory. The main memory may store software (control logic) and data. In one embodiment, the main memory may be in the form of Random Access Memory (RAM).

The system may also include a graphics processor and a display (e.g., a computer monitor). In an embodiment, the GPU includes one or more shader modules, rasterizer modules, or other modules known or developed in the art. Each of these modules may be disposed on a separate semiconductor platform, forming a Graphics Processing Unit (GPU).

It will be appreciated by those skilled in the art that an individual semiconductor platform may refer to an integrated circuit or chip based on a single semiconductor. It should be noted that the term "stand-alone semiconductor platform" may also refer to a multi-chip module with enhanced connectivity that mimics on-chip operation and greatly improves over the use of conventional central processor and bus implementations. Alternatively, the various modules may also be arranged individually or in different semiconductor platform combinations. The system may also be implemented by reconfigurable logic, which may include, but is not limited to, field Programmable Gate Arrays (FPGAs).

The system may also include a secondary memory including, for example, a hard disk drive and/or a removable storage drive representing a floppy disk drive, a magnetic tape drive, or an optical disk drive. The removable storage drive reads from and/or writes to a removable storage unit in a manner known in the art.

The computer program or computer control logic algorithm may be stored in the main memory and/or the secondary memory. The computer programs, when executed, cause the system to perform various functions. Main memory, secondary memory, volatile or non-volatile memory, and/or any other type of memory are all available examples of non-transitory computer-readable media.

In an embodiment, the architecture and/or functionality of the various systems disclosed may be implemented in the context of a host processor, a graphics processor, or an integrated circuit having at least some of the capabilities of both a host processor and an graphics processor, or a chipset (i.e., a set of integrated circuits designed to operate and sell one unit for performing related functions), and/or any other integrated circuit for that purpose.

In another embodiment, the architecture and/or functionality of the various systems disclosed may be implemented in the context of a general purpose computer system, a circuit board system, a gaming machine system dedicated for entertainment purposes, a special purpose system, and/or other desired systems. For example, the system may be in the form of a desktop computer, a portable computer, and/or other types of logic. Also, the system may be in the form of various other devices including, but not limited to, a personal digital assistant (PAD), a cell phone, or other similar devices.

In the various embodiments disclosed, the system may be connected to a network (e.g., a communication network, a Local Area Network (LAN), a wireless network, a Wide Area Network (WAN) such as the internet, a peer-to-peer network, or a wired network) for communication purposes.

The various embodiments described above are for illustrative purposes only and should not be construed as limiting the application. Various modifications and alterations of this application will become apparent to those skilled in the art without departing from the true spirit and scope of this application, and the application is not to be limited to the embodiments and applications illustrated and described herein.

Claims

1. A method for rendering point-based graphics on a computer, comprising:

storing, in an image buffer, point attributes of an image space at an orientation determined by projecting three-dimensional point locations in image plane coordinates of a camera image plane during one or more rasterization channels;

executing one or more filling channels on the point attribute of the image space, wherein each filling channel accesses each pixel part in the image buffer, calculates updated geometric parameters on each pixel part, and stores the updated geometric parameters in a second image buffer;

after performing the fill channel for each pixel site in the buffer, performing a blend channel, wherein updated geometric parameters from each pixel of the second image buffer are applied to each pixel; and

after the filling channel and mixing channel, a new image space is rendered based on the image buffer.

2. The method of claim 1, wherein the point attribute is a point color.

3. The method of claim 1, wherein the point attribute is UV texture coordinates.

4. The method of claim 1, wherein the point attribute is a geometric parameter of a point.

5. The method of claim 1, wherein the geometric parameter is a 2D center of a point in image plane coordinates, an image space radius for a spherical point primitive, or a geometric parameter for defining an ellipse for an ellipsoidal point primitive.

6. The method of claim 1, wherein calculating the updated geometric parameters comprises: the geometric parameters of 8 immediately adjacent pixels of each pixel site are accessed and a determination is made as to whether a ray intersection exists between each pixel site and any of the 8 immediately adjacent pixels.

7. The method of claim 6, wherein the mixing channel further comprises mixing colors from each pixel site and the 8 immediately adjacent pixels.

8. The method of claim 6, further comprising applying a color effect to pixels in the image space stored in the image buffer.