US20070165035A1 - Deferred shading graphics pipeline processor having advanced features - Google Patents

Deferred shading graphics pipeline processor having advanced features Download PDF

Info

Publication number
US20070165035A1
US20070165035A1 US11/613,093 US61309306A US2007165035A1 US 20070165035 A1 US20070165035 A1 US 20070165035A1 US 61309306 A US61309306 A US 61309306A US 2007165035 A1 US2007165035 A1 US 2007165035A1
Authority
US
United States
Prior art keywords
data
primitive
tile
memory
texture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US11/613,093
Other versions
US7808503B2 (en
Inventor
Jerome Duluk
Richard Hessel
Vaughn Arnold
Jack Benkual
Joseph Bratt
George Cuan
Stephen Dodgen
Emerson Fang
Zhaoyu Gong
Thomas Ho
Hengwei Hsu
Sidong Li
Sam Ng
Matthew Papakipos
Jason Redgrave
Sushma Trivedi
Nathan Tuck
Shun Go
Lindy Fung
Tuan Nguyen
Joseph Grass
Bo Hung
Abraham Mammen
Abbas Rashid
Albert Tsay
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Apple Inc
Original Assignee
Apple Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US9733698P priority Critical
Priority to US09/377,503 priority patent/US6717576B1/en
Priority to US10/458,493 priority patent/US7167181B2/en
Application filed by Apple Inc filed Critical Apple Inc
Priority to US11/613,093 priority patent/US7808503B2/en
Assigned to RAYCER, INC. reassignment RAYCER, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: DODGEN, STEPHEN L., TUCK, NATHAN D., BENKUAL, JACK, ARNOLD, VAUGHN T., FUNG, LINDY, HONG, BO, NGUYEN, TUAN D., REDGRAVE, JASON R., GO, SHUN WAI, GRASS, JOSEPH P., HSU, HENGWEI, LI, SIDONG, RASHID, ABBAS, TRIVEDI, SUSHMA S., TSAY, ALBERT SUAN-WEI, BRATT, JOSEPH P., FANG, EMERSON S., HO, THOMAS Y., MAMMEN, ABRAHAM, NG, SAM, CUAN, GEORGE, GONG, ZHAOYU, DULUK, JEROME F., JR., HESSEL, RICHARD E.
Assigned to APPLE INC. reassignment APPLE INC. CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: APPLE COMPUTER, INC.
Assigned to APPLE COMPUTER, INC. reassignment APPLE COMPUTER, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: RAYCER, INC.
Publication of US20070165035A1 publication Critical patent/US20070165035A1/en
Publication of US7808503B2 publication Critical patent/US7808503B2/en
Application granted granted Critical
Application status is Active legal-status Critical
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/10Geometric effects
    • G06T15/20Perspective computation
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/60Memory management
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/001Texturing; Colouring; Generation of texture or colour
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/40Filling a planar surface by adding surface attributes, e.g. colour or texture
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/005General purpose rendering architectures
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/04Texture mapping
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/10Geometric effects
    • G06T15/30Clipping
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/10Geometric effects
    • G06T15/40Hidden part removal
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/50Lighting effects
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/50Lighting effects
    • G06T15/80Shading
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/50Lighting effects
    • G06T15/80Shading
    • G06T15/83Phong shading
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/50Lighting effects
    • G06T15/80Shading
    • G06T15/87Gouraud shading

Abstract

A deferred shading graphics pipeline processor and method are provided encompassing numerous substructures. Embodiments of the processor and method may include one or more of deferred shading, a tiled frame buffer, and multiple?stage hidden surface removal processing. In the deferred shading graphics pipeline, hidden surface removal is completed before pixel coloring is done. The pipeline processor comprises a command fetch and decode unit, a geometry unit, a mode extraction unit, a sort unit, a setup unit, a cull unit, a mode injection unit, a fragment unit, a texture unit, a Phong lighting unit, a pixel unit, and a backend unit.

Description

    RELATED APPLICATIONS
  • This application is a continuation of U.S. application Ser. No. 09/377,503, filed 20 Aug. 1999, which is hereby incorporated by reference and which claims the benefit under 35 USC Section 119(e) of U.S. Provisional Patent Application Ser. No. 60/097,336 filed 20 Aug. 1998 and entitled GRAPHICS PROCESSOR WITH DEFERRED SHADING; and claims the benefit under 35 USC Section 120 of U.S. patent application Ser. No. 09/213,990 filed 17 Dec. 1998 entitled HOW TO DO TANGENT SPACE LIGHTING IN A DEFERRED SHADING ARCHITECTURE; each of which is hereby incorporated by reference.
  • This application is also related to the following U.S. patent applications, each of which are incorporated herein by reference:
  • Ser. No. 09/213,990, filed 17 Dec. 1998, entitled HOW TO DO TANGENT SPACE LIGHTING IN A DEFERRED SHADING ARCHITECTURE;
  • Ser. No. 09/378,598, filed 20 Aug. 1999, entitled APPARATUS AND METHOD FOR PERFORMING SETUP OPERATIONS IN A 3-D GRAPHICS PIPELINE USING UNIFIED PRIMITIVE DESCRIPTORS;
  • Ser. No. 09/378,633, filed 20 Aug. 1999, now U.S. Pat. No. 6,552,723 entitled SYSTEM, APPARATUS AND METHOD FOR SPATIALLY SORTING IMAGE DATA IN A THREE-DIMENSIONAL GRAPHICS PIPELINE;
  • Ser. No. 09/378,439, filed 20 Aug. 1999, entitled GRAPHICS PROCESSOR WITH PIPELINE STATE STORAGE AND RETRIEVAL, now U.S. Pat. No. 6,525,737;
  • Ser. No. 09/378,408, filed 20 Aug. 1999, entitled METHOD AND APPARATUS FOR GENERATING TEXTURE, now U.S. Pat. No. 6,288,730;
  • Ser. No. 09/379,144, filed 20 Aug. 1999, entitled APPARATUS AND METHOD FOR GEOMETRY OPERATIONS IN A 3D GRAPHICS PIPELINE;
  • Ser. No. 09/372,137, filed 20 Aug. 1999, entitled APPARATUS AND METHOD FOR FRAGMENT OPERATIONS IN A 3D GRAPHICS PIPELINE;
  • Ser. No. 09/378,391, filed 20 Aug. 1999, entitled Method And Apparatus For Performing Conservative Hidden Surface Removal In A Graphics Processor With Deferred Shading, now U.S. Pat. No. 6,476,807;
  • Ser. No. 09/378,299, filed 20 Aug. 1999, entitled DEFERRED SHADING GRAPHICS PIPELINE PROCESSOR, now U.S. Pat. No. 6,229,553; and
  • Ser. No. 10/358,134, filed 3 Feb. 2003, entitled GRAPHICS PROCESSOR WITH DEFERRED SHADING, hereby incorporated by reference, which is a continuation of Ser. No. 09/378,637, filed 20 Aug. 1999, entitled DEFERRED SHADING GRAPHICS PIPELINE PROCESSOR, hereby incorporated by reference, which claims the benefit of the filing date of U.S. Provisional Application Ser. No. 60/097,336, filed 20 Aug. 1999.
  • FIELD OF THE INVENTION
  • This invention relates to computing systems generally, to three-dimensional computer graphics, more particularly, and more most particularly to structure and method for a three-dimensional graphics processor implementing differed shading and other enhanced features.
  • BACKGROUND OF THE INVENTION
  • The Background of the Invention is divided for convenience into several sections which address particular aspects conventional or traditional methods and structures for processing and rendering graphical information. The section headers which appear throughout this description are provided for the convenience of the reader only, as information concerning the invention and the background of the invention are provided throughout the specification.
  • Three-Dimensional Computer Graphics
  • Computer graphics is the art and science of generating pictures, images, or other graphical or pictorial information with a computer. Generation of pictures or images, is commonly called rendering. Generally, in three-dimensional (3D) computer graphics, geometry that represents surfaces (or volumes) of objects in a scene is translated into pixels (pasture elements) stored in a frame buffer, and then displayed on a display device. Real-time display devices, such as CRTs used as computer monitors, refresh the display by continuously displaying the image over and over. This refresh usually occurs row-by-row, where each row is called a raster line or scan line. In this document, raster lines are generally numbered from bottom to top, but are displayed in order from top to bottom.
  • In a 3D animation, a sequence of images is displayed, giving the illusion of motion in three-dimensional space. Interactive 3D computer graphics allows a user to change his viewpoint or change the geometry in real-time, thereby requiring the rendering system to create new images on-the-fly in real-time.
  • In 3D computer graphics, each renderable object generally has its own local object coordinate system, and therefore needs to be translated (or transformed) from object coordinates to pixel display coordinates. Conceptually, this is a 4-step process: 1) translation (including scaling for size enlargement or shrink) from object coordinates to world coordinates, which is the coordinate system for the entire scene; 2) translation from world coordinates to eye coordinates, based on the viewing point of the scene; 3) translation from eye coordinates to perspective translated eye coordinates, where perspective scaling (farther objects appear smaller) has been performed; and 4) translation from perspective translated eye coordinates to pixel coordinates, also called screen coordinates. Screen coordinates are points in three-dimensional space, and can be in either screen-precision (i.e., pixels) or object-precision (high precision numbers, usually floating-point), as described later. These translation steps can be compressed into one or two steps by precomputing appropriate translation matrices before any translation occurs. Once the geometry is in screen coordinates, it is broken into a set of pixel color values (that is “rasterized”) that are stored into the frame buffer. Many techniques are used for generating pixel color values, including Gouraud shading, Phong shading, and texture mapping.
  • A summary of the prior art rendering process can be found in: “Fundamentals of Three-dimensional Computer Graphics”, by Watt, Chapter 5: The Rendering Process, pages 97 to 113, published by Addison-Wesley Publishing Company, Reading, Mass., 1989, reprinted 1991, ISBN 0-201-15442-0 (hereinafter referred to as the Watt Reference), and herein incorporated by reference.
  • FIG. 1 shows a three-dimensional object, a tetrahedron, with its own coordinate axes (xobj,yobj,zobj). The three-dimensional object is translated, scaled, and placed in the viewing point's coordinate system based on (xeye,yeye,zeye). The object is projected onto the viewing plane, thereby correcting for perspective. At this point, the object appears to have become two-dimensional; however, the object's z-coordinates are preserved so they can be used later by hidden surface removal techniques. The object is finally translated to screen coordinates, based on (xscreen,yscreen,zscreen), where zscreen is going perpendicularly into the page. Points on the object now have their x and y coordinates described by pixel location (and fractions thereof) within the display screen and their z coordinates in a scaled version of distance from the viewing point.
  • Because many different portions of geometry can affect the same pixel, the geometry representing the surfaces closest to the scene viewing point must be determined. Thus, for each pixel, the visible surfaces within the volume subtended by the pixel's area determine the pixel color value, while hidden surfaces are prevented from affecting the pixel. Non-opaque surfaces closer to the viewing point than the closest opaque surface (or surfaces, if an edge of geometry crosses the pixel area) affect the pixel color value, while all other non-opaque surfaces are discarded. In this document, the term “occluded” is used to describe geometry which is hidden by other non-opaque geometry.
  • Many techniques have been developed to perform visible surface determination, and a survey of these techniques are incorporated herein by reference to: “Computer Graphics: Principles and Practice”, by Foley, van Dam, Feiner, and Hughes, Chapter 15: Visible-Surface Determination, pages 649 to 720, 2nd edition published by Addison-Wesley Publishing Company, Reading, Mass., 1990, reprinted with corrections 1991, ISBN0-201-12110-7 (hereinafter referred to as the Foley Reference). In the Foley Reference, on page 650, the terms “image-precision” and “object-precision” are defined: “Image-precision algorithms are typically performed at the resolution of the display device, and determine the visibility at each pixel. Object-precision algorithms are performed at the precision with which each object is defined, and determine the visibility of each object.”
  • As a rendering process proceeds, most prior art renderers must compute the color value of a given screen pixel multiple times because multiple surfaces intersect the volume subtended by the pixel. The average number of times a pixel needs to be rendered, for a particular scene, is called the depth complexity of the scene. Simple scenes have a depth complexity near unity, while complex scenes can have a depth complexity of ten or twenty. As scene models become more and more complicated, renderers will be required to process scenes of ever increasing depth complexity. Thus, for most renders, the depth complexity of a scene is a measure of the wasted processing. For example, for a scene with a depth complexity of ten, 90% of the computation is wasted on hidden pixels. This wasted computation is typical of hardware renderers that use the simple Z-buffer technique (discussed later herein), generally chosen because it is easily built in hardware. Methods more complicated than the Z Buffer technique have heretofore generally been too complex to build in a cost-effective manner. An important feature of the method and apparatus invention presented here is the avoidance of this wasted computation by eliminating hidden portions of geometry before they are rasterized, while still being simple enough to build in cost-effective hardware.
  • When a point on a surface (frequently a polygon vertex) is translated to screen coordinates, the point has three coordinates: (1) the x-coordinate in pixel units (generally includi