Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—Three-dimensional [3D] image rendering
  • G06T15/005—General purpose rendering architectures

Definitions

• the invention relates generally to computer graphics.
• the invention relates to the collection of graphics data and sending the data to graphics hardware.
• the production of computer graphics images comprises two fundamental stages.
• the first is the generation of data that specifies a frame or scene, known hereinafter as scene data.
• the second is the rendering of an image for display, based on the scene data, by graphics hardware.
• Some current computer graphics systems utilize one processor to handle the generation of scene data, and another processor to send the scene data to the graphics hardware.
• the two stages typically take place alternately and in serial.
• Scene data is generated by the first processor.
• the second processor then sends the scene data to the geometry engine, the first module in a graphics hardware suite.
• the first processor then generates data for the next scene, which the second processor then sends to the geometry engine.
• the process continues in serial fashion.
• This architecture has a number of drawbacks.
• the invention described herein comprises a method, system, and computer program product for sending scene data to a geometry engine.
• a processor generates scene data for a frame in accordance with an application program, and writes the scene data to a first memory location, known hereinafter as an intermediate buffer.
• Scene data for the next frame is then generated and written to a second intermediate buffer, while a geometry engine concurrently reads and renders the scene data in the first intermediate buffer.
• Scene data for the following frame is then generated and written to the first intermediate buffer, while a geometry engine reads and renders the scene data in the second intermediate buffer. The process continues in this manner until the application program is finished executing.
• the invention has the feature of reading and rendering scene data while simultaneously generating additional scene data.
• the invention has the additional feature of generating and rendering scene data in a pipeline architecture.
• the invention has the advantage of relatively high throughput in the processing of scene data.
• the invention has the further advantage of requiring only a single processor to both generate scene data and make the scene data available to a geometry engine.
• FIG. 1 is a block diagram illustrating the architecture of a computer graphics system, according to current art.
• FIG. 2 is a timing diagram illustrating the sequence of events that results from a computer graphics system of the type illustrated in FIG. 1.
• FIG. 3 is a block diagram illustrating the architecture of a computer graphics system, according to an embodiment of the invention.
• FIG. 4 is a block diagram illustrating the interaction between a processor, a geometry engine, and intermediate buffers, according to an embodiment of the invention.
• FIG. 5 A is a block diagram illustrating the flat organization of scene data as contained in an intermediate buffer, according to an embodiment of the invention.
• FIG. 5B is a block diagram illustrating the hierarchical organization of scene data as contained in an intermediate buffer and in other memory locations, according to an embodiment of the invention.
• FIG. 6 is a block diagram illustrating the interaction between multiple processors, a geometry engine, and intermediate buffers, according to an embodiment of the invention.
• FIG. 7 is a timing diagram illustrating the temporal relationship between the scene data generation by a processor and the reading and rendering of scene data by graphics hardware, according to an embodiment of the invention.
• FIG. 8 is a flowchart showing the overall operation of an embodiment of the invention.
• FIG. 9 is a flowchart showing the process of making scene data available to a geometry engine, according to an embodiment of the invention.
• FIG. 10 is a flowchart showing the process of reading and rendering scene data, according to an embodiment of the invention.
• FIG. 11 is a flowchart showing the process of context switching, according to an embodiment of the present invention.
• FIG. 12 is a block diagram illustrating the computing environment of the invention.
• a processor to generate scene data in accordance with an application program running on the processor.
• a second processor is commonly allocated for the purpose of relaying the generated scene data from the first processor to a geometry engine, where the process of transforming the scene data into a display image is begun.
• FIG. 1 A first processor 105 receives an application program 110. By executing application 110, processor 105 generates scene data for a frame. In so doing, processor 105 interacts as necessary with a memory unit
• processor 115 The scene data generated by processor 105 is then transferred to geometry engine 125 by a second processor 118. It is the responsibility of processor 118 to transfer scene data 120 from processor 105 to geometry engine 125.
• FIG. 2 The sequence of events resulting from such an architecture is illustrated in FIG. 2.
• the events are illustrated as a function of time 205.
• a first processor generates scene data corresponding to a frame n .
• a second processor sends the scene data of frame n to the geometry engine.
• the first processor generates scene data for the next frame, frame n+1 .
• the second processor sends the scene data pertaining to frame n+1 to the geometry engine.
• the process continues in serial fashion. Each scene data generation event is followed in serial by an event wherein a second processor sends the scene data to a geometry engine. This process is repeated for each successive frame.
• a second processor is unnecessary.
• a single processor generates scene data and transfers the scene data to a intermediate buffer in memory.
• the processor then issues instructions to the geometry engine to retrieve the scene data.
• the processor sends to the geometry engine the address, in memory, of the intermediate buffer.
• the geometry engine then retrieves the scene data from the indicated address.
• the processor While the geometry engine is retrieving the scene data and performing the necessary rendering of the scene data, the processor is generating scene data for the next frame. When completed, the scene data for the next frame is sent by the processor to a second intermediate buffer. Once this transfer to memory is completed and the graphics hardware has finished rendering the scene data from the first intermediate buffer, the processor issues another read instruction to the geometry engine. The geometry engine proceeds to read the next segment of scene data from the second intermediate buffer location.
• the processor While the geometry engine does this, the processor generates scene data for a third frame, placing the scene data into the first memory location. The process continues with the scene data generation proceeding in parallel, concurrent with the scene data reading and rendering operations. Therefore, while the graphics hardware reads and renders a frame of scene data, the processor is generating scene data for the following frame. As a result, the collective processes of generating scene data and transferring the scene data to graphics hardware become more efficient. The collective processes are pipelined, and throughput is increased. Moreover, only one processor is used. //. Terminology
• Application is used broadly herein to describe a series of instructions to be executed by a processor.
• the term refers to a series of instructions that, when executed, produce data that can be rendered to form an image, a portion of an image, or a series of images.
• Frame refers to a segment of data that, when input to a rendering process, yields an image or a portion of an image.
• Geometry engine refers to a hardware component that performs the first steps in a rendering process. These steps can comprise geometric calculations, lighting calculations, perspective operations, and color processing of pixels.
• the output of a geometry engine comprises data that can be rasterized.
• Intermediate buffer refers to a location in a memory medium where scene data (defined below) can be stored immediately after generation, according to an embodiment of the invention.
• An intermediate buffer serves as a temporary location for scene data storage, pending subsequent access of the data by a geometry engine.
• Scene data refers to data generated by the execution of an application by a processor. Scene data is input to a geometry engine to begin the rendering process, and is eventually rendered into an image. ///. System
• FIG. 3 An embodiment of the present invention is illustrated in FIG. 3.
• a processor 305 is shown loading an application 110.
• the application 110 may be written in any of several languages suitable for programming graphics hardware.
• Scene data 120 contains descriptions of objects and features that will ultimately comprise a displayed frame. If OpenGL ® is used, scene data 120 comprises display lists. Once generated, scene data 120 associated with a given frame is transferred to a memory module 115. In particular, scene data 120 for the frame is written to one of a plurality of specific locations in memory 115, known as intermediate buffers.
• Geometry engine 310 is responsible for beginning the rendering process, and performs such tasks as the initial manipulation of the geometric entities that will eventually form a scene. Instructions 315 tell the geometry engine 310 to read scene data 120 from memory 115. Moreover, instructions 315 tell the geometry engine 310 the address of the intermediate buffer in which scene data 120 can be found. The geometry engine 310 then retrieves scene data 120 from the specified intermediate buffer in memory 115.
• Memory 115 comprises two intermediate buffers 405 A and 405B.
• Processor 305 which generates scene data on a frame-by-frame basis, first generates scene data associated with a frame 415.
• Processor 305 sends the scene data associated with frame 415 to intermediate buffer 405 A. Geometry engine 310 can then access the scene data associated with frame 415 from intermediate buffer 405 A when told to do so by processor 305. While intermediate buffer 405 A is being accessed and its scene data rendered, processor 305 generates scene data associated with a next frame 420.
• Processor 305 writes the scene data associated with frame 420 into intermediate buffer 405B.
• geometry engine 310 reads the scene data associated with frame 420 from intermediate buffer 405B. While doing so, processor 305 generates scene data for the next frame and writes the scene data to intermediate buffer 405A. The process continues in this fashion. Processor 305 therefore writes alternately to intermediate buffers 405 A and 405B. Geometry engine 310 reads scene data from whichever intermediate buffer is not currently being addressed by processor 305. This effects a pipelined data flow process which will be described in greater detail below.
• more than two intermediate buffers may be used. If, for example, the processor finishes writing to a first intermediate buffer before the geometry engine has finished reading fron the second intermediate buffer, then the processor will have nowhere to which it can write the scene data for the next frame. To handle such conditions, three or more intermediate buffers may be used to support the pipeline process of the invention.
• FIGs. 5A and 5B Possible formats for scene data, according to an embodiment of the invention, are described in FIGs. 5A and 5B.
• scene data for a frame is illustrated.
• the scene data comprises a plurality of object descriptions.
• the first description defines an object 505.
• the second and third object descriptions define objects 510 and 515, respectively.
• the data in each object description comprises information sufficient for the graphics system, including the geometry engine, to begin rendering that portion of the frame corresponding to the object.
• Scene data consisting of a series of explicit object descriptions is said to have a flat structure.
• FIG. 5B illustrates an exemplary hierarchical structure.
• the scene data 550 comprises descriptions of objects, such as object 555.
• the scene data also comprises pointers, such as pointers 560, 570, and 575, which refer to object descriptions elsewhere in memory.
• pointer 560 serves as a reference to an object described elsewhere in memory, object 565.
• pointer 570 also references object 565.
• Pointer 575 references a different object 580. Descriptions of objects 565 and object 580 are not explicitly included in the scene data written by the processor to the intermediate buffer.
• Scene data 550 written to the intermediate buffer therefore comprises object 555, and pointers 560, 570, and 575.
• Scene data 550 is equivalent to scene data comprising explicit descriptions of objects 555, 565, 565 (again), and 580. Because a pointer requires less memory space than an explicit object description, scene data in a hierarchical format is a more compact representation of a scene. D. Multiple processors and scene generation
• a plurality of processors execute their own respective applications.
• all the processors write their own scene data to a single intermediate buffer.
• the resulting plurality of scene data blocks is interpreted collectively as a single frame.
• An exemplary embodiment is illustrated in FIG. 6.
• processors 605A, 605B, and 605C Each processor writes its own scene data to an intermediate buffer 601B.
• Scene data 610A is provided by processor 605A
• scene data 610B is provided by processor 605B
• scene data 610C is provided by processor 605C.
• These blocks of scene data may be either flat or hierarchical, as illustrated in FIGs. 5A and 5B.
• a pipelined sequence of operations takes place, similar to the sequence illustrated in FIG. 4. While processors 605A through 605C are generating and writing the scene data to intermediate buffer 60 IB, geometry engine 125 is reading the contents of intermediate buffer 601 A, which contains scene data written in the previous step by processors 605A through 605C. After the scene data in intermediate buffer 601 A has been rendered, and after processors 605A through 605C have completed writing their scene data to intermediate buffer 601B, geometry engine 125 will read the scene data in intermediate buffer 60 IB. Meanwhile, processors 605 A through 605C will generate scene data for the next frame. This scene data will be written to intermediate buffer 601 A. Operations continue in this manner until all scene data has been generated and processed.
• the method of the invention provides for parallel execution of two sets of processes. While graphics hardware reads and renders the scene data generated by a processor for a given frame, the processor generates scene data for the next frame and writes it to memory. In this manner, the generation of scene data is performed during the reading and rendering of previously generated scene data.
• This pipeline approach allows for greater throughput in a graphics system, and provides for efficient usage of resources.
• a timeline of these events is shown in FIG. 7.
• the events are shown as a function of time 705.
• a processor generates scene data associated with a frame n and writes it to an intermediate buffer in memory.
• a processor generates and writes scene data associated with a frame n+1 .
• the processor In an event 720, the processor generates and writes scene data associated with a frame n+2 .
• scene data is also being read from memory and rendered by graphics hardware. While, in event 710, the processor is generating and writing scene data associated with frame,, the geometry engine is also acting. At this point, in an event 725, the geometry engine is reading scene data associated with frame n _ j from an intermediate buffer in memory.
• the graphics hardware is reading and rendering scene data that had previously been generated for frame n in an event 730.
• the scene data generated in event 715 is read and rendered by the graphics hardware in an event 735. The latter event takes place while the processor is generating scene data associated with the next frame, frame n+2 , in event 720.
• any flags or indices which are used in the process are set to appropriate initial values. As will be described below, such flags may be used for signalling the completion of various tasks.
• scene data for a first frame, frame 0 is made available in a first intermediate buffer, denoted in FIG. 8 as buffer A.
• a processor generates scene data and writes the scene data to buffer A in memory.
• step 815 two events happen in an approximately contemporaneous way.
• the scene data that had just been written to buffer A is read from that intermediate buffer by the geometry engine and rendered by the graphics hardware.
• the processor generates scene data associated with a next frame, frame ! , and writes it to a second intermediate buffer
• buffer B (identified in FIG. 8 as buffer B).
• step 830 takes place.
• the scene data associated with frame ! is read from buffer B by the geometry engine and rendered by the graphics hardware. This is the scene data that was made available in step 825.
• the processor generates scene data associated with a next frame, frame 2 .
• the processor also writes this scene data to intermediate buffer A.
• a step 840 the scene data associated with frame 2 is read from buffer A by the geometry engine and rendered by the graphics hardware. Meanwhile, in a step 845, the processor generates scene data associated with the next frame, frame The processor makes this scene data available in buffer B.
• a step 850 the scene data associated with frame, is read from buffer B by the geometry engine and rendered by the graphics hardware. The process continues in parallel, pipeline fashion until the application has completed execution in the processor.
• a processor makes scene data available in memory (steps 815, 825, 835, and 845) are illustrated in greater detail in FIG. 9.
• the process begins with a step 900.
• the processor generates scene data for a frame n .
• a step 910 a determination is made as to whether the geometry engine is finished reading from the intermediate buffer to which the frame, scene data is to be written. In an embodiment of the invention, this determination can be made by using a system of flags which are set and reset depending on the status of the geometry engine's read operation. If the geometry engine is not finished reading from this buffer, then, in a step 915, the process must wait until the geometry engine is finished doing so. When and if the geometry engine is finished reading from the buffer, then, in a step 920, the processor proceeds to write the scene data associated with frame n to the buffer. The process concludes with a step 925.
• step 1000 a determination is made as to whether the geometry engine is ready to read scene data from an intermediate buffer.
• the geometry engine may, for example, still be reading from the another memory location. In this case, the geometry engine is not yet ready to read from the present buffer. Therefore, in a step 1010, the process must wait until the geometry engine is ready to read from the buffer.
• the processor tells the geometry engine to read and begin rendering the scene data associated with frame n .
• the processor specifies the memory address, i.e., buffer location, from which the geometry engine is to read the scene data.
• the determination as to whether the geometry engine is ready to read scene data can be made by using a system of flags which are set and reset depending on the status of the geometry engine's readiness to read.
• the geometry engine reads and begins rendering the scene data associated with frame n from the buffer. The process ends with a step 1025.
• the context of scene data refers to the cu ⁇ ent value of state variables, such as the current color or current polygon drawing mode, and is defined by the application.
• some embodiments of the invention include a plurality of processors, each executing its own respective application. A given processor may even execute multiple applications if the processor has multitasking capability. Because each application has its own sequence of instructions, each application is said to have its own thread. In such an embodiment, a plurality of blocks of scene data are written to an intermediate buffer. The blocks of scene data collectively represent the scene data for a frame.
• the graphics hardware must be cognizant of the plurality of contexts represented in the plurality of scene data blocks. Moreover, the graphics hardware must be able to handle the necessary context switching as it renders a block of scene data from one application, then moves to another block.
• FIG. 11 The process of switching contexts, according to an embodiment of the invention, is illustrated in FIG. 11.
• the rendering of a block; of scene data ends, and the rendering of a block j begins.
• Block ; and block j have been produced by separate processors executing separate applications.
• block; and block j have potentially different contexts, and the geometry engine must be able to switch contexts.
• the process begins with a step 1105.
• a step 1110 the rendering of block; is halted.
• the context of block is saved.
• the context of a block j is loaded.
• the rendering of block j is commenced.
• the process concludes with a step 1130.
• the present invention operates in the setting of a computer graphics system, and can be implemented in hardware, firmware, or software, or a combination thereof. A more complete illustration of such a system is presented in FIG. 12. After reading this description, it will become apparent to a person skilled in the relevant art how to implement the invention using other computer systems and/or computer architectures.
• the computer system 1200 includes one or more processors, such as processor 305.
• Processor 305 is connected to a communication infrastructure 1206, such as a bus or network.
• Computer system 1200 also includes memory 115, preferably random access memory (RAM).
• Processor 305 uses communication infrastructure 1206 to communicate with memory 115.
• Computer system 1200 may also include a secondary memory 1210.
• the secondary memory 1210 may include, for example, a hard disk drive 1212 and/or a removable storage drive 1214, representing a floppy disk drive, a magnetic tape drive, an optical disk drive, etc.
• the removable storage drive 1214 reads from and/or writes to a removable storage unit 1218 in a well known manner.
• Removable storage unit 1218 represents a floppy disk, magnetic tape, optical disk, or other storage medium which is read and written to by removable storage drive 1214.
• the removable storage unit 1218 includes a computer usable storage medium having stored therein computer software and/or data.
• secondary memory 1210 may include other means for allowing computer programs or other instructions to be loaded into computer system 1200.
• Such means may include, for example, a removable storage unit 1222 and an interface 1220.
• Examples of such means may include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM, or PROM) and associated socket, and other removable storage units 1222 and interfaces 1220 which allow software and data to be transferred from the removable storage unit
• Computer system 1200 may also include a communications interface
• Communications interface 1224 allows software and data to be transferred between computer system 1200 and external devices. Examples of communications interface 1224 may include a modem, a network interface (such as an Ethernet card), a communications port, a PCMCIA slot and card, etc.
• Software and data transferred via communications interface 1224 are in the form of signals 1228 whichmay be electronic, electromagnetic, optical or other signals capable of being received by communications interface 1224. These signals 1228 are provided to communications interface 1224 via a communications path (i.e., channel) 1226.
• This channel 1226 ca ⁇ ies signals 1228 and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link and other communications channels.
• geometry engine 310 receives instructions from processor 305 via interface 1224 and communications path 1226. In an embodiment of the invention, geometry engine 310 also receives scene data 120 from memory 115 via interface 1224 and communications path 1226.
• computer program medium and “computer usable medium” are used to generally refer to media such as removable storage units 1218 and 1222, a hard disk installed in hard disk drive 1212, and signals
• Computer programs are stored in memory 115 and/or secondary memory 1210. Computer programs may also be received via communications interface 1224. Such computer programs, when executed, enable the computer system 1200 to implement the present invention as discussed herein. In particular, the computer programs, when executed, enable the processor 305 to implement the present invention. Accordingly, such computer programs represent controllers of the computer system 1200. Where the invention is implemented using software, the software may be stored in a computer program product and loaded into computer system 1200 using removable storage drive 1214, hard drive 1212 or communications interface
• application program 110 is implemented in software and can therefore be made available to processor 305 through any of these means.

Landscapes

• Engineering & Computer Science (AREA)
• Computer Graphics (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Image Generation (AREA)
• Image Processing (AREA)
• Processing Or Creating Images (AREA)
• Computer And Data Communications (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Publications (2)

Family

ID=23525345

Family Applications (1)

Country Status (6)

Families Citing this family (4)

Family Cites Families (15)

• 1999
  • 1999-08-31 US US09/386,378 patent/US6636224B1/en not_active Expired - Lifetime
• 2000
  • 2000-08-31 EP EP00974067A patent/EP1210691B1/en not_active Expired - Lifetime
  • 2000-08-31 DE DE60003320T patent/DE60003320T2/de not_active Expired - Lifetime
  • 2000-08-31 JP JP2001520354A patent/JP4497445B2/ja not_active Expired - Lifetime
  • 2000-08-31 AT AT00974067T patent/ATE242900T1/de not_active IP Right Cessation
  • 2000-08-31 WO PCT/US2000/040786 patent/WO2001016883A2/en not_active Ceased

Also Published As

Similar Documents

Legal Events