CN1272725C - Hierarchial graph processing system and method - Google Patents

Hierarchial graph processing system and method Download PDF

Info

Publication number
CN1272725C
CN1272725C CN 02147328 CN02147328A CN1272725C CN 1272725 C CN1272725 C CN 1272725C CN 02147328 CN02147328 CN 02147328 CN 02147328 A CN02147328 A CN 02147328A CN 1272725 C CN1272725 C CN 1272725C
Authority
CN
China
Prior art keywords
parts
data
engine
information
graphical information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN 02147328
Other languages
Chinese (zh)
Other versions
CN1417700A (en
Inventor
J·S·蓓达
G·D·斯维博格
O·D·温古雷亚努
K·T·加洛
P·C·戴维
M·W·卡尔金斯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Technology Licensing LLC
Original Assignee
Microsoft Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Corp filed Critical Microsoft Corp
Publication of CN1417700A publication Critical patent/CN1417700A/en
Application granted granted Critical
Publication of CN1272725C publication Critical patent/CN1272725C/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Landscapes

  • Processing Or Creating Images (AREA)

Abstract

A multiple-level graphics processing system and method (e.g., of an operating system) for providing improved graphics output including, for example, smooth animation. One such multiple-level graphics processing system comprises two components, including a tick-on-demand or slow-tick high-level component, and a fast-tick (e.g., at the graphics hardware frame refresh rate) low-level component. In general, the high-level, less frequent component performs computationally intensive aspects of updating animation parameters and traversing scene data structures, in order to pass simplified data structures to the low-level component. The low-level component operates at a higher frequency, such as the frame refresh rate of the graphics subsystem, to process the data structures into constant output data for the graphics subsystem. The low-level processing includes interpolating any parameter intervals as necessary to obtain instantaneous values to render the scene for each frame of animation.

Description

Hierarchial graph processing system and method
The related application contrast
The present invention requires the right of priority of the U.S. Provisional Patent Application sequence number 60/330,244 of application on October 18 calendar year 2001.The U.S. Patent application of " Generic Parameterization for aScene Graph " (attorney docket 3130) and " Intelligent Caching Data Structurefor Immediate Mode Graphics " (attorney docket 3150) by name that the present invention relates to apply for simultaneously, the both is transferred to assignee of the present invention and is incorporated herein by comprehensive reference.
Technical field
The present invention relates generally to computer system, relate in particular to the figure that on computer system, shows and the processing of other video information.
Background technology
In the computing system in the present age, the ability of figure and vision hardware increases just fast.In fact, on certain degree, can more think coprocessor to the graphics system in the contemporary computing system, rather than simple graphics subsystem.Simultaneously, for example no matter watch monitor, televisor or Cellular Networks phone set display, the user is just expecting that the quality in the shown image is more and more higher.
Yet storer and Bus Speed are not also caught up with the progress in primary processor and/or the graphic process unit.As a result, reaching the limitation of traditional direct mode model of access figure on the computer system.Simultaneously, developer and user are just requiring traditional graph window body architecture unappeasable new feature of institute and special-effect.
Utilize graphic hardware though designed some games, such games are mainly that to be different from the requirement operation of multipad etc. described recreation does not need to relate to other program that may move simultaneously.Be different from such games, application program need be shared figure and other system resource with other application program.Yet, with regard to graphics process, the general shared model that they is not written as machine cooperation, complete.
For example, carry out the special private code of the current requirement of cartoon making or use other application program with multipad.Even now even be not impossible, realizes that in multi-windowed environment smoothly moving picture also is difficult.In general, this animation level and smooth because of finishing, at a high speed requires with higher frame rate, it is desirable to the hardware refresh speed with described graphics device, upgrades animation parameters and repaints scene (this requires traversal and drawing data structure).Yet the data structure of upgrading animation parameters and traversal and drafting definition scene is intensive on calculating generally.The scene of making is big more or many more, calculation requirement is just big more, this limited the complexity of the scene that can smoothly make.
When described graphic hardware shows when refreshing, need to calculate, draw and prepare to show that the requirement of each frame of animation complicates the issue.Also be not ready for if work as the hardware requirement time frame, the result is the frame of losing or postpone.If lost considerable frame, then in animation display, just have and see easily intermittently.And, not synchronous if frame is prepared with described refresh rate, may be called as undesirable influence of tearing.In fact, Dang Dai multiple task operating system distributes calculation resources between many system tasks.Yet the operating system task scheduler program is handled the time quantum that provides to frame will seldom aim at described graphic hardware frame rate.Therefore, even when having enough computational resources, because scheduling problem makes still possibility lost frames of described animation system.For example, may dispatch animation task and make it to move the day after the fair, or before finishing a frame, try to be the first, and the hardware refresh next time that can not in time reschedule to screen provides next frame.If animated graphs need be synthetic with other source of the frame of video or asynchronous generation, then these problems will be more complicated.
In general, when requiring complex figure effect (as the animation of complexity), the current model (as WM_PAINT) that is used to prepare frame requires too many data processing, to such an extent as to can not catch up with refresh rate.The result, when attempting to obtain the complex figure effect with conventional model, can be added in variation on the different frame of the unwanted results that causes visually and see easily, rather than in next frame, finish described variation, described next frame will cause the perceptible visual effect in the time of next frame.Have computation model, they attempt to allow each object by in the scene figure that object handle is provided, and selectively insert to change.Yet even described object handle does not still exist when application program is not want object changed, such model requires a considerable amount of objects of application tracks, and consumes too many resource.
In a word, the existing model of access figure is just becoming and is being not enough to the expectation of working and satisfying the user with current viewing hardware on computer system.The new model that needs a kind of processing graphics and video.
Summary of the invention
Briefly, the invention provides a kind of Hierarchial graph processing system and method (as for operating system), be used to provide comprise for example improved figure access and the output of smoothly moving picture.In one implementation, described graphic system comprises two parts, comprises request formula timeslice (tick-on-demand) or timeslice (slow-tick) high-level components and timeslice (fast-tick) (as with graphic hardware frame rate or its multiple) low layer parts at a slow speed fast.In general, described high-level components travels through scene to display and uses the time interval of interpolation after a while to upgrade animation parameters, and the data structure of simplifying is passed to described low layer parts.Described low layer parts processing said data structure comprises that to obtain output data any parameter of interpolation obtains instantaneous value at interval as required, and each frame of animation is drawn scene.
In general, the present invention decomposes by graph data being carried out the factor, make in the calculating of upgrading animation parameters and traversal scene data structure on the high-rise processing element by the plan of less requirement intensive aspect, (and other) problem of having determined above having solved.Described low layer processing element turns round more continually, but handles intensive task in the less calculating, and this is because the data structure of relatively simplification and the high-rise pre-service that instruction is handled are provided.Can also during these low layer timeslices, become synthetic ingredient to sets of video frames.
Native system and method and benefit comprise the part of the animation of similar television-quality as operating system shell, and as the animation engine of activity description.Further benefit comprises the ability that video image and figure is synthetic and be used for the ability to a plurality of terminal distributing informations that high-quality video shows on network is connected, it must not be high bandwidth that described network connects, at least not abundant high bandwidth, carry the figure bit of conventional rasterisation with desired high frame rate.
Can provide the present invention by a kind of system, described system comprises the graphics subsystem of the frame that is used to export the display message that comprises figure; Provide 1st parts of graph data with the 1st speed to described graphics subsystem with the frame of output display message; Come to produce the 2nd parts of contextual data with the program code interface, described the 2nd component configuration is become with the 2nd speed that is lower than described the 1st speed described contextual data to be processed into graphical information, and provide this graphical information to described the 1st parts for described program code; And the cached data structure, described the 2nd parts remain at least some contextual data in the described cached data structure.
The present invention also provides a kind of method in the computing environment of the graphics subsystem of the frame with output display message, comprising:
At the 1st parts place:
1) receives and to comprise, be used for output corresponding to the calling of the data of graph image;
2) keep described data as scene information; And
3) with the 1st operating rate:
A) described scene information is processed into graphical information;
B) at least some results behind the described scene information of cache handles; And
C) propagate described graphical information to the 2nd parts;
And
At described the 2nd parts place, with faster than the 2nd operating rate of described the 1st operating rate and according to the frame updating rate of described graphics subsystem:
4) receive described graphical information;
5) described graphic information processing is become described graphics subsystem and formative graph data; And
6) propagate described graph data to described graphics subsystem, with output frame.
A kind of method and computer-readable media with the executable instruction of computing machine can comprise, receive the request that comprises corresponding to the data of the graph image that is used to export at the 1st parts place, described data are preserved as scene information, with the 1st arithmetic speed described scene information is processed into graphical information, and propagates described graphical information to the 2nd parts.At described the 2nd parts place, with than fast the 2nd speed of described the 1st arithmetic speed and based on the frame updating rate of described graphics subsystem, described method can comprise the described graphical information of reception, described graphic information processing is become the formative graph data of described graphics subsystem, and described graph data is propagated to described graphics subsystem with output frame.
A kind of computer-readable media that stores data structure thereon can comprise identifier domain with visual pattern identifier and upgrade piece corresponding to the visual pattern in the territory of instruction block, described instruction block corresponding to one group with relevant data of visual pattern by described visual pattern identifier sign, these group data are passed to low layer graphics process parts from high-rise figure processing element, and described high-level components construction data structure from the scene figure also offers described low layer parts with this data structure, described low layer parts are processed into graph data with described data structure, are used to pass to graphics subsystem with output frame.
Description of drawings
In conjunction with the accompanying drawings, other benefit and advantage will become clearer from following detailed, in the accompanying drawing:
Fig. 1 is that expression can be in conjunction with the block diagram of exemplary computer system of the present invention;
Fig. 2 is a block diagram of representing media integration layer architecture according to an aspect of the present invention;
Fig. 3 be represent according to an aspect of the present invention the intelligent cache data structure and the block diagram of the relation of it and each parts;
Fig. 4 represents that high level according to an aspect of the present invention synthesizes the block diagram of the general flow of the control that reaches animation engine and other interlayer;
Fig. 5 is the block diagram of representing according to an aspect of the present invention in example container and other node and their relation of simple data structure high speed buffer memory;
Fig. 6 is a block diagram of representing the general parts of and animation engine synthetic with the synergistic low layer of other parts according to an aspect of the present invention;
Fig. 7 is the synthetic block diagram of representing according to an aspect of the present invention that reaches the general parts of animation engine of low layer;
Fig. 8 is the block diagram of logical organization of representing the connection of and animation engine synthetic to described low layer according to an aspect of the present invention;
Fig. 9 and 10 comprises according to an aspect of the present invention synthetic and animation engine is synthetic to low layer and the block diagram of the information flow of animation engine from high level of expression;
Figure 11 and 12 is the block diagrams of representing according to an aspect of the present invention that the media integration layer architecture layers arrives the information flow of graphics subsystem that pass through;
Figure 13-22 comprises data structure according to an aspect of the present invention and has described the out of Memory that is used for from described high level synthesizes and animation engine is synthetic to described low layer and animation engine diffuses information; And
Figure 23 and 24 comprises the process flow diagram for the general description of the logic that is used for handle packet according to an aspect of the present invention.
Embodiment
The exemplary operation environment
Fig. 1 has illustrated the example that can realize suitable computingasystem environment 100 of the present invention thereon.Described computingasystem environment 100 only is that the example of a suitable computing environment and not planning proposes any restriction about use of the present invention or functional scope.Also computing environment 100 should be construed to the correlativity or the requirement of combination with any one or they of relating to the parts shown in the exemplary operation environment 100.
The present invention can be with many other universal or special computingasystem environment or configuration operation.Can be fit to well-known computing system, environment that uses with the present invention and/or the example that disposes comprises, but be not limited to, personal computer, server computer, portable or laptop devices, tablet equipment, multicomputer system, the system based on microprocessor, set-top box, programmable consumer electronic devices, network PC, microcomputer, mainframe computer, comprise distributed computing environment of any said system or equipment or the like.
To in the normal ranges of the executable instruction of carrying out by computing machine of computing machine such as program module, the present invention be described.Program module generally comprises routine, program, object, parts, data structure or the like, and they are carried out specific task or realize specific abstract data type.Can also in by the distributed computing environment of executing the task, carry out the present invention by the teleprocessing equipment of communication network link.In distributed computing environment, program module can be positioned to comprise in the local and remote computer storage media may of memory storage device.
With reference to figure 1, be used to realize that example system of the present invention comprises the universal computing device with the form of computing machine 110.The parts of described computer equipment 110 can include, but not limited to processing unit 120, system storage 130 and will comprise that the various system units of described system storage are coupled to the system bus 121 of described processing unit 120.Any in the bus structure that described system bus 121 can be some types comprises memory bus or Memory Controller, peripheral bus and uses multiple bus-structured any local bus.As an example, rather than restriction, such structure comprises industrial standard architectures (EISA) bus, VESA (VESA) local bus, AGP (AGP) bus, the peripheral component interconnect (pci) bus of industrial standard architectures (ISA) bus, Micro Channel Architecture (MCA) bus, enhancing, is also referred to as mezzanine (Mezzanine) bus.
Described computing machine 110 typically comprises multiple computer-readable media.Computer-readable media can be any can be by the useable medium of computing machine 110 accesses, and comprise volatibility and non-volatile two kinds of medium and dismountable and non-removable medium.As an example rather than the restriction, computer-readable media can comprise computer storage media may and communication medium.Computer storage media may comprises with any method of information or the technology volatibility that realizes and non-volatile media, the dismountable and non-removable medium of storage such as computer-readable instruction, data structure, program module or other data.Computer storage media may includes but not limited to, random-access memory (ram), ROM (read-only memory) (ROM), EEPROM (Electrically Erasable Programmable Read Only Memo) (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), universal disc (DVD) or other optical disc memory, tape cassete, tape, magnetic disk memory or other magnetic storage apparatus, or can be used for storing the information wanted and can be by any other medium of computing machine 110 accesses.Communication medium typically shows into computer-readable instruction, data structure, program module or such as other data in the modulated data-signal of carrier wave or other connecting gear, and comprises that any information transmits medium.Term " modulated data-signal " refers to a signal that is provided with or changes its one or more characteristics in this mode that the information in the described signal is encoded.As an example rather than restriction, communication medium comprises the wired media that connects such as cable network or direct cable, and such as the wireless medium of sound, radio frequency (RF), infrared ray and other wireless medium.Above-mentioned any combination also should be included within the scope of computer-readable media.
Described system storage 130 comprises the computer storage media may of volatibility and/or nonvolatile memory form, such as ROM (read-only memory) (ROM) 131 and random-access memory (ram) 132.Comprise such as at the basic input/output 133 (BIOS) that helps the basic routine of transmission information between the starting period between the parts in computing machine 110, typically be stored among the ROM 131.But RAM 132 typically comprises data and/or program module processing unit 120 immediate accesses and/or that can be carried out by processing unit 120.As an example rather than the restriction, Fig. 1 has illustrated operating system 134, application program 135, other program module 136 and routine data 137.
Described computing machine 110 can comprise that also other is detachable/computer storage media may of non-removable, volatile, nonvolatile.Only as an example, Fig. 1 has illustrated the hard disk drive 141 that reads and write, the disc driver 151 that reads and write and the CD drive 155 that reads and write from the dismountable non-volatile CD 156 such as CD ROM or other optical medium from dismountable non-volatile disk 152 from non-removable non-volatile magnetic medium.Other detachable/computer storage media may non-removable, volatile, nonvolatile that can be used for described exemplary operation environment includes but not limited to tape cassete, flash-storing card, universal disc, digital recording band, solid-state RAM, solid-state ROM or the like.Described hard disk drive 141 typically is connected to described system bus 121 by the non-removable memory interface such as interface 140, and disc driver 151 and CD drive 155 typically are connected to described system bus 121 by the dismountable memory interface such as interface 150.
Driver above-mentioned and illustrated in fig. 1 and relevant computer storage media may thereof provide the storage of computer-readable instruction, data structure, program module and other data for computing machine 110.For example in Fig. 1, hard disk drive 141 is illustrated as storage operating system 144, application program 145, other program module 146 and routine data 147.Notice that these parts can be identical or different with operating system 134, application program 135, other program module 136 and routine data 137.Here they are different copies at least with explanation operating system 144, application program 145, other program module 146 and routine data 147 to have been provided different digital.The user can be by such as tablet (electronic digitizer) 164, microphone 163, keyboard 162 and often be called that input equipment the positioning equipment of mouse, trace ball or touch pad will be ordered and information is input in the computing machine 20.Other input equipment (not shown) can comprise operating rod, game paddle, satellite retroreflector, scanner or the like.Usually by the user's input interface 160 that is coupled to described system bus these and other input equipment is connected to described processing unit 120, but can be connected with bus structure by other interface such as parallel port, game port or USB (universal serial bus) (USB).Also the display device of monitor 191 or other type is connected to described system bus 121 by the interface such as video interface 190.Described monitor 191 can also be with the digitizing such as hand-written input can be input to touch screen display 193 in the computing machine 110 etc. by the interface such as touch screen interface 192 integrated.Notice that described monitor and/or touch screen display can physically be coupled to a shell, in this shell, combine described computing equipment 110, such as in the personal computer of tablet type, wherein said touch screen display 193 is served as described tablet 164 in fact.In addition, the computing machine such as computing equipment 110 also can comprise the output peripheral equipment such as loudspeaker 195 and printer 196 that can connect by outside output interface 194 grades.
Described computing machine 110 can move in the networked environment that the logic that uses one or more remote computers such as remote computer 180 connects.Described remote computer 180 can be personal computer, server, router, network PC, peer device or other common network node, though and in Fig. 1, only showing memory storage device 181, it can typically comprise above-mentioned many or all parts that are associated with computing machine 110.The logic of describing among Fig. 1 connects and comprises Local Area Network 171 and wide area network (WAN) 173, but can also comprise other network.Such networked environment is general in computer network, Intranet and the Internet of office, enterprise-wide.
When being used for the LAN networked environment, computing machine 110 is connected to LAN 171 by network interface or adapter 170.When being used for the WAN networked environment, described computing machine 110 typically comprises modulator-demodular unit 172 or other device that is used for setting up communication on the WAN such as the Internet 173.Can be that described the modulator-demodular unit 172 inner or outside can be connected to system bus 121 by user's input interface 160 or other suitable mechanism.In networked environment, program module or its each several part described about computing machine 110 can be stored in the remote memory storage device.As an example rather than the restriction, Fig. 1 has illustrated the remote application 185 that resides on the memory devices 181.It is exemplary that network shown in will appreciate that connects, and can use other means of setting up communication link between computing machine.
The media integration layer structure
One aspect of the present invention is generally at the more ability that is present in the graphic hardware in the typical computer system with lever control.For this reason, as among Fig. 2 present like that, provided media integration layer structure 200.Media integration layer structure 200 as described in application program, control or other similar higher-layer programs code (as the user interface of operate system components) 202 are visited by one group of application programming interface (API) 204 etc. is with access (write or read) graphical information.Will be referred to the application program with api interface though notice many examples described herein, understand other higher level program code and parts (as the user interface of operating system) also can with low layer unit interface described herein.Similarly, any no matter whether the quoting of such higher level program code be called as application program, user interface etc., all should think of equal value.
Should be noted that described media integration layer 200 (its output pattern) best incorporated is in operating system owing to comprise a variety of causes of security.For example, be feasible though allow some or part of media integrated layer 200 to be inserted between application program and the operating system, do like this and will make rogue program can show any figure that it is wanted, thereby and cause endangering.For example, malicious code may show the dialog box that request is entered password, thereby steals user's password.Other reason that described media integration layer 200 is incorporated in the operating system comprises stability and validity, and for example, lower level can be trusted the data and the instruction of crossing checking already from higher level effectively.In addition, lower level can expose and have only interface that operating system trusts to call reliably, not that is to say those interfaces are exposed to unpredictalbe program, thereby has guaranteed bigger stability.
In one embodiment, described media integration layer structure 200 comprises high level synthetic and animation engine 206, timing and animation parts 208 and the synthetic animation engine 210 that reaches of low layer.As used herein term " high level " and " low layer " are similar to those those terms that are used for other calculated case like that, and wherein in general, software part is low more with respect to higher parts, and described parts and hardware are approaching more.Thereby, for example, can receive from the synthetic graphical information that reaches animation engine 206 of high level at synthetic animation engine 210 places that reach of low layer, wherein use described information to come to the graphics subsystem transmission graph data that comprises hardware.
As described below such, described high level synthesizes and animation engine (being also referred to as high-rise compositor and cartoon making device or high-rise engine or parts here) 206 structure demonstration trees are represented the graphic scene that application program 202 provides, and described timing and furnishing an explanation property of animation parts (or other) animation and timing controlled.Same as described below such, synthetic and the animation engine (being also referred to as low layer compositor and cartoon making device or low layer engine or parts here) 210 of low layer comprises scene painted of a plurality of application programs, and have the painted parts that are also referred to as tinter, on screen, realize the actual painted of figure.Yet, notice that it may be that be necessary and/or favourable some taking place painted in higher level sometimes.For example, when lower level service during, illustration higher level on the basis of every application program, thereby may carry out the painted of time-consuming or application-specific, and transmit the bitmap reference to lower level in higher level from a plurality of application requests.
In general, the synthetic and animation engine 206 of described high level makes up display structures and travels through described structure, and generation will pass to the painted instruction of the synthetic and animation engine 210 of low layer and simple animation at interval.The painted instruction that is produced by described high-rise compositor can comprise regularly and animation information.Synthetic and the animation engine 210 of described low layer is obtained described painted instruction and animation at interval, and the management cartoon making, painted and synthetic subsequently will be to graphics subsystem (as graphics software and hardware) 212 scenes that provide.
On the other hand or except local repressentation output, the synthetic available suitable form of animation engine 206 (or parts similar to it) that reaches of described high level provides painted and animation instructions to the low layer print code 220 that is used for to transmission still image data such as printers 222, and/or available suitable form provides painted instruction and simple animation at interval to the low layer terminal transmission server 226 that is used for to remote computer 228 transmission.Notice and to cross the abundanter information of network delivery, for example can require to make remote computer to handle the mouse rollovers effect in this locality and do not have any Internet traffic.
The multiple graphs processing layer
According to an aspect of the present invention, described media integration layer structure 200 so is divided into multilayer with graphics process.Each of these layers is carried out some intelligent graphic processing, and they allow 202 outputs such as application program, user interface to have the figure of smoothly moving picture together, and the figure and the frame of video of described figure and other application program are synthesized.Also can make described animation and/or synthetic synchronous with audio frequency output.For example, synchronous by the frame rate that makes audio frequency and described low layer parts place, the timing of audio frequency can be basically accurately meets with the timing of video or figure, and do not rely on plan target, ability that refresh rate is caught up with in complicated pre-service.
As ordinary representation among Fig. 3, when by API 204 communications, under application program 202, described high-rise compositor and animate device engine 206 be in tree construction 300 high speed buffer memory application program image graphic data, with the described data of intelligent manner pre-service and carry out many other computings (following) so that the output of complex figure.In general, described high-rise compositor and animate device engine 206 carry out complex process (being sometimes referred to as compiling), and this has simplified treatment capacity significantly and has reduced lower level significantly provides correct output and the data volume of needs processing.Yet, notice the amount of the processing of being undertaken by higher level and load, configuration and the performance that type can depend on lower level to a large extent.For example, if there is high performance graphic hardware, then higher level can be carried out smaller amounts of process, and vice versa.Described high level and low layer are adapted to these factors.
Consistent with the present invention, the synthetic animation engine 206 that reaches of described high level can be finished such complex process and not bury the available system resource, because it is to turn round than following layer or the relatively low speed of some layers.As an example rather than restriction, described lower level can be by the running of frame (refreshing) speed of hardware graphics processor.For example, described high-rise compositor and animate device 206 are only when need (as per half second) realizing showing variation on the basis of request or on other time schedule basis.Though notice and described single high-rise compositor and animate device engine 206 among Fig. 3, but a plurality of their examples can be arranged, such as one of each application program, and each graphics device typically only has the synthetic and animation engine 210 of a low layer, as one on each graphic hardware card on the machine.
And described high-rise compositor and animate device 206 can be revised its form that is output as suitable layer below (or be designed to output) or some layers (as essence what the abstract equipment 302 of taking up an official post).For example, high-rise compositor and animate device 206 can produce the output through compiling, this output finally is used for printer, be used on network, being sent to some remote terminals to show thereon, perhaps as here as described in inciting somebody to action mainly, be used to be present in lower level compositor and animate device 210 on local graphics software and the hardware 212.Single high-rise compositor and animate device are handled the output of the application program of a plurality of abstract equipment, perhaps can have the example of a suitable high-rise compositor and cartoon making device to come abstract equipment (for example one be used for local figure, one be used for printer and one and be used for terminal server) for each type to handle the output of application program.
In addition, can simplify performance and requirement that the order that provided by described high-rise compositor and animate device 206 and other data are mated hardware, for example hardware is few more, and required high-rise pre-service is many more.Further, for example high-rise pretreated amount can be dynamic, so that be adapted to the processing demands of the variation on lower level or some layers.
For local figure output, comprise high-rise compositor and animate device 206, low layer compositor and animate device 210 in media integration layer structure 200 described in the configuration.As described below, in general described high-rise compositor and animate device carry out complex process to make up graphic structure and these Structure Conversion are become graph command stream to the graphical information that receives from CLIENT PROGRAM (as application program).Described then lower level engine 210 uses these graph command streams from different CLIENT PROGRAM to constitute the desktop that the computer user watches, and for example described lower level compositor constitutes desktop by the graph command that will be combined into by the consumption of figure Compositing Engine by the command stream that the various CLIENT PROGRAM that are present on the desktop are sent.
In this was implemented, described high-rise compositor and animate device 206 were carried out the complex process computings, to make up and to convert described structure 300 to graph command stream with the general much lower speed of hardware refresh speed than the graphic hardware of graphics subsystem 212.Because this high level pre-service, described low layer engine 210 can carry out its oneself processing computing at interval in the hardware refresh of the graphic hardware of graphics subsystem 212.Yet as mentioned above, described low layer engine 210 can be on Return Channel and described high-rise engine 206 reverse link communications, makes described high-rise pre-service can dynamically be adapted to the demand that described low layer is handled.Notice that Return Channel from described low layer compositor and animate device 210 to high-rise compositor and animate device 206 is mainly used in to high-rise engine 206 and transmit current control (low layer engine 210 signalings, its needs more multidata or is just receiving too many data) and/or be used for the error condition that high-rise engine 206 can act on.An advantage of such communication is that the synthetic and animation engine 210 of low layer does not need to relate to priority or scheduling, but can keep with refresh rate synchronously.On the contrary, the high-rise cpu process scheduling that is present in already in the contemporary operating system will be controlled priority.Therefore for example, share if program process attempts to have got many high-rise pretreated of it with respect to its priority, this application program will be adversely affected in its output so.Yet, notice that when described lower-level system is in heavy duty it can select the variation of a process/high-level components and demand are classified as preferential above other.For example, can give foreground application priority.
High-rise compositor and animate device
In one embodiment, comprise that the media integration layer 200 of high-rise compositor and animate device 206 is adapted to the difference of the hardware on the given machine, because can not in fact be written as each user application the many types and the variation of processing graphics hardware.Yet the improved graphics process that provided by media integration layer 200 also can be provided application program, just by providing the information that passes to the figure API more (and different) of operating system than current to high-rise compositor and animate device 206.For example, can provide different data to the application program of perceiving media integration layer 200, comprise animation intention or the like by media integration layer API 202.As an example, application program can provide the instruction that should how to move (as with respect to fixed background) about specific image in time, rather than finishes animation by repainting the image that changes a little continuously.Then as following usually described, described media integration layer 200 is handled animation in the mode of level and smooth drafting.
In general, as represented in Fig. 3 and 4, application program 202 makes up the scene graphic data structure by API 204.Described data comprise high-level structure and primitive data, and are converted into and are used for the cached data structure 300 of high-speed cache visual pattern information intelligently.
Be container one of in the object (or structure) in whole intelligent cache data structures 300, in Fig. 4 with container 402,404 or 408 expressions (being called 2 dimension visions on the other hand).In one implementation, container (as 404) provides homogeneity, and promptly application program can keep the handle to it, and comprises that the parameter on the program, the parameter on the described program can be used for hook interlock picture and molding, choose test and user data.Yet, notice that the container of expression is not only Container Type that can expose here.Other example can be included as and be stored in the paragraph row or the container for optimizing in many child node storages and the grid.Though the container of some type may not allow the random access to child node, can add or remove sub-container and do not remove the tabulation of current son.As required can the adaptive structure that exposes by API.
Other of this data structure (inside) node comprises conversion 406, alphabetical node, cache nodes and primitive node 410,412, and they are used to store the internal data that directly is not associated with the API container.Generally primitive is stored as instruction stream, can directly be passed to graphics device to described instruction stream.
As represented in the tree segment 500 of Fig. 5, the container such as 510 can so keep other container 512 or drawing primitive 516, wherein can be thinking graphics command stream at the primitive of any container storage.Container also can be stored other container, in fact creates figure, and promptly container can make that described data structure is the directed acyclic graph (DAG) and the primitive tabulation (wherein the neither one container can comprise one of its father's container) of container by more than container visit.Same as represented among Fig. 5, visit other tree by allowing tree in the mode that is similar to figure, can reuse any container of the container 518 in all tree segments as described 502 in different positions and with different parameters.
Represented as in the drawing context 416 of Fig. 4, by opening/closing modal sets packaging container.Especially described high layer identification code 202 is opened the container 408 in the data structure, provides drawing context 416 drawing primitive being write and/or other container is added in the described data structure, and closes described container 408 then.Select fully in the realization one, when closing containers, its data are placed in the conversion formation 418 of a certain moment application after a while.The opening and closing of described container are to be used to one of main mechanism that changes data structure.Notice and to use other application model, in particular for dissimilar containers.
Select fully in the realization at this, owing to will place formation, just started and be used for the more system that is similar to issued transaction (or being similar to batch processing) of new data structure the change of data structure.As a result, when opening and write container, on described container closure screen, just present variation.The variation of described screen is small and does not have the temporary transient artifact (being also referred to as structural picture break-up) of the screen of part drafting.In addition, can expand such transaction behavior, the feasible change that once applies a plurality of containers.Like this, described higher level code 202 can be provided with many changes of scene and can once apply all these changes.
Select in the realization fully one, change by disclosing to formation 418 through display manager 420, carry out the change of data structure asynchronously, make and on painted thread 422, to handle described change, and for example send it to low layer compositor and animate device 210 (wherein the abstract equipment 302 of Fig. 3 comprises abstractly, and the conversion encapsulation of the described abstract painted order that will be sent by high-rise compositor 206 becomes to flow to the painted order of low layer compositor 210).The described model that is similar to issued transaction can also carry out the modification of data structure is not disturbed from this data structure and read.Though above-mentioned queuing model makes from the reading to transmit and can not rely on the action that Any user takes and move of high-rise engine 206, user application needs high-speed cache to keep the consistent pattern of API, and this may cause poor efficiency.As an example, think that the user application on the main user thread is provided with character (object in the high-rise engine 206) on container.In queuing model, this character is placed in the formation that will be used by high-rise engine 206 data structures.Yet, if user application attempts to read back from container immediately this character, then system will need according to current what the described character (this is inefficient) of reading back in formation, with painted thread synchronization and in formation, apply variation co-pending (this is inefficient and will will negate the benefit with formation), perhaps in container, keep the copy (colored version and version co-pending) (this be poor efficiency makes memory-aided) of user's variable data.
Owing to may have suitable amount by reading back of application program, select embodiment fully and eliminated formation by described high-rise engine 206 data structures of synchronous renewal and described main user thread.Though it is painted that this can freeze user application, total system is more effective.Yet, may freeze the influence discovered in order to relax, the various piece of described animation and timing system can be moved independently so that information is propagated into low layer engine 210 downwards, and entrusts described low layer engine 210 not rely on the more animation process of high-rise engine 206.Then, if owing to user action has freezed described high-rise engine 206, the output of screen will be more level and smooth relatively and consistent.
Another embodiment eliminates described painted thread, and makes described main user thread carry out the required any processing of high-rise engine, so that painted instruction is passed to low layer engine 210.In some cases, this is the more effective use to thread.
Get back to Fig. 4, container 408 comprises basic coequal nodes, this node comprises drawing primitive, and described drawing context 416 comprises the graphic generator (as the help program object) that obtains from container, and it can be used for adding primitive, conversion, montage or other mapping operation to container.Display manager 420 comprises main object, and it represents the example of described high-rise compositor and animate device 206, and for example can depend on hwnd (window handle) or hvisual (visual container handle).Described display manager 420 has the pointer of the root container 402 that points to scene, when container is invalid when maybe needing to repaint with event scheduling to high layer identification code, and provides the visit to the service such as choosing test and coordinate transform.
Though described higher level code 202 can remain on handle of some object or the like in data structure or the container, the most of objects in the container do not possess the homogeneity from the perspective of application program.Especially the visit of this structure being restricted to most of use patterns is " only writing ".By limiting homogeneity by this way, can the more information of optimal Storage in this data structure, and the essential storage object information of described higher level code 202 or life cycle of handling management object.
For example, have as after need to ask to repaint the application program of scene, be recoverable to and keep the unwanted part figure resource of (as corresponding to by the visual information outside the montage or the screen that rolls out).Thereby usually when container was opened, its content just was eliminated and ignores.If those contents do not have homogeneity, they can disappear safely so, make the recoverable resource that is used for them of system.If some other parts of higher level code 202 or figure remain on the sub-container, also can be inserted again around then those containers are fixed on.Yet, can change and adapt to this pattern according to the needs of higher level code 202.
Thereby generally speaking, described container is the object with homogeneity, promptly uses the high layer identification code of data structure can keep the handle of this object.Object reverse side with homogeneity is a clear data, and when using the thinking model that the data that do not have homogeneity are treated as object, in case these data are submitted to system, just has no idea to quote this object after a while when subscriber-coded.So, can change and change object with the mode of being convenient to system.
Example as simplifying is used for the capable api function of rendering text and can comprises the line of text object.The actual text that the user of this object will draw is loaded described line of text object together with the information of the different distances of swimming (font, size, paintbrush etc.) of other relevant how painted text.When the user program code was thought in fact to be added to this article one's own profession in the data structure, described program code can be regarded rendering text as a certain specific unlatching node, and described line of text object is passed to the drafting function of drawing in the context.In fact described system obtains the data in this article one's own profession object, and these data are replicated in the described data structure.Because these data do not have homogeneity, described high-rise compositor and animate device 206 can freely be obtained the content of this row, operation algorithm (as OpenType), text being resolved into the symbol with position, and storage determines that the symbol data of position replaces urtext.After having drawn this row, system will not have draws quoting of capable line of text object to being used to, and promptly the data that system stored do not have any homogeneity.
On the other hand, described higher level coding 202 can ask to preserve homogeneity in this article one's own profession object, requires storage quoting this object in data structure.So, if described higher level code 202 has changed described line of text object after a while, then system will find this variation and this variation of reflection in the output of treated expression.Notice in than real example, homogeneity will not be exposed on the line of text object self, but application program will keep the handle of container, and change with this container of parametric representation on demand, described in " Generic Parameterization for a Scene Graph " U.S. Patent application by name as the aforementioned.Yet a main aspect of described data structure is to reduce higher level code 202 to create this needs with object of homogeneity, thereby the quantity of control coding 202 points of quoting in data structure reduces.This more optimizes data structure.
For example, owing to be exposed to the minimizing of the amount of the homogeneity outside the described data structure, just allow the optimization such as the intensive storage of primitive.For this reason, vector figure data is stored in " primitive tabulation " or the primitive container.These containers are to implement appointment, and are not exposed to described high layer identification code 202 with homogeneity.When calling program write container with data, this data or be stored in the object that separates that chain follows were similar to container (as linking together with conversion), perhaps can flow in encapsulation and the flat data array.The mode that this array not only can compress is stored vector figure data, can also follow the tracks of the resource of following these primitive.Because single primitive does not have homogeneity, does not need to tell primitive or provides a kind of method to change those primitive after a while to the user, allows more effectively to store described primitive.
As another kind optimization, when spirte is not changing, just may store the bitmap of the content of this tree, and make described bitmap depend on container, thereby reduce needed high-rise treatment capacity.In addition, when the part of subgraph or primitive tabulation can be sent to it lower level code be used for painted before and when requiring effectively to handle, (as inlaying of the vector graphics before switching to hardware device), can high-speed cache through the result of aftertreatment for heavily using after a while.
And because except concrete read operation (following), described structure does not have exposure, it is identical needing only through painted result, and so described data structure can freely reorganize container.Therefore, container can be stored sub-container in the tree of spatial division, optimizes painted and other read operation.In addition, can on identical equipment or a plurality of equipment, repeatedly show described data structure.For this reason, can whether be the described high-speed cache of locking that relies on according to equipment.If think that subtree is static, normal can guarantee resource consumption because its animation on every side repaints and it is enough intensive to be, can automatically insert cache nodes to this subtree so.
For painted, read described data structure (perhaps in a certain schedule time or by different threads) and the treated information of certain form is passed to lower level animate device and compositor 210.For this purpose, select fully in the realization one, the described data structure 300 of colored object and thread (each process) 422 traversals drives painted process.In another was selected fully, in the configuration of " cooperation multitask " type, described painted process can be on common thread be shared the time with remaining personal code work, rather than only moved on its oneself thread.Though preferably described data structure 300 is compiled into the visual information that is supplied to the lower level parts, be used for synthetic very fast and animate, described data structure 300 can be used for directly painted.Can also different modes compile described data structure 300, as pass network and send to remote terminal, printer and/or be serialized into disc or some other more nonvolatil medium, be used for exchange or high-speed cache.
Select fully in the realization at one, on another thread 422, read described data structure 300 and be used for painted.Yet, should be noted that the use to another thread is not requirement, for example " painted thread " selectively is included in the shared mechanism of the cooperation that moves on the thread identical with other each thread.
In the alternative model that uses painted process/thread, described painted thread moves as required so that predetermined effect to be provided.When each described thread moved, it at first used the variation any co-pending in the described conversion formation 418.Described then painted thread 422 is travelled described data structure 300 with the information of collection such as bounding box, and collects invalid information (following).At last, it travels that those changed since last time or owing to some other reason needs painted zone, and carries out the painted instruction that is stored in the described data structure.Notice in the described alternative model that does not use the conversion formation, make when changing directly applying variation when, therefore do not need here to apply.
Thereby, from the painted of described data structure 300 is to handle at the multiple path of painted thread 422 operations that separate, comprise the change application of the queuing that will do to the path of data structure, the path that carries out precomputation and painted path, described precomputation comprises the described data structure of iteration and calculates such as painted required data such as bounding box, movement parameter values.It is painted that described painted path uses abstract equipment 302 to carry out, and this entrusts to low layer compositor and animate device 210 the most at last.During described painted path, cacheable resource 426 through intermediate cache is so that the colorability on the improvement frame subsequently.
The possible outcome of travelling for the last time of described data structure comprises, directly carry out and on display screen video data, or on the reverse buffer that described end of travelling for the last time triggers, carry out described data.Other result comprise with the timing of expansion and animation information (as following) with as described in data acquisition be delivered to painted thread/process with the operation of faster frequency together and downwards.Described travelling also can cause the data of carrying out on bitmap owing to screen capture or other reason being sent to printer or passing network and transmit, and for above-mentioned reasons any and using on remote machine then.These results' combination also is possible.
As can be appreciated, the storage data may require a large amount of storeies in described data structure 300.In addition because owing to montage, roll or that other reason causes is invisible, the many data in the data structure 300 may be unwanted.In order to reduce resource requirement, can make up described data structure 300 on request.In order to allow like this, provide a kind of method that is used to adjust back described higher level code 202 with the each several part of creating data structure 300 as required.This method has been called as engineering noise, and is similar to the WM PAINT callback method that is used for conventional graphics system, but is applied to described structure 300 and through the content of high-speed cache rather than directly apply to position on the screen.Yet, to select fully in the queuing model at one, read operation (being similar to following choosing tests and coordinate transform) is application change at first, makes that the described user's of presenting to model is synchronous.
When because the reason system of low-resource directly asks container when invalid when abandoning content or when described higher level code, when setting up container, just can make them invalid.For example, described higher level code can be set up container and feature size is provided, and where and much described feature size defines this container.During shading operations, invalid if this container is marked as, but to be confirmed as now be needed, so described painted thread 422 will require described higher level code 202 to fill described container.Described painted thread 422 can wait for that described higher level code 202 finishes described request, and perhaps continuing does not have the described painted of desired data.Described the 1st selection is not desirable, but is necessary in some cases.
When having filled data, described painted thread 422 will move once more to show those new variations when final.In a current realization, the request of filling containers is placed another formation, to come back to the thread of the described higher level code 202 of operation.Yet this can carry out in other mode, comprises the synchronization call to the higher level code 202 on the painted same thread of operation.Yet, carry out any such invoke synchronous and will stop described painted thread.
Except the renewal of queuing, need read back for service provides from described data structure 300 to data structure 300.Such service comprises chooses test, some conversion and subgraph size to adjust.
Choosing test is a kind of processing, provides a little in the coordinate space of certain root of data structure whereby, and surveys described data structure, makes and returns container or the primitive of being chosen by this point.In current realization, the described value control (though extra sign also is feasible) of choosing test processes by 3 signs storing on each container.The 1st sign comprises a setting, and this is provided with indication and chooses testing algorithm to stop and returning the collected so far test result of choosing.The 2nd sign comprises a setting, if selected point has been chosen container really, this setting just informs that the described testing algorithm of choosing comprises this container in the results list.Whether the 3rd sign control should choose the child of this container of test once more.
Another service of reading is a conversion, wherein provides two nodes that connect by figure, has a service routine, can be transformed into the point in the frame of axes of a container frame of axes of another container whereby.3 kinds of general subtypes are arranged, comprise from ancestors being transformed into the offspring, being transformed into ancestors and being transformed into ad eundem (taking office what its arbitrary node) from arbitrary node from ad eundem from the offspring.Thereby described read to serve a kind of data query structural approach that is used for coordinate transform is provided, and the described tree construction of balance is to travel and to calculate described conversion.Lockable animation/variation when carrying out multiple transform, and can provide by public ancestors and carry out conversion.
Another service of reading is that the subgraph size is adjusted.A given node, the feature size of this node and subgraph thereof is returned in this service.Described size can be represented with following form, guarantees enough can comprise greatly the size of subgraph, and some may vary in size, and just in time is large enough to comprise subgraph, or describes the more complicated shape of subgraph profile in detail.
A kind ofly realize wishing that the variation of these read operations and data structure is synchronous.For this reason, if before calling any of these read operation, used described conversion formation, just present to the more consistent pattern of higher level code.
Low layer compositor and animate device
The fundamental purpose of low layer animate device and compositor engine 210 provides the painted storehouse of low layer abstract of media integration layer 200, this considers the high frame rate animation of (1) client's graphics application program, (2) realization of similar window management support, and (3) support the remote graphics service of connection Network Based.As Fig. 6 and 7 represented, described low layer cartoon and Compositing Engine 210 serve as server, especially the service by using tinter collection 602 to be provided, the high frame rate animation request of coordinating to be received (as using high-rise compositor separately corresponding) with a plurality of application programs from a plurality of CLIENT PROGRAM.Described tinter 602 produces the painted action that acts on painted abstract (being also referred to as visual pattern) of being realized by figure Compositing Engine 606.
Described low layer cartoon and Compositing Engine 210 also provide the top layer vision management that comprises foundation structure to support, described foundation structure purpose is to allow special CLIENT PROGRAM (top layer vision manager 604) to handle the employed visual pattern of client applications as the painted areas on the screen.To each painted client applications 202 of painted storehouse 1-202 3(only illustrate 3 among Fig. 7, but can be any amount) has the top layer vision (is respectively TLV 1-TLV 3), and described top layer vision manager 604 is the CLIENT PROGRAM of right with layout of the top layer visual pattern on the management screen.In general, the command stream that described low layer cartoon and Compositing Engine 210 send by the various CLIENT PROGRAM that will be present on the desktop is combined into the graph command that is consumed by described figure Compositing Engine 606, constitutes desktop.Described low layer cartoon and Compositing Engine 210 also help to use its parts to realize making remote machine 610 1-610 nThe programmed user interface painted framework identical with local machine.
Fig. 7 show between low layer cartoon and Compositing Engine (server) 210 and its CLIENT PROGRAM alternately.As mentioned above, described top layer vision manager 604 also is a CLIENT PROGRAM.Similarly, as described above, the CLIENT PROGRAM 202 of described low layer cartoon and Compositing Engine 210 1-202 3Use the example of high-rise compositor and animate device engine 206, make up graphic structure and convert these graph command stream of low layer cartoon and Compositing Engine 210 uses to, to constitute the desktop of being seen.In one embodiment, a low layer cartoon and Compositing Engine 210 are only arranged, the command stream of handling or sending in local machine or the CLIENT PROGRAM (as high-rise compositor) on remote machine, moved.
Return Fig. 6, can carry out the communication between process by the character system that keeps by described low layer cartoon and Compositing Engine (server) 210.Store the character that is associated with each top layer vision by this character system.But the CLIENT PROGRAM write or read is got these character, but and when request changes character notification client.
Described low layer cartoon and Compositing Engine 210 provide client-server communication, the painted and top layer vision management support of animation fast.In one implementation, communicating by letter between low layer cartoon and Compositing Engine 210 and the CLIENT PROGRAM taken place by single two-way byte stream and/or shared storer.For described byte stream, local client uses interprocess communication, and remote client's program is opened the connection of network bytes stream.Described byte stream is the transport layer of the communication protocol of control client server interconnection.
Described communication protocol comprises 3 main message, promptly asks, answer and event message.Error message also is provided.As described below, the client mainly comprises painted instruction to server communication, and server to client communication mainly is the feedback with the form of response and error message.
Generate request by the client to server, and described request can comprise the administration order of top layer vision, painted designated command and fixed time interval order.Can send answer to the client by server and respond described request.Yet, should be noted that the request that is not with answering reduction into possession; Only produce answer in response to the suitable request of search information.For example, drawing instruction does not need to answer.Yet " obtaining window size " request needs and receives to answer.
Send incident from server to the client, and described incident comprises about equipment action or about the information of the spinoff of last request.For example, because resource is invalid, server can propagate into the client with event message, and notifies the client target frame rate.The ability that can transmit target frame rate allows variable frame rate, and this is desirable, because it has guaranteed constant frame rate rather than high frame rate.
Also mistake can be sent to the client.Mistake is similar to incident, but it generally handle in a different manner by the client, as the compensation mistake.
Before CLIENT PROGRAM can be used the service that low layer cartoon and Compositing Engine 210 provided, described CLIENT PROGRAM is being connected by the entrance foundation that provided by connection manager 710 (Fig. 7) and described engine 210 at first.Described connection manager 710 distributes the communication object of the two-way byte stream transport layer that encapsulates client/server protocol (as 712 1).It goes back the distribution instruction list manager (as 714 1), described instruction list manager keeps following the tracks of to the painted instruction that comes with instruction stream, and they are associated with correct visual pattern.
In case set up connection, the top layer vision is created in CLIENT PROGRAM 202 requests.In response to this request, described low layer cartoon and Compositing Engine 210 are created described top layer vision (as TLV by the service of using figure Compositing Engine 606 to be provided 1).Being organized in the tree structure 718 to the visual pattern that its CLIENT PROGRAM kept by described low layer cartoon and Compositing Engine 210.When CLIENT PROGRAM was finished the top layer vision, it just asked its elimination.Notice that root node 710 is special visual patterns of representing desktop background, and the child node of described visual pattern is the top layer visual pattern.
As represented among Fig. 4, a vital role of described low layer cartoon and Compositing Engine 210 is painted to computer desktop of management, this is by realizing that by the service that relies on two other parts described two other parts are figure Compositing Engine 606 and tinter collection 602.Described figure Compositing Engine 606 provides low layer synthetic service by being called as the painted abstract of visual pattern.Visual pattern is the painted areas that is used to form the rectangle of desktop, and can be undertaken painted by one group of API that described figure Compositing Engine 606 provides.Form desktop when time when having arrived, painted transmission manager 722 travels through described tree from left to right and is using colouring component to carry out painted to visual pattern for each node.
Except the lifetime management of top layer vision, described low layer cartoon and Compositing Engine 210 also support the top layer vision to decorate, and come down to add around described top layer visual pattern to modify.Decorate 730 1-730 3It is the visual pattern of drawing the painted modification that described top layer vision manager provides with the form of painted instruction list.These visual patterns are children of top layer visual pattern, and these children belong to described top layer visual pattern.Described CLIENT PROGRAM (as application program) can by as described in predetermined character be set on the top layer visual pattern control the modification that top layer vision manager is provided.
Described low layer cartoon and Compositing Engine 210 are also supported the service of top layer vision is minimized, and this also can be provided according to painted instruction list by described top layer vision manager 604.Support the framing of top layer vision, size to adjust and the ordering of Z axle, and move the specified three-dimensional effect for close and minimize the vision manager that waits as visual pattern.Thereby, though related generally to 2 the dimension pattern descriptions realization, native system can be advantageously used in storing the medium of other type, comprise 3 the dimension figure, video and audio frequency.
As mentioned above, the required painted instruction list of described top layer vision manager is produced by high-rise animate device and compositor 206.One group of top layer visual pattern action of described low layer cartoon and Compositing Engine 210 definition with default behavior, as minimize or close.If described top-level visual wishes that apart from manager 604 it just uses high-rise API to make up the description of the action of its desired replacement from customizing such behavior.Then, it sends instruction stream to described low layer cartoon and Compositing Engine 210.Described low layer cartoon and Compositing Engine 210 are stored this and are described in its character system 614, and use this description when the specified action of client requests.
Modify by using described character system 614 to carry out the top layer vision.When starting, described top layer vision manager sends the instruction list that high-rise engine produces, and describes top layer vision manager and modifies.By described character system 614, these modifications are upgraded, promptly to wish to upgrade when modifying when CLIENT PROGRAM, the character of this CLIENT PROGRAM appointment is set to desirable value.Described then low layer cartoon and Compositing Engine 210 have changed character notifies described top layer vision manager 604.In response, described top layer vision manager 604 reads described character and upgrades geometric figure on described low layer cartoon and the Compositing Engine 210.
What further describe in asking in the United States Patent (USP) of " Generic Parameterization for a Scene Graph " by name as described above is such, make described instruction list parametrization, this means that generally described top layer vision manager 604 does not need to relate to simple variation, as the color of correction pattern image.Under these circumstances, CLIENT PROGRAM changes the new parameter of downward transmission (as new color) into, and with identical instruction list but use different parameters that described modification is restained.This also provides and has only stored the ability that each modifies a copy of describing.
Fig. 8 shows the logical organization of the formation 801-804 that realizes described client-server communication channel.In the painted instruction of activity, embedded fixed time interval.At place of painted time, described low layer cartoon and Compositing Engine 210 pass to tinter to the current time together with fixed time interval.Then, as described below, described tinter uses described fixed time interval to come interpolation to be used for painted correct parameter.In response to the designated command that is received from high-rise CLIENT PROGRAM, the painted instruction of the described activity of instruction list manager 714 management.When receiving described painted instruction, described instruction list manager 714 makes their queuings.Described painted formation is Z axle ordering, and described painted transmission manager 722 the composition time loss they.
Except the painted instruction queuing that makes time control, described instruction list manager 714 is also supported other operation, comprise empty described formation 801-804, from described formation, remove instruction, to described formation insert instruction, with new instruction list replacement formation and with regular time offset applications in described formation.Being subjected to time-controlled painted special circumstances is when only controlling the observability of painted instruction.In such incident, described fixed time interval can be used for controlling the life cycle of the instruction in the painted formation.
May have such situation, promptly CLIENT PROGRAM needs nested visual pattern correctly to draw its content, such as when having video in scene.Upgrade owing to carry out video by painted process independently, described low layer engine 210 depends on described figure Compositing Engine, the geometric figure that constitutes described video and cover it.This is to realize by the new visual pattern that establishment is contained in the top layer visual pattern of client applications, described top layer vision image concealing the asynchronous behavior of the video that in the synthetic transmission of described figure Compositing Engine, upgrades.The described covering geometric figure of sharing the visual pattern needs has the α behavior (every pixel or transparent) of same type.
Animation
In general, realize animation together by described high-rise compositor and animate engine 206 and described low layer compositor and animate engine 210.As mentioned above, described media integration layer is divided into a plurality of graphics process layers under user interface or the application layer.Described high-rise engine 206 traversal scenes are upgraded animation parameters with being suitable for after a while the interval of interpolation, and the data structure of these simplification are packaged into the instruction that is passed to described low layer engine 210.This can carry out with synchronous and/or asynchronous mode.Can regard described interval data as and comprise regularly end points (starting and ending timing data), and the parameterized value that is used for described painted instruction.Notice that described high-rise engine 204 can carry out the interpolation that some or all are asked, for example, if to such an extent as to interpolation or other motor function can not be handled for described low layer engine 210 is too complicated, perhaps described low layer engine can not be caught up with processing demands placed on it, so the higher level engine can carry out some or all calculating and to lower level provide simplification data, instruct, inlay or the like, with the result who realizes being wanted.Carry out in the typical case of interpolation at lower level, for each frame of animation, described low layer engine 210 interpolation parameters are at interval obtaining instantaneous value, and decode the instruction into the painted order of being carried out by graphics device.Described graphics device constitutes any final scene of frame of video that may appear in the scene of interpolation.Can also add other data, the content of protecting as digital managing entitlement.
As described below, realize communication between described high-rise engine 206 and the described low layer engine 210 by instruction stream.Described high-rise engine 206 is with its lower frequency, or by request, painted instruction is written in the described stream.Described low layer engine 210 reading command and draw scene from described stream.Notice that described low layer engine 210 also can obtain to draw the required data of scene from other source, as from the bitmap of shared storage etc.
Thereby described high level (as request formula timeslice) engine 210 upgrades animation parameters as few as possible and travels through scene data structure and keep level and smooth animation.The described scene data structure of described high-rise engine 206 traversals is calculated the interval of describing each movement parameter in cycle time, and these are delivered to low layer engine 210 with the parameterized drafting instruction of simplifying at interval.Described supplemental characteristic comprises zero-time, concluding time, interpolater and interpolative data.As an example, described high-rise compositor and animate engine 206 can indicate described low layer compositor and animate engine 210 should how to pass in time about image and change, time quantum (at interval) that image should move as described in as origin coordinates, end coordinate, between coordinate and for example linear movement function, rather than wipe or redraw and imagedly make it look motion.Described low layer compositor and animate engine 210 to determine the new position between the frame, convert interpolation the accessible drafting instruction of to described graphics device with them, and these orders are passed to described graphics device.
Each transmission of described high-rise engine 206 is preferably described low layer engine 210 enough data is provided, to carry out smoothly moving picture on some frames.Can use the shortest interlude length to determine that described high-rise engine 206 keeps the minimum frequency of the required operation of smoothly moving picture.Fully static or only comprise and can only require by the scene of the simple animation of single interval definition, when the change made by calling program 202 scene, described high-rise engine 206 operations.The scene that comprises more complicated animation requires described high level to require 206 with less relatively interval operation, as approximately per half second once, in this scene, can during short period (but still more much bigger), predict also accurately interpolation parameter than hardware refresh speed.At least one parameter can not be predicted the animation of high complexity will require described high-rise engine 206 to move (will be degenerated to the limit of individual layer animation system in fact) more continually up to system.
The frequency of described high-rise engine 206 operations does not need unanimity or fixing.For example, can arrange described high-rise engine 206 to move with the consistent interval that is not more than the minimum interval that animation parameters was provided in the described scene.On the other hand, the described minimum interval of being calculated on the each run of described high-rise engine 206 can be used for arranging operation next time, guarantees and will new data be offered in time described low layer engine 210.Similarly, when described scene and/or its movement parameter are made structural variation, can move the frequency of described high-rise engine 206 and guarantee accurately to make the scene that makes new advances.
Described low layer (as quick timeslice) engine 210 be with as described in high-rise engine 206 task of separating.Described low layer engine 210 receives from the parameterized drafting instruction of the simplification of described high-rise engine 206 and the parameter interval of description scene.These data structures are kept and traveled through to described low layer engine, till high-rise engine 206 provides new data structure.Described low layer engine can be served a plurality of high-rise engines 206, for each high-rise engine provides data structure separately.Relation permission system's while of the one-to-many between described low layer engine 210 and the high-rise engine 206 is a plurality of scenes of cartoon making smoothly.
The interval that described low layer engine 210 provides according to described high-rise engine to the instantaneous animation parameters of each frame interpolation, is upgraded and is drawn instruction and draw scene.In system, move described low layer engine 210 tasks, to guarantee that being ready to frame shows by the graphic hardware screen refresh rate with high priority.Thereby the interpolation of generally described low layer engine 210 being carried out is restricted to simply, function fast, has the function of similar speed as linearity, piecewise linearity, cubic spline and those.Described low layer engine 210 is with regular frequency or frame rate operation, and this speed is the integral factor of described hardware refresh speed.Will be by described graphic hardware with the quantity of unanimity or refresh each frame that shows that described low layer engine 210 is drawn.
High-rise, low-level communication
Turn to the interaction between the described high-rise compositor of explanation and animate engine 206 and described low layer compositor and the animate engine 210, some configurations are feasible, comprise in the process, wherein said low layer engine 210 is only served a high-rise engine 206, and both all are in the same process.The process configuration of intersecting also is feasible, and wherein said low layer engine 210 is still only served a high-rise engine 206, but described high-rise engine 206 and described low layer engine 210 are in independently in the process.
Desktop configure is the expansion of described intersection process configuration, the example service of 210 pairs of many high-rise engines of wherein said low layer engine, and control the painted of whole desktop.When described high-rise engine 206 on server machine, and described low layer engine 210 is on client machine the time, Remote configuration also is feasible.Described remote scenario energy or have one-one relationship between described high-rise engine 206 and described low layer engine 210, perhaps described low layer engine 210 can be served many local and remote high-rise engines.Can regard described Remote configuration as and be very similar to described intersection process and desktop configure, but high-rise engine 206 and low layer engine 210 are on different machines.
Consistent with the present invention, the data of preparing in the described high-rise engine 206 are used for to described low layer engine 210 transmission, and described low layer engine 210 is handled them once receiving described data.Following general description the form of described data, and in described high-rise engine 206, how to produce data and how described low layer engine 210 is used for these data painted.
Fig. 9-12 shows overall interaction and the data stream between high-rise engine 206, low layer engine 210 and the shape equipment, and Figure 13,14 and 15 provides the example of the various data structures that are used to communicate by letter.In general, the painted traversal of high-rise engine (as circulation) the 902 repeatedly iteration by substep travel through scene high-speed cache tree 900 and create data structure 904 (comprising that the visual pattern among Figure 11 upgrades piece 1114).This comprises creates the resource that is used for this scene high-speed cache, and creates the illustrative animation that is used for showing that tree is used.For the value that will change produces animate device object, thereby in media integration layer, use animate device rather than constant to draw API.Create the video data structure 904 of described high-rise engine 206 then, quoted the resource and the animate device of previous establishment.Described video data also can be quoted other subtree.In order to make it to activate, animate device (as 1114 among Figure 11) is connected to regularly tree/node structure (as 1102 among Figure 11).Typically, application program is controlled creation-time node 1102 beginning, is stopped and the process of each animate device 1104.
In case created after the model that comprises timing and animation, the traversal of described high-rise engine circulation 902 will travel through described demonstration tree 900 and begin to draw.As mentioned above, the frequency of described high-rise engine 206 traversals depends on regularly and the amount and the complexity of animation.For example, for static scene, have only when application program is made variation just need travel through to described static scene.
Described high-rise traversal processing 902 takes place in a plurality of transmission (pass).In precomputation was transmitted, the 1st traversal of described tree 900 drawn required calculating.For example, to each subtree computation bound frame, and upgrade animation value.As mentioned above, animate device 1104 can provide one at interval, rather than single instantaneous value.Described interval comprises starting and ending time, the data that will use the interpolater of which kind of type and be used for interpolation.Then the resource of described demonstration tree use being sent to described low layer engine 210 is used for realizing.
The 2nd time the traversal packing will send to the information that low layer engine 210 is used for painted description scene.For this reason, data structure and information shown in Figure 13-15 are represented, with each subtree in one or more instruction block presentation graphs.High-rise engine 206 for each top layer visual pattern (being window) with instruction block (as Figure 15 1500) be accumulated to single visual pattern and upgrade in the piece 1114.Do not added also unaltered instruction block subtree since traveling through last time.The instruction block that only has the subtree of illustrative animation variation will only comprise the variable piece of interpolation information.The instruction block that is changed the subtree of position by variation only comprises head.
The operation in circulation 1208 (Figure 12) of described low layer engine 210 painted threads (or process) with high frame rate, is the refresh rate of display ideally, comes painted and the composition visual pattern.Some steps are carried out in described circulation 1208, comprise that application upgrades piece (as 1114 from any visual pattern that described high-rise engine 206 receives 1-1114 3).Be updated in the resource of drafting in this transmission and the interpolate value of window.For each variable in the variable list of described instruction block, calculate the instantaneous value that is used for next frame.Then, described circulation 1208 is duplicated new value for the enterprising row iteration of skew of the position in its employed instruction list in storer.
Though when first the use, can draw other resource, what drawn is the outer resource of upgrading of screen.Draw the instruction list of each top layer visual pattern, and data " blt-ed " to display, through montage or arrangement, so that on screen, upgrade each top layer visual pattern.
Remote transmission
As mentioned above, described high-rise engine 206 produces the relevant data set that is sent to described low layer engine 210, comprises animation interval/variable resource 1204 1-1204 3(Figure 12), for example image and letter symbol, they are described in the short cycle internal variable and how are used from painted variation together with employed information one, and comprise and describe the instruction list of drawing the desired location of top layer visual pattern (window) shading operations.Instruction list can comprise variable rather than quoting quiescent value.
At first, described high-rise engine 206 establishing resources need be created this resource before use, and be quoted by opaque handle in the instruction list.Described high-rise engine 206 by at first create equipment independently resource representation come establishing resource.Under the situation of image, described expression is full frame bitmap or the coded image to encode such as JPEG.For communication, described high-rise engine 206 is submitted to resource data the communication that unique handle is distributed to described data subsequently and is accepted program (stub).Described handle produces in described high-rise engine 206 processes, and is unique in this process only.
As represented among Figure 16, the data structure 1600 that program is packaged into described resource data on the head with predetermined operation and handle type is accepted in described communication.The described program of accepting turns back to described handle described high-rise engine 206 immediately, is used in painted use.
Instruction list, variable, time value and animation one work at interval.Described instruction list is described described painted and can comprise variable rather than quoting quiescent value.Animation is described the value that how to change variable on short cycle at interval.
Described high-rise engine 206 is collected instruction list and animation interval, as the part of its painted transmission.Described high-rise engine 206 is dressed up the one or more instruction blocks of each top layer visual pattern (window) with described painted packets of information.Each piece is represented the subtree of the figure of certain window.Each piece comprises the variable list that influences described instruction list.The instruction list that described high-rise engine 206 is kept comprises quoting employed variable.Each variable is collected these variable references and converted them in the instruction list position (skew), must before painted, replace described position with the instantaneous value of variable.As Figure 16-22 is represented, packaging together these skews with the animation interval information to this piece.
Figure 15 shows more new data structure of visual pattern.In general, as mentioned above the set for the instruction block of describing window is accumulated visual pattern and upgrade piece.Then with above-mentioned resources-type like mode, described renewal piece is packaged into packet.Upgrade piece and may be huge and, if just in a plurality of packets, send for the more effective words of transmission.
Described low layer engine 210 has the thread of common priority, and it listens to them when described transport stream and process data packet arrival.Preferably on the thread that separates with painted thread, carry out described operation, can both be painted to guarantee refreshing described low layer engine 210 each time.Communication thread is analyzed each packet according to operational code (opcode), high-rise engine identification symbol (ID), object handle.Described these resources and the more processing of new situation of visual pattern with reference to Figure 23.
Especially, recognizing received packet when described low layer engine 210 is (step 2300) when being used for resource 1204, just handles described resource data bag.Then, at step 2302 place, described low layer engine 210 extracts described high-rise engine 206ID and resource handle, and creates the resource of required type at step 2304 place.Described low layer engine 210 is filled the equipment absolute version of described resource at step 2306 place, and described resource is placed the handle table (step 2308) of specified high-rise engine 206.
Handle the individual data bag that sensed image upgrades piece similarly with described resource data bag, but require more the processing.When described low layer engine 210 recognized that described packet is sensed image renewal piece 1114, step 2312 was transferred in described processing at step 2300 place.At step 2312 place, described low layer engine 210 extracts high-rise Engine ID and vision handle from described packet.Then, state low layer engine 210 in step 2314 place and in the handle table of specified high-rise engine 206, search sensed image.If do not find sensed image, just create new visual object, and add it to described handle table at step 2316 place.As represented among Figure 24, by step 2400 and 2416, described low layer engine 210 carries out circular treatment by each instruction block in the described packet then.
For this reason, at step 2400 place, described low layer engine 210 selection instruction pieces, and search piece ID in the current sensed image at step 2402 place.The if block identifier does not exist, and just creates new piece, and uses this piece of data initialization from described packet.If opposite, described block identifier exists, and so described low layer engine 210 upgrades described conversion and skew (step 2406).If described comprises variable list, replace existing variable list by step 2408 and 2410 so.If described comprises instruction list, then replace existing instruction list by step 2412 and 2414.
Handle the situation of a plurality of packets with the mode identical with the situation of described individual data bag.Unique difference is exactly that described low layer engine 210 is at first collected described sensed image renewal piece from a plurality of packets.
As institute can obviously understand,, just reduced the needed data volume of communication between described high-rise engine and low layer engine significantly owing to send graphics command rather than individual other figure position.As a result, can connect and will be sent to remote terminal from the graphics command of high-rise disposal system through network, each of described remote terminal has the low layer disposal system.Even the bandwidth of described connection can not carry conventional graph data, the figure that is produced output also can look substantially the same on some terminals.
Conclusion
As can from above-mentioned detailed description, being found out, provided Hierarchial graph processing system and method, it has simplified the complex process of relatively low frequency, allows at relative upper frequency output complex figure.Described system and method makes the output of improved figure can the relatively large graphics subsystem power of balance by utilizing a plurality of processing layers, described improved figure output comprises smoothly moving picture, figure is synthetic and with the combination of video and audio frequency.And described multilayer has been simplified to the graphics command of the terminal that causes complex figure output and the telecommunication of data.
Though the present invention is easy to carry out various corrections and selects structure fully, some illustrated embodiment shown in the drawings and describing in detail in the above.However, it is to be understood that not to be intended to limit the invention to the concrete form that is disclosed that antithesis, the present invention will be contained all and belong to correction in main idea of the present invention and the scope, selects structure and of equal value the realization fully.

Claims (32)

1. a kind of system in the computing environment is characterized in that, comprising:
Graphics subsystem is used to export the frame of the display message that comprises figure;
The 1st parts, these parts provide graph data with the 1st speed to described graphics subsystem, with the frame of output display message;
The 2nd parts, these parts and program code interface, for described program code produces contextual data, described the 2nd component configuration is become with the 2nd speed that is lower than described the 1st speed described contextual data to be processed into graphical information, and provide described graphical information to described the 1st parts;
The cached data structure, described the 2nd parts remain at least some contextual data in the described cached data structure.
2. the system as claimed in claim 1 is characterized in that described graphical information comprises the pictorial data by described the 1st form that parts are understood.
3. the system as claimed in claim 1 is characterized in that described graphical information comprises instruction.
4. system as claimed in claim 3, it is characterized in that described instruction comprises at least one interpolation order, and wherein, described the 1st parts comprise by as time goes by image interpolation being gone into the interpolation mechanism that graph data changes the graph data of the graphics subsystem that offers.
5. the system as claimed in claim 1 is characterized in that described the 2nd parts access animation parameters changes the graphical information of described the 1st parts that offer.
6. the system as claimed in claim 1 is characterized in that described the 2nd parts running exports to respond the variation in the described contextual data.
7. the system as claimed in claim 1 is characterized in that when described the 1st parts indicate the described graphical information of needs, described the 2nd parts running.
8. the system as claimed in claim 1 is characterized in that described the 2nd parts are according to the amount of dynamically adjusting the described graphical information that offers described the 1st parts from the feedback of described the 1st parts.
9. the system as claimed in claim 1 is characterized in that described the 2nd parts are with the frequency run based on the animation parameters in the described contextual data.
10. the system as claimed in claim 1 is characterized in that described the 2nd parts provide described graphical information by instruction stream to described the 1st parts.
11. the system as claimed in claim 1 is characterized in that described the 2nd parts provide described graphical information by the network connection to described the 1st parts.
12. the system as claimed in claim 1 is characterized in that the graphical information that is provided by described the 2nd parts is provided described the 1st parts in a group of at least one formation.
13. the system as claimed in claim 1, it is characterized in that further comprising at least one other the 2nd parts that other graphical information is provided to described the 1st parts, and wherein, described the 1st parts will be formed the graph data that offers described graphics subsystem from the combination of the graphical information of described the 2nd parts and described at least one other the 2nd parts.
14. system as claimed in claim 13 is characterized in that keeping in the group of at least one formation of described the 1st parts each in other the 2nd parts of group of at least one formation of described the 2nd parts and at least one graphical information that is provided by described the 2nd parts and described at least one other the 2nd parts.
15. the system as claimed in claim 1 is characterized in that with the described cached data structure of the hierarchical tree structure organization of node.
16. system as claimed in claim 15 is characterized in that at least one node comprises structure of container, described structure of container comprises draws instruction.
17. the system as claimed in claim 1 is characterized in that described the 1st parts will be divided into visual pattern piece and resource data structure by the graphical information that described the 2nd parts provide.
18. the system as claimed in claim 1 is characterized in that described the 1st parts and the 2nd parts carry out on different threads.
19. a kind of method in the computing environment of the graphics subsystem with frame of exporting display message is characterized in that comprising:
At the 1st parts place:
1) receives and to comprise, be used for output corresponding to the calling of the data of graph image;
2) keep described data as scene information; And
3) with the 1st operating rate:
A) described scene information is processed into graphical information;
B) at least some results behind the described scene information of cache handles; And
C) propagate described graphical information to the 2nd parts;
And
At described the 2nd parts place, with faster than the 2nd operating rate of described the 1st operating rate and according to the frame updating rate of described graphics subsystem:
4) receive described graphical information;
5) described graphic information processing is become described graphics subsystem and formative graph data; And
6) propagate described graph data to described graphics subsystem, with output frame.
20. method as claimed in claim 19 is characterized in that described the 1st operating rate is corresponding to receiving the described speed of calling.
21. method as claimed in claim 19 is characterized in that keeping described scene information to comprise structure scene graphic data structure.
22. method as claimed in claim 21 is characterized in that keeping described scene information to comprise and container in the Open Scene graphic data structure comes access drafting instruction wherein.
23. method as claimed in claim 21 is characterized in that handling described scene graphic data structure and comprises at least one value that changes variable.
24. method as claimed in claim 19, it is characterized in that keeping described scene information to comprise will call the variation that receives and rank by described, and uses described variation through queuing.
25. method as claimed in claim 19 is characterized in that handling described scene information and comprises at least one value that obtains variable.
26. method as claimed in claim 19 is characterized in that propagating described graphical information to described the 2nd parts comprises to described the 2nd parts transmission instruction stream.
27. method as claimed in claim 26, it is characterized in that described instruction stream comprises the resource data bag, and wherein described graphic information processing is become described graphics subsystem and the step of formative graph data comprises, handle described resource data bag and this resource is associated with described the 1st parts by establishing resource.
28. method as claimed in claim 26, it is characterized in that described instruction stream comprises more new data packets of visual pattern, and wherein described graphic information processing is become described graphics subsystem and the step of formative graph data comprises, handle more new data packets of described visual pattern by determining whether visual pattern to be associated with described the 1st parts, if and be not associated and just create new visual pattern and it is associated with described the 1st parts, if just be associated according to described visual pattern more the information in the new data packets revise the information of this visual pattern.
29. method as claimed in claim 19 is characterized in that further comprising the information from described the 2nd parts is turned back to described the 1st parts.
30. method as claimed in claim 29 is characterized in that the information from described the 2nd parts is turned back to described the 1st parts to be comprised at least one event queuing.
31. method as claimed in claim 19 is characterized in that further being included in the 3rd parts place and receives and comprise corresponding to other of other data of other graph image that is used to export and calling; Keep described other data as other scene information, and described other scene information is processed into other graphical information, and propagate this other graphical information to described the 2nd parts with the 3rd operating rate.
32. method as claimed in claim 31, it is characterized in that further being included in described the 2nd parts place and receive described other graphical information, described other graphic information processing is become described graphics subsystem and formative other graph data, comprise constituting the figure desktop and propagating described figure desktop with output frame that described figure desktop comprises corresponding to the graphic data of partial graph at least of described the 1st parts and corresponding to the graphic data of partial graph at least of described the 3rd parts to described graphics subsystem.
CN 02147328 2001-10-18 2002-10-18 Hierarchial graph processing system and method Expired - Fee Related CN1272725C (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US33024401P 2001-10-18 2001-10-18
US60/330,244 2001-10-18
US10/184,795 2002-06-27

Publications (2)

Publication Number Publication Date
CN1417700A CN1417700A (en) 2003-05-14
CN1272725C true CN1272725C (en) 2006-08-30

Family

ID=23288909

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 02147328 Expired - Fee Related CN1272725C (en) 2001-10-18 2002-10-18 Hierarchial graph processing system and method

Country Status (1)

Country Link
CN (1) CN1272725C (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8289333B2 (en) * 2008-03-04 2012-10-16 Apple Inc. Multi-context graphics processing
WO2012168961A1 (en) * 2011-06-07 2012-12-13 三菱電機株式会社 Drawing device
US20140344729A1 (en) * 2013-05-15 2014-11-20 Microsoft Corporation Primitive-based composition
WO2020163088A1 (en) * 2019-02-05 2020-08-13 Numenta, Inc. Inferencing and learning based on sensorimotor input data

Also Published As

Publication number Publication date
CN1417700A (en) 2003-05-14

Similar Documents

Publication Publication Date Title
CN1860505B (en) System and method for a unified composition engine in a graphics processing system
JP5101648B2 (en) Visual and scene graph interface
JP4371351B2 (en) Intelligent caching data structure for Immediate Mode graphics
JP4371350B2 (en) Multi-level graphics processing system and method
US7443401B2 (en) Multiple-level graphics processing with animation interval generation
US7486294B2 (en) Vector graphics element-based model, application programming interface, and markup language
US7671862B1 (en) Systems and methods for providing an enhanced graphics pipeline
US7808506B2 (en) Intelligent caching data structure for immediate mode graphics
US20050243094A1 (en) Systems and methods for providing an enhanced graphics pipeline
EP1326165A2 (en) Methods and system for managing computational resources of a coprocessor in a computing system
CN101421761A (en) Visual and scene graph interfaces
WO1994017495A1 (en) Polymorphic graphic device
US10719970B2 (en) Low latency firmware command selection using a directed acyclic graph
US9355464B2 (en) Dynamic generation of texture atlases
US20130063459A1 (en) Primitive composition
CN105190701B (en) Synthesis system based on primitive and method
US9153193B2 (en) Primitive rendering using a single primitive type
CN1272725C (en) Hierarchial graph processing system and method
WO2023160041A1 (en) Image rendering method and apparatus, computer device, computer-readable storage medium and computer program product
US8203567B2 (en) Graphics processing method and apparatus implementing window system
CN118079397A (en) Method for changing and rendering game roles

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
ASS Succession or assignment of patent right

Owner name: MICROSOFT TECHNOLOGY LICENSING LLC

Free format text: FORMER OWNER: MICROSOFT CORP.

Effective date: 20150507

C41 Transfer of patent application or patent right or utility model
TR01 Transfer of patent right

Effective date of registration: 20150507

Address after: Washington State

Patentee after: Micro soft technique license Co., Ltd

Address before: Washington State

Patentee before: Microsoft Corp.

CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20060830

Termination date: 20191018