AU2009201706A1 - Group effects for standards-based graphics interfaces - Google Patents

Description

S&F Ref: 892029 AUSTRALIA PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT Name and Address Canon Kabushiki Kaisha, of 30-2, Shimomaruko 3 of Applicant: chome, Ohta-ku, Tokyo, 146, Japan Actual Inventor(s): Philip Doak Armstrong Address for Service: Spruson & Ferguson St Martins Tower Level 35 31 Market Street Sydney NSW 2000 (CCN 3710000177) Invention Title: Group effects for standards-based graphics interfaces The following statement is a full description of this invention, including the best method of performing it known to me/us: 5845c(2081676_1) - I GROUP EFFECTS FOR STANDARDS-BASED GRAPHICS INTERFACES TECHNICAL FIELD The present invention relates to computer vector graphics and more particularly to handling group effects in a graphics command sequence. BACKGROUND Computer vector graphics comprise descriptive primitives such as curves, lines, bitmap images and text. The primitives may have associated attributes such as line width, fill type (solid colour, gradient fill, tiled bitmap fill, etc), additional functions to apply to the colour or shape, and various operations to use when blending or compositing the graphics. The ) additional functions are hereinafter referred to as effects. Certain of the primitives may form groups and those groups may also have associated attributes that apply only to the group of primitives as a unit. Effects applied to a group of primitives as a unit are referred to as group effects. Computer vector graphics can be generated using illustration tools such as AdobeTM 5 IllustratorTM. Some illustration tools support a rich array of primitives, attributes, operations, and grouping. The graphics primitives generated by illustration tools may be stored in similarly rich data formats such as SVG. The data format may be used as the input (source graphic) for a renderer. A renderer converts vector graphics from primitives to an output surface such as a bitmap image comprising an array of pixels. 0 A graphics client is a software application which interfaces to and sends graphics commands to a graphics Command Interpreter or renderer. If a graphics Command Interpreter or renderer supports a standards-based graphical command interface, it has the advantage of enabling various graphics client applications to simply incorporate the renderer into the system or workflow. Renderers that support a wide variety of graphics commands and effects !5 can accept and render complex graphics efficiently, without requiring graphics clients to break down such graphics into a simpler form. Some standards-based graphics interfaces such as OpenVG(TM) are targeted towards embedded devices which are traditionally characterized as possessing limited resources for graphics processing. Accordingly, such interfaces typically provide limited support for 30 complex graphics commands. 20771371 When developing computer graphics, a developer may have multiple options for achieving a desired result. The same result (e.g., visual appearance) may be achievable using two different techniques, however, it may be recognized that one of the two techniques has a lower computational overhead. In such cases, the designer can choose the technique that will be best suited to the particular rendering application. However, it is not possible to have control over the source graphic in cases when the source graphic is unknown, dynamically generated, or loaded from an arbitrary source such as an internet web page. It is thus desirable to support a standards-based graphics command interface to a renderer for effective re-use as a system component, while retaining the ability to render complex graphics efficiently, particularly complex vector graphics created by popular illustration tools or easily generated by applications or scripts. Furthermore, a need exists to provide a standards-based graphics command interface with extended capability for handling group effects. SUMMARY An aspect of the present invention provides a method for rendering graphics primitives with a specified group effect applied only to a subset of said graphics primitives. The method comprises the steps of: sequentially processing graphics commands for rendering the graphics primitives to identify ones of the graphics commands that the specified group effect applies to; ) storing the identified graphics commands in a command queue; and providing graphics commands from the command queue to a graphics renderer together with an indication that the graphics commands are to be rendered with the specified group effect. The graphics commands to which the group effect does not apply can be provided directly to the graphics renderer without being stored in the command queue. 5 The method may further comprise one or more of the following steps: modifying one or more graphics instructions in the command queue to incorporate the specified group effect; and adding one or more graphics instructions to the command queue to incorporate the specified group effect. The method may comprise the further steps of: identifying a Create Image command; W identifying a Create Surface command; and creating the command queue in response to identifying the Create Surface command, wherein the command queue is associated with a virtual surface. 2077137 1 The method may comprise the further steps of: identifying a Draw Image command in the command queue; determining the specified group effect; and providing the Draw Image command to the graphics renderer together with an indication that the image is to be rendered with the specified group effect. The step of determining the specified effect may comprise i determining a group effect specified by a graphics client using states in a context. The step of determining the specified effect may comprise determining a group effect specified by a graphics client using a Virtual Surface created in response to said identified Create Surface command. The specified group effect may comprise uniform opacity, a Gaussian blur, or an arbitrary ) function on the colour and opacity of the group. The method may be performed by a graphics command processor. Another aspect of the present invention provides a computer system for rendering graphics primitives with a specified group effect applied only to a subset of said graphics primitives. 5 The computer system comprises: a memory for storing data and program instructions; an output interface for outputting data; and a processor coupled to the memory and the output interface. The processor is programmed to: sequentially process graphics commands for rendering the graphics primitives to identify ones of the graphics commands that the specified group effect applies to; store the identified graphics commands in a command queue in the D memory; and provide graphics commands from the command queue via the output interface to a graphics renderer together with an indication that the graphics commands are to be rendered with the specified group effect. Another aspect of the present invention provides a computer program product Z5 comprising a computer readable medium comprising a computer program recorded therein for rendering graphics primitives with a specified group effect applied only to a subset of the graphics primitives. The computer program product comprises: computer program code means for sequentially processing graphics commands for rendering the graphics primitives to identify ones of the graphics commands that the specified group effect applies to; computer 30 program code means for storing the identified graphics commands in a command queue; and computer program code means for providing graphics commands from the command queue to a graphics renderer together with an indication that the graphics commands are to be rendered with the specified group effect. 2077137 1 -4 BRIEF DESCRIPTION OF THE DRAWINGS One or more embodiments of the present invention are described hereinafter with reference to the following drawings, in which: 5 Figs. 1 and 2 are schematic block diagrams of a general purpose computer with which embodiments of the present invention may be practised; Fig. 3 is a schematic block diagram of a graphics rendering system in accordance with an embodiment of the present invention; Fig. 4 is a table of a subset of commands and arguments used by a graphics client; 3 Fig. 5 is a schematic flow diagram of a method for recognising and processing group effects commands according to an embodiment of the present invention; Fig. 6 is a schematic block diagram illustrating steps in rendering graphics primitives; Fig. 7 is a schematic block diagram illustrating steps in rendering graphics primitives subject to a group opacity effect; 5 Fig. 8 is a diagram illustrating a result of rendering graphics primitives; Fig. 9A is a diagram illustrating a result of rendering graphics primitives with opacity effects; Fig. 9B is a diagram illustrating a result of rendering graphics primitives with a group opacity effect; 0 Fig. 10 is a schematic flow diagram of a method for rendering a group effect; Fig 11 A is a schematic flow diagram of a method for rendering a group opacity effect; Fig 1 B is a schematic flow diagram of a method for determining group opacity effect according to an embodiment of the present invention; Fig 12A is a schematic flow diagram of a method for rendering a group image effect; 25 Fig. 12B is a schematic flow diagram of a method for determining group image effects according to an embodiment of the present invention; and Fig. 13 is a schematic flow diagram of a method for rendering graphics primitives with a specified group effect applied only to a subset of the graphics primitives in accordance with an embodiment of the present invention. 30 Where reference is made in any one or more of the accompanying drawings to steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears. 2077137_1 -5 DETAILED DESCRIPTION Embodiments of the present invention facilitate identification and/or provision of grouped vector primitives with effects that apply to the group to a renderer capable of accepting the 5 group effects by a graphics interface that does not, of itself, support group effects. Figs. 1 and 2 collectively form a schematic block diagram of a general purpose computer system 100, with which embodiments of the present invention can be practiced. Specifically, the computer system 100 may be programmed to perform the steps of the methods described D hereinafter. As seen in Fig. 1, the computer system 100 is formed by a computer module 101, input devices such as a keyboard 102, a mouse pointer device 103, a scanner 126, a camera 127, and a microphone 180, and output devices including a printer 115, a display device 114 and loudspeakers 117. An external Modulator-Demodulator (Modem) transceiver device 116 may 5 be used by the computer module 101 for communicating to and from a communications network 120 via a connection 121. The network 120 may be a wide-area network (WAN), such as the Internet or a private WAN. Where the connection 121 is a telephone line, the modem 116 may be a traditional "dial-up" modem. Alternatively, where the connection 121 is a high capacity (e.g., cable) connection, the modem 116 may be a broadband modem. A 0 wireless modem may also be used for wireless connection to the network 120. The computer module 101 typically includes at least one processor 105 and a memory 106, for example, formed from semiconductor random access memory (RAM) and semiconductor read only memory (ROM). The at least one processor 105 may comprise multiple processors, for example, arranged in a pipelined or parallel configuration. The module 101 also includes 25 an number of input/output (1/0) interfaces including an audio-video interface 107 that couples to the video display 114, loudspeakers 117 and microphone 180, an 1/0 interface 113 for the keyboard 102, mouse 103, scanner 126, camera 127 and optionally a joystick (not illustrated), and an interface 108 for the external modem 116 and printer 115. In some implementations, the modem 116 may be incorporated within the computer module 101, for example within the 30 interface 108. The computer module 101 also has a local network interface Il1 which, via a connection 123, permits coupling of the computer system 100 to a local computer network 122, known as a Local Area Network (LAN). As also illustrated, the local network 122 may also couple to the wide network 120 via a connection 124, which would typically include a so 2077137 1 -6 called "firewall" device or device of similar functionality. The interface 111 may be formed by one or more of an Ethernetim arrangement, a BluetoothTh wireless arrangement or an IEEE 802.11 wireless arrangement. The interfaces 108 and 113 may afford either or both of serial and parallel connectivity, the former typically being implemented according to the Universal Serial Bus (USB) standards and having corresponding USB connectors (not illustrated). Storage devices 109 are provided and typically include a hard disk drive (HDD) 110. Other storage devices such as a floppy disk drive and a magnetic tape drive (not illustrated) may also be used. An optical disk drive 112 is typically provided and acts as a non-volatile source of data. Portable memory devices, such as optical disks (e.g., CD-ROM, DVD), USB-RAM, and floppy disks for example may then be used as appropriate sources of data to the system 100. The components 105 to 113 of the computer module 101 typically communicate via an interconnected bus 104 and in a manner which results in a conventional mode of operation of the computer system 100 known to those skilled in the relevant art. Examples of computers 5 on which the described arrangements or embodiments can be practiced include IBM-PCs and compatibles, Sun Sparcstations, Apple MacTM or similar computer systems. The methods or processes described hereinafter may be implemented as software, such as one or more application programs 133 executable within the computer system 100. In 3 particular, the steps of the methods or processes described hereinafter may be implemented as programmed instructions 131 in the software 133 that are executed by the computer system 100. The software instructions 131 may be formed as one or more code modules, each for performing one or more particular tasks. The software may also be divided into two separate parts, in which a first part and the corresponding code modules performs the methods !5 described herein and a second part and the corresponding code modules manage a user interface between the first part and the user. The software 133 is generally loaded into the computer system 100 from a computer readable medium (the software 133 and computer readable medium together form a computer program product), and is then typically stored in the HDD 110, as illustrated in Fig. IA, or the 30 memory 106, after which the software 133 can be executed by the computer system 100. In some instances, the application programs 133 may be supplied to the user encoded on one or more CD-ROM 125 and read via the corresponding drive 112 prior to storage in the memory 110 or 106. Alternatively the software 133 may be read by the computer system 100 from the 2077137 1 -7 networks 120 or 122 or loaded into the computer system 100 from other computer readable media. A computer readable storage media refers to any storage medium that participates in providing instructions and/or data to the computer system 100 for execution and/or processing. Examples of such storage media include floppy disks, magnetic tape, CD-ROM, a i hard disk drive, a ROM or integrated circuit, USB memory, a magneto-optical disk, or a computer readable card such as a PCMCIA card and the like, whether or not such devices are internal or external to the computer module 101. Examples of computer readable transmission media that may also participate in the provision of software, application programs, instructions and/or data to the computer module 101 include radio or infra-red transmission ) channels as well as a network connection to another computer or networked device, and the Internet or Intranets including email transmissions and information recorded on Websites and the like. The second part of the application programs 133 and the corresponding code modules mentioned above may be executed to implement one or more graphical user interfaces (GUIs) 5 to be rendered or otherwise represented upon the display 114. Through manipulation of typically the keyboard 102 and the mouse 103, a user of the computer system 100 and the application may manipulate the interface in a functionally adaptable manner to provide controlling commands and/or input to the applications associated with the GUI(s). Other forms of functionally adaptable user interfaces may also be implemented, such as an audio D interface utilizing speech prompts output via the loudspeakers 117 and user voice commands input via the microphone 180. Fig. 2 is a detailed schematic block diagram of the at least one processor 105 and a "memory" 134. While only a single processor is shown in Figs. I and 2, those skilled in the 25 art will appreciate that multiple processors or processor cores may be used to practice embodiments of the present invention. The memory 134 represents a logical aggregation of all the memory devices (including the HDD 110 and semiconductor memory 106) that can be accessed by the computer module 101 in Fig. IA. When the computer module 101 is initially powered up, a power-on self-test (POST) 30 program 150 executes. The POST program 150 is typically stored in a ROM 149 of the semiconductor memory 106. A program permanently stored in a hardware device such as the ROM 149 is sometimes referred to as firmware. The POST program 150 examines hardware within the computer module 101 to ensure proper functioning, and typically checks the 2077137 1 processor 105, the memory (109, 106), and a basic input-output systems software (BIOS) module 151, also typically stored in the ROM 149, for correct operation. Once the POST program 150 has run successfully, the BIOS 151 activates the hard disk drive 110. Activation of the hard disk drive 110 causes a bootstrap loader program 152 that is resident on the hard 5 disk drive 110 to execute via the processor 105. This loads an operating system 153 into the RAM memory 106 upon which the operating system 153 commences operation. The operating system 153 is a system level application, executable by the processor 105, to fulfil various high level functions, including processor management, memory management, device management, storage management, software application interface, and generic user interface. The operating system 153 manages the memory 109, 106 in order to ensure that each process or application running on the computer module 101 has sufficient memory in which to execute without colliding with memory allocated to another process. Furthermore, the different types of memory available in the system 100 must be used properly so that each process can run effectively. Accordingly, the aggregated memory 134 is not intended to 5 illustrate how particular segments of memory are allocated (unless otherwise stated), but rather to provide a general view of the memory accessible by the computer system 100 and how such is used. The processor 105 includes a number of functional modules including a control unit 139, an arithmetic logic unit (ALU) 140, and a local or internal memory 148, sometimes called a 0 cache memory. The cache memory 148 typically includes a number of storage registers 144 146 in a register section. One or more internal buses 141 functionally interconnect these functional modules. The processor 105 typically also has one or more interfaces 142 for communicating with external devices via the system bus 104, using a connection 118. The application program 133 includes a sequence of instructions 131 that may include 25 conditional branch and loop instructions. The program 133 may also include data 132 which is used in execution of the program 133. The instructions 131 and the data 132 are stored in memory locations 128-130 and 135-137 respectively. Depending upon the relative size of the instructions 131 and the memory locations 128-130, a particular instruction may be stored in a single memory location as depicted by the instruction shown in the memory location 130. 30 Alternately, an instruction may be segmented into a number of parts each of which is stored in a separate memory location, as depicted by the instruction segments shown in the memory locations 128-129. 2077137 1 -9 In general, the processor 105 is given a set of instructions which are executed therein. The processor 105 then waits for a subsequent input, to which it reacts to by executing another set of instructions. Each input may be provided from one or more of a number of sources, including data generated by one or more of the input devices 102, 103, data received from an 5 external source across one of the networks 120, 122, data retrieved from one of the storage devices 106, 109 or data retrieved from a storage medium 125 inserted into the corresponding reader 112. The execution of a set of the instructions may in some cases result in output of data. Execution may also involve storing data or variables to the memory 134. The embodiments disclosed hereinafter may use input variables 154 that are stored in the 0 memory 134 in corresponding memory locations 155-158. The embodiments disclosed hereinafter may produce output variables 161 that are stored in the memory 134 in corresponding memory locations 162-165. Intermediate variables may be stored in memory locations 159, 160, 166 and 167. The register section 144-146, the arithmetic logic unit (ALU) 140, and the control unit 139 5 of the processor 105 work together to perform sequences of micro-operations needed to perform "fetch, decode, and execute" cycles for every instruction in the instruction set making up the program 133. Each fetch, decode, and execute cycle comprises: (a)a fetch operation, which fetches or reads an instruction 131 from a memory location 128; 0 (b)a decode operation in which the control unit 139 determines which instruction has been fetched; and (c)an execute operation in which the control unit 139 and/or the ALU 140 execute the instruction. Thereafter, a further fetch, decode, and execute cycle for the next instruction may be 25 executed. Similarly, a store cycle may be performed by which the control unit 139 stores or writes a value to a memory location 132. Each step or sub-process in the methods or processes described hereinafter is associated with one or more segments of the program 133, and is performed by the register section 144 147, the ALU 140, and the control unit 139 in the processor 105 working together to perform 30 the fetch, decode, and execute cycles for every instruction in the instruction set for the noted segments of the program 133. 2077137_1 - IV An output image can be thought of as being constructed from a stack of vector graphics primitives. The primitives are specified as graphics commands, where the order of graphics commands specifies the order of vector graphics primitives in the stack. As an example, Fig. 8 shows an output image comprising three graphics primitives, namely, a circle 830, a large rectangle 810, and a small rectangle 820. The output image is generated by providing, in order, commands to draw the circle 830, followed by the large rectangle 810, followed by a command to draw the small rectangle 820. In the example of Fig. 8, all three of the graphics primitives are opaque, meaning that the colour of the output image at any point is the colour of the topmost primitive at that point. ) When primitives are not completely opaque, the output image is formed by combining, at each point, the contributions of all shapes whose extents overlap the point according to a compositing model, such as the Porter and Duff model. Fig. 9A shows the same three shapes shown in Fig. 8, except that the rectangles 910 and 920 are not completely opaque (i.e., the rectangles 910 and 920 have an opacity attribute). Accordingly, the colour of the circle 930 is combined with the colours of the rectangles 910 and 920 in the areas where the relevant shapes overlap. The colour of the large rectangle 910 is also combined with the colour of the small rectangle 920 where the rectangle shapes overlap. 3 In contrast, Fig. 9B shows the result of rendering the same three shapes as in Figs. 8 and 9A, except that the rectangles form a group. The group itself is non-opaque but the individual rectangle primitives themselves are opaque. In this case, the colour of the circle 960 is combined with the colour of the rectangles 940 and 950 where the relevant shapes overlap, but the colour of the large rectangle 950 is not combined with the colour of the small rectangle 5 940 where the rectangles overlap. To illustrate a method for generating an output image with a group opacity effect, a technique for rendering primitives individually to images comprising a colour and opacity at each point is firstly described hereinafter with reference to Fig. 6. W Fig. 6 shows the technique applied to primitives having no transparency (i.e., maximum opacity). Commands are provided to a renderer to draw a triangle, followed by a rectangle, followed by an ellipse. When the renderer receives the command to render the opaque triangle, the renderer produces an output image 610 of the triangle comprising the colour of 2077137 1 - I I the triangle with 100% opacity within the extent of the triangle shape, and black with 0% opacity outside the triangle shape. Next, the renderer receives the command to render an opaque rectangle. The renderer produces an output image 620 of the rectangle comprising the colour of the rectangle with 100% opacity within the extent of the rectangle shape, and black with 0% opacity outside the rectangle shape. The renderer then composites the two output images 610 and 620 together according to a compositing model (e.g., the Porter & Duff over operator), thus producing an intermediate image 640. Next, the renderer receives a command to render an opaque ellipse. The renderer produces an output image of the ellipse 630 comprising the colour of the ellipse with 100% opacity within the extent of the ellipse shape, ) and black with 0% opacity outside the ellipse shape. Finally, the renderer composites the intermediate image 640 and the output image of the ellipse 630 (e.g., using the Porter and Duff over operator), thus producing the final image 650. The output image of the ellipse 630 is composited over the intermediate image 640. 5 Fig. 7 illustrates the same technique as shown in Fig. 6, but applied to a group of graphics primitives comprising an opacity attribute. Commands are once again provided to the renderer to draw a triangle, a rectangle, and an ellipse but additional commands are included (as will be described hereinafter) to indicate to the renderer that the rectangle and ellipse form a group with a transparency effect (i.e., partially opaque). As in Fig. 6, the renderer produces output 3 images of the triangle 710, the rectangle 720, and the ellipse 730. Because the rectangle and ellipse are grouped, the renderer first composites the image of the ellipse 730 over the image of the rectangle 720. The result is an intermediate image 750 in which the colour within the extent of the ellipse shape comprises only the colour of the ellipse, with no contribution from the underlying rectangle. ?5 The group opacity effect is applied to the intermediate image 750 by applying the effect at each point in the intermediate image 750 (in this case by multiplying the opacity at each point in the intermediate image 750 by the group opacity value specified by the group opacity effect). The renderer then composites the intermediate image 750 and the output image of the triangle 710 (e.g., using the Porter & Duff over compositing operator) to produce a final 30 output image 760. As can be seen in the final output image 760, the colour of the triangle is combined with the colour of the rectangle where the relevant shapes overlap and the colour of the triangle is combined with the colour of the ellipse where the shapes overlap, but the colour 2077137 1 - IL of the ellipse obscures the colour of the rectangle. The final output image 760 has the desired group opacity effect, in a manner similar to that as shown in Figure 9B. To aid in describing the steps involved in processing a group effect, Fig. 4 defines a subset 5 of graphic commands which a graphics client may use. For example, the Create Image command 420 is for instructing a graphics renderer to allocate a new image which can be used as an output image for the renderer. The Create Image command 420 returns an Image ID 429 to the graphics client so that the graphics client can refer to the image in subsequent commands. The Draw Shape command 410 includes Bezier curve data 411 defining a shape's D outline and a Colour 412. The Draw Image command 430 includes an Image ID 431 for identifying the source image. The Draw Image command 430 instructs the renderer to composite the specified source image onto the current output surface. The Image Effect command 440 includes a Source Image ID 441, Dest (Destination) Image ID 442 and an Effect 443, and instructs the renderer to apply the specified Effect 443 to the specified image 5 with the corresponding Source Image ID 441 and to write the result to the image with the corresponding Dest Image ID 442. An example of an image effect is the Gaussian Blur effect, but could be any function which is applied to one image to produce another image. The Set State command 460 sets a parameter defined by Parameter Name 461 in the renderer to a specified value defined by Parameter Value 462. An example of a parameter which can be set 0 is 'image drawing mode'. The Create Surface command 470 allows the image referenced by Image ID 471 to be used as an output surface by the renderer. A Surface ID 479 is returned to the graphics client. The Set Output Surface command 450 instructs the renderer to make the surface referenced by Surface ID 451 the current output surface for the renderer. 25 Fig. 10 is a flow diagram of a method, which can be used by a graphics client to prepare to apply group effects to graphics primitives. For ease of description, the method of Fig. 10 is described hereinafter with reference to the graphics primitives of Fig. 7 and the graphics commands of Fig. 4. Figs. I 1A and 12A, described hereinafter, are flow diagrams for specific application of a group opacity effect and a group image effect, respectively. 30 Referring to Fig. 10, at step 1010, the graphics client sends a Create Surface command 470 to create the main output surface. The graphics client stores the returned Surface ID 479 as mainID. At step 1020, the graphics client sends a Set Output Surface command 450 specifying mainID as the Surface ID 451. At step 1025, the graphics client sends a Draw Shape 2077137 1 command 410 with Bezier data 411 and Colour 412 of the triangle 710. At step 1030, the graphics client sends a Create Image command 420 to create the group intermediate image 750 and stores the returned identifier Image ID 429 as groupImgID. At step 1035 the graphics client sends a Create Surface command 470 with grouplmglD as Image ID 471 to enable i rendering to the group image. The graphics client stores the returned Surface ID 479 as groupID. At step 1040, the graphics client sends a Set Output Surface command 4350 to specify grouplD as the Surface ID 451 to set the group image as the current output surface for the renderer. At step 1043, the graphics client sends a Draw Shape command 410 with the path data and colour of the rectangle 720. At step 1044, the graphics client sends a draw shape ) command 410 with the path data and colour of the ellipse 730. At step 1050 the graphics client sends a Set Output Surface command 450 to specify mainiD as the Surface ID 451 to set the main surface as the current output surface. At step 1090, the graphics client sends commands specific to the desired group effect. 5 Fig. 1 IA is a flow diagram of a method, which can be used by a graphics client to render a group opacity effect. The method of Fig. 1 1 A may be used to implement step 1090 of the method of Fig. 10 to apply and render a group opacity effect and is described hereinafter with reference to the graphics primitives of Fig. 7 and the graphics commands of Fig. 4. Referring to Fig. 1 IA, at step 1155, the graphics client sends a Set State command 460 0 with Parameter Name 461 set to image drawing mode and Parameter Value 462 set to multiply mode to set the image drawing mode to multiply. Multiply image drawing mode means that the colour and alpha of each pixel of the image being drawn is multiplied by the current fill colour before being rendered. At step 1157, the graphics client sets the current fill colour to white and the current fill opacity to the group opacity value. At step 1160, the 25 graphics client sends a draw image command 430, specifying groupImgID as the Image ID 431 for the source image to the renderer. Fig. 12A is a flow diagram of a method which can be used by a graphics client to render a Gaussian Blur image effect on a group of graphics primitives instead of the group opacity 30 effect rendered in the method fo Fig. 11 A. The method of Fig. 12A may be used to implement step 1090 of the method of Fig. 10 to apply and render a group image effect and is described hereinafter with reference to the graphics primitives of Fig. 7 and the graphics commands of Fig. 4. 2077137_1 Referring to Fig. 12A, at step 1250, the graphics client sends a Create Image command 420 to create a temporary image to hold the result of the image effect and stores the returned identifier Image ID 429 as tmpImgID. At step 1255, the graphics client sends an Image Effect command 440 with grouplmgld as Source Image ID 441, tmplmgID as Dest Image ID 442, i and gaussian blur(sx, sy) as Effect 443. The argument gaussian blur(sx, sy) means apply a Gaussian blur effect on the source image with parameters of standard deviation sx in the x direction and standard deviation sy in the y direction. At step 1260, the graphics client sends a Draw Image command 430 with tmpImglD as Image ID 331 to the renderer. ) In contrast to the method described hereinbefore with reference to Fig. 7, the methods described in Figs. 10, 11 A and 12A do not create an image for each graphic primitive, but only an image for the group at step 1030 of Fig. 10 and an image for the Gaussian Blur effect at step 1250 of Fig. 12A. By reducing the number of images created, less storage (memory) is advantageously required to render the graphic image. 5 A renderer may draw a series of graphics primitives sequentially to the same surface by compositing each primitive on-the-fly with the contents of the output surface (e.g., using Painter's algorithm). Rendering multiple graphics primitives to the same surface has further advantages (in addition to reduced storage requirements) over rendering each primitive to 0 separate images: the output image only needs to be modified at points inside the extent of a graphical primitive, rather than the entire image; and there is no need to composite full images into an intermediate result as is the case for the method described hereinbefore with reference to Fig. 6. Compositing full images can be a computationally expensive operation, requiring per-pixel computation when the image is represented as an array of pixels. It is thus highly 25 desirable to eliminate most or all intermediate images when rendering a sequence of graphics primitives. Fig. 3 is a schematic block diagram of a graphics rendering system in accordance with an embodiment of the present invention. The graphics rendering system of Fig. 3 may be 30 implemented wholly or in part using one or more general purpose computer systems such as the computer system 100 of Figs. I and 2. For example, the Command Interpreter 330 and/or the Graphics Client 310 may be implemented using a general purpose computer system such as the computer system 100 of Figs. I and 2. 2077137_1 - 1:, Referring to Fig. 3, a Graphics Client application 310 sends graphics commands to a Command Interpreter 330 via a standards-based graphics command interface 320. In the present embodiment, the graphics command interface 320 is a collection of OpenVG(TM) functions. The standards-based graphics command interface 320 supports commands i equivalent to those described hereinbefore with reference to Fig. 4. The standards-based graphics command interface 320 supports the application of effects to images as described hereinbefore with reference to Figs. 10, 11 A and 12A, but does not support the application of effects to groups of vector primitives. That is, the standards-based graphics command interface 320 only supports single object effects. The Command Interpreter 330 converts commands from the format of the standards-based graphics command interface 320 to a format acceptable to the Extended Functionality Graphics Renderer 380. The Extended Command Interpreter 380 is capable of handling grouped primitive operations that are not supported by the original Command Interpreter 330. The Command Interpreter 330 sends the converted commands to the Extended Functionality Graphics Renderer 380 via the Command 5 Interpreter Output Interface 370. Most commands can be passed directly from the standards based graphics command interface 320 to the Extended Functionality Graphics Renderer 380 unchanged. The Extended Functionality Graphics Renderer 380 has support for grouping primitives and support for applying effects such as opacity to groups of primitives. The Command Interpreter 330 can create a Virtual Surface 360, including a Command Queue 350 3 and an Image ID 355 in response to a Create Surface command 470 from the Graphics Client 310. The Create Surface command 470 forms part of the standards-based graphics command interface 320. The Command Interpreter 330 can store and retrieve graphics commands in and from the Command Queue 350, as indicated by the Command Queue command stream 340. Additional Virtual Surfaces 365 can be created by the Command Interpreter 330, each in !5 response to a Create Surface command 470. The Command Interpreter 330 stores state information in the Current Context 392. State information comprises the current output surface and the current image mode. An Image List 390 stores image references whenever the Graphics Client 310 sends a Create Image 420 command (retrieving the image references from the Extended Functionality Graphics Renderer). 30 Fig. 5 is a flowchart of a method 500 performed in response to a sequence of commands from a graphics client to render group effects according to an embodiment of the present invention. The method of Fig. 5 is described hereinafter with reference to the graphics 2077137 1 -10 rendering system and Command Interpreter 330 described hereinbefore with reference to Fig. 3 and the subset of client graphics commands described hereinbefore with reference to Fig. 4. However, persons skilled in the art will appreciate that the method of Fig. 5 may alternatively be performed by other graphics rendering systems and/or other graphics commands. Referring to Fig. 5, at step 501, the Command Interpreter accepts a command from the Graphics Client 310. At step 575, the Command Interpreter 330 determines whether the accepted command is a Create Image command 420. If a Create Image command is recognised (Yes), the Command Interpreter 330 allocates a unique Image ID as the returned Image ID 429 for the graphics client 310 at step 570. The Command Interpreter 330 does not ) allocate storage resources for the image at this point (for example, the Command Interpreter 330 does not immediately pass a Create Image command to the Extended Functionality Graphics Renderer 380). Rather, the Command Interpreter 330 adds the Image ID 429 to the Image List 390 at step 570. After step 570, the method returns to step 501 to process a subsequent graphics command. 5 If a Create Image command is not recognised (No), at step 575, the Command Interpreter 330 determines whether the accepted command is a Create Surface command 470. If a Create Surface command is recognised (Yes), the Create Surface command 470 serves to signal the beginning of a group and the Command Interpreter 330 proceeds to create a Virtual Surface 360 including a Command Queue 350 at step 510. The Image ID 471 argument in the Create 0 Surface command 470 is stored as the Image ID 355 in the Virtual Surface 360. A reference to the Virtual Surface 360 is returned to the graphics client 310 as the return value for Surface ID 479 of the Create Surface command 470. By returning the reference to the Virtual Surface 360 as the return value for the Create Surface command 470, subsequent references to Surface ID 479 by the graphics client 310 can be associated with the newly created Virtual Surface 360 25 and Command Queue 350 by the Command Interpreter 330. Effectively, the Command Interpreter 330 uses the Virtual Surface 360 in place of an image buffer for the surface. After step 510, the method returns to step 501 to process a subsequent graphics command. If a Create Surface command is not recognised (No), at step 505, the Command Interpreter 330 performs actions to determine the specified effect at step 535. Specific actions or steps for 30 determining group opacity effect and an image effect are described hereinafter with reference to Figs. II B and 12B, respectively. Returning to Fig. 5, the Command Interpreter 330 determines whether the current surface is the virtual surface at step 515. The Command Interpreter 330 recognises commands targeting 2077137_1 - I / the newly created surface by comparing the Surface ID 415 of the current output surface in the Current Context 392 with the Virtual Surface reference 360 for each Virtual Surface 360, 365. If a corresponding Virtual Surface is found (Yes), at step 515, the current command is added to the Command Queue 350 in the Virtual Surface 360 corresponding to the current output surface in the Current Context 392 at step 520. Thereafter, the method returns to step 501 to process a subsequent graphics command. Alternatively, if the current surface is not a virtual surface (No), at step 515, the Command Interpreter 330 identifies Draw Image commands 430 which include the Virtual Surface reference 360 as the source image by comparing the image ID 431 of the Draw Image ) command 430 with the Image ID 355 for each Virtual Surface 360 at step 525. If a matching Virtual Surface 360 is found (Yes), at step 525, the Command Interpreter 330 examines the Current Context 392 and the matching Virtual Surface 360 at step 540 to determine the group effect specified by the Graphics Client 310. Examples of specific actions that may be performed in step 540 are described hereinafter with reference to Figs. II B and 12B. After 5 step 540, the drawing commands in the corresponding queue are modified (for example translated, scaled, output surface changed) and sent to the Extended Graphics Renderer 380 along with grouping commands and the group effect at step 550. Thereafter, the method returns to step 501 to process a subsequent graphics command. Alternatively, if no matching queue is found (No), at step 525, the graphics command is ) sent directly to the Extended Graphics Renderer 380 at step 560. Thereafter, the method returns to step 501 to process a subsequent graphics command. Fig. 11B is a flow diagram showing sub-steps of the Determine Effects Setup Processing step 535 of Fig. 5 specifically for determining a group opacity effect. The steps of Fig. 1 1B 5 are typically performed by the Command Interpreter 330 in response to the sequence of commands/steps described hereinbefore with reference to Fig. I 1A to render a group opacity effect. Referring to Fig. 11 B, at step 1135, the Command Interpreter 330 determines whether the current command is the Set State command 460. If not (No), the method proceeds to step 515 50 of Fig. 5. However, if the command is the Set State command 460 (Yes), the method continues at step 1130, at which the Command Interpreter 330 sets the parameter corresponding to Parameter Name 461 in the Current Context 392 to a value equal to Parameter Value 462. In particular, when Parameter Name 461 is image drawing mode, the 2077137I - IS parameter image drawing mode in the Current Context 392 is set to Parameter Value 462. Similarly, the current fill in the Current Context 392 is set by the Graphics Client 310 by issuing a Set State command 460 with Parameter Name 461 set to fill and Parameter Value 462 set to the colour of the fill. Step 540 in Fig. 5, which determines the group effect specified by the graphics client, checks if the image drawing mode in the Current Context 392 is multiply image drawing mode and the current fill is white with an opacity value not equal to 100%. If the check succeeds, the effect is determined to be group opacity. Advantageously, no intermediate group images need to be created for the group opacity example. Image creation is delayed by the Command Interpreter 330 but the Image List 390 tracks the image reference until it becomes necessary to create the image. Referring to the example described hereinbefore with reference to Fig. 7, the method of Figs. 5 and 11 B require generation of the final output image 760 only. Costly per-pixel compositing operations are advantageously saved by not requiring full image compositing of intermediate images such 5 as the intermediate group image 750. Fig. 12B is a flow diagram showing sub-steps of the Determine Effects Setup Processing step 535 of Fig. 5 specifically for determining image effects. The steps of Fig. 12B are typically performed by the Command Interpreter 330 in response to the sequence of 0 commands/steps described hereinbefore with reference to Fig. 12A to render a group image effect. Referring to Fig. 12B, at step 1235, the Command Interpreter 330 determines whether the current command is the Image Effect command 440. If not (No), the method proceeds to step 515 of Fig. 5. However, if the current command is the Image Effect command 440 (Yes), the 25 Command Interpreter 330 creates a new Virtual Surface 360 at step 1230, setting the Image ID 355 to the Dest Image ID 442. At step 1236 the Command Interpreter 330 determines whether the Source Image ID 441 of the Image Effect command 440 is a Virtual Surface 360 by comparing the Source Image ID 441 with the Image ID 355 for each Virtual Surface 360, 365. If a corresponding Virtual Surface 360 is found (Yes), at step 1236, the Command Interpreter 30 330 replaces the Dest Image ID 442 in the Image Effect command 440 with a reference to the command currently at the end of the Command Queue 350 in the corresponding Virtual Surface 360, at step 1237. Knowing the commands in the Virtual Surface 360 at the time of applying the image effect enables more graphics primitives to be drawn to the Virtual Surface 2077137_1 -19 360 (used as the Source Image ID 441 for the Image Effect command 440) without affecting the image effect. Then, at step 1238, the Command Interpreter 330 adds the Image Effect command 440 as the first command in the new Virtual Surface 360. Step 540 of Fig. 5, which determines the group effect, checks if the first command in the Command Queue 350 is an Image Effect command 440 with the Source Image ID 441 referring to a Virtual Surface 360. If the check succeeds, the effect is determined to be the specified image effect. Use of the method described hereinbefore with reference to Fig. 12B advantageously obviates the need for creating any temporary or intermediate images, both for the create group ) image step 1030 of Fig. 10 and the create temporary image step 1250 of Fig. 12A. Fig. 13 is a schematic flow diagram of a method for rendering graphics primitives with a specified group effect applied only to a subset of the graphics primitives in accordance with an embodiment of the present invention. The method may, for example, be performed by a 5 graphics command processor. The graphics command processor may be implemented using the general purpose computer system 100 described hereinbefore with reference to Figs. I and 2. Referring to Fig. 13, graphics commands for rendering the graphics primitives are sequentially processed at step 1310 to identify which of the graphics commands that the group 0 effect applies to. While the graphics commands are processed sequentially, the identified commands may be non-sequential. The graphics commands identified in step 1310 are stored in a command queue in step 1320. In step 1330, the graphics commands in the command queue are provided to a graphics renderer together with an indication that the graphics commands are to be rendered with the specified group effect. 25 The remaining graphics commands (i.e., those graphics commands to which the group effect does not apply) can be provided directly to the graphics renderer without being stored in the command queue. The graphics instructions in the command queue may be modified and/or further graphics commands may be added to the command queue to incorporate the group effect. 30 Sequential processing of the graphics commands to identify which graphics commands (and thus graphics primitives) the group effect applies to may include identifying a Create Image command and/or identifying a Create Surface command. The command queue, which is 2077137 1 - LU associated with a virtual surface (i.e., the surface to be created as a result of the Create Surface command) may be created in response to identification of the Create Surface command. Sequential processing of the graphics commands to identify which graphics commands (and thus graphics primitives) the group effect applies to may further include identifying a 5 Draw Image command in the command queue, determining the specified group effect (which is typically provided by the Graphics Client), and providing the Draw Image command to the graphics renderer together with an indication that the image is to be rendered with the specified group effect. The group effect may be specified by a graphics client using states in a context. The group effect may be specified by a graphics client using a Virtual Surface created ) in response to the identified Create Surface command. The specified group effect may, for example, comprise uniform opacity, a Gaussian blur, or an arbitrary function on the colour and opacity of the group. However, those skilled in the art will appreciate that other group effects may alternatively or additionally be applied to graphics commands representing graphics primitives in accordance with the principles of the present 5 invention. The methods described hereinbefore may be practiced using a general purpose computer system comprising a memory for storing data and program instructions, an output interface for outputting data, and a processor coupled to the memory and the output interface. The 0 processor is programmed to perform the steps of a method described hereinbefore (e.g., the method of Fig. 13). Furthermore, the methods described hereinbefore may be provided as a computer program product comprising a computer readable medium comprising a computer program recorded therein. The computer program product comprises computer program code means for performing the steps of a method described hereinbefore (e.g., the method of Fig. 25 13). Embodiments of methods, systems and computer program products have been described hereinbefore for rendering graphics primitives with a specified group effect. The foregoing description provides exemplary embodiments only, and is not intended to limit the scope, 30 applicability or configurations of the present invention. Rather, the description of the exemplary embodiments provides those skilled in the art with enabling descriptions for implementing embodiments of the invention. Various changes may be made in the function 2077137 I -21 and arrangement of elements without departing from the spirit and scope of the invention as set forth in the claims hereinafter. For example, instead of the Set Output command 450, the target output surface may be specified as an argument of the Draw Shape command 410, the Draw Image command 430, or the Image Effect command 440. Also, parameters such as 'image drawing mode' may be specified as additional arguments to a Draw Shape command 410 or other command rather than by use of a Set State command 460. Furthermore, additional commands may be defined and/or used to draw shapes other than Bezier paths, such as line segments, polygons, circles, etc. While flat colours have been used in embodiments described hereinbefore, fill colours may also be used in commands such as the Draw Shape command 410 to enable gradient fills, tiled pattern fills, and/or other functions that generate colours over an x, y pixel array. Furthermore, rather than the Command Interpreter 330 returning an Image ID 429 for the Create Image command 420 or a Surface ID 479 for the Create Surface command 470, the Graphics Client 310 may specify an Image ID or a Surface ID as an additional command argument. Furthermore, the standards based graphics command interface 320 may be implemented as a command stream or other interface instead of a collection of functions. For all types of command interface, it is possible to combine commands. For example, the ) Draw Image command 430 may specify an Image Effect 443, indicating that the image should have the effect applied before or during drawing of the image. Where specific features, elements and steps referred to herein have known equivalents in the art to which the invention relates, such known equivalents are deemed to be incorporated 5 herein as if individually set forth. Furthermore, features, elements and steps referred to in respect of particular embodiments may optionally form part of any of the other embodiments unless stated to the contrary. The term "comprising" (and its grammatical variations such as "comprises" and "is 0 comprised of') as used herein are used in the open sense of "having" or "including" and not in the closed sense of "consisting only of'. 2077137_1

Claims (24)

1. A method for rendering graphics primitives with a specified group effect applied only to a subset of said graphics primitives, said method comprising the steps of: sequentially processing graphics commands for rendering said graphics primitives to identify ones of said graphics commands that said specified group effect applies to; storing said identified graphics commands in a command queue; and providing graphics commands from the command queue to a graphics renderer together with an indication that said graphics commands are to be rendered with the specified group ) effect.

2. The method of claim 1, comprising the further step of: providing graphics commands that said specified group effect does not apply to directly to said graphics renderer without storing said graphics commands in said command queue.

3. The method of claim 1, wherein graphics commands that said specified group effect does not apply to are non-sequential.

4. The method of claim 1, further comprising one or more steps of the group of steps 0 consisting of: modifying one or more graphics instructions in said command queue to incorporate said specified group effect; and adding one or more graphics instructions to said command queue to incorporate said specified group effect. 25

5. The method of claim 1, comprising the further steps of: identifying a Create Image command; identifying a Create Surface command; and creating said command queue in response to identifying said Create Surface command, 30 wherein said command queue is associated with a virtual surface.

6. The method of claim 5, comprising the further steps of: identifying a Draw Image command in said command queue; 2077137_1 determining said specified group effect; and providing said Draw Image command to said graphics renderer together with an indication that said image is to be rendered with the specified group effect. i

7. The method of claim 6, wherein said step of determining said specified group effect comprises determining a group effect specified by a graphics client using states in a context.

8. The method of claim 6, wherein said step of determining said specified group effect comprises determining a group effect specified by a graphics client using a Virtual Surface ) created in response to said identified Create Surface command.

9. The method of claim 1, wherein said specified group effect comprises a group effect selected from the group of group effects consisting of: uniform opacity; 5 a Gaussian blur; and an arbitrary function on the colour and opacity of the group.

10. The method of claim 1, wherein said method is performed by a graphics command processor. 0

11. A computer system for rendering graphics primitives with a specified group effect applied only to a subset of said graphics primitives, said computer system comprising: a memory for storing data and program instructions; an output interface for outputting data; and Z5 a processor coupled to said memory and said output interface, said processor programmed to: sequentially process graphics commands for rendering said graphics primitives to identify ones of said graphics commands that said specified group effect applies to; store said identified graphics commands in a command queue in said memory; and 30 provide graphics commands from the command queue via said output interface to a graphics renderer together with an indication that said graphics commands are to be rendered with the specified group effect. 2077137 1 - 24

12. The computer system of claim 11, wherein said processor is further programmed to: provide graphics commands that said specified group effect does not apply to directly to said graphics renderer without storing said graphics commands in said command queue. i

13. The method of claim 11, wherein graphics commands that said specified group effect does not apply to are non-sequential.

14. The computer system of claim 11, wherein said processor is further programmed to perform one or more tasks from the group consisting of: ) modifying one or more graphics instructions in said command queue to incorporate said specified group effect; and adding one or more graphics instructions to said command queue to incorporate said specified group effect. 5

15. The computer system of claim 11, wherein said processor is further programmed to: identify a Create Image command; identify a Create Surface command; and create said command queue in response to identifying said Create Surface command, wherein said command queue is associated with a virtual surface. D

16. The computer system of claim 15, wherein said processor is further programmed to: identify a Draw Image command in said command queue; determine said specified group effect; and provide said Draw Image command via said output interface to said graphics renderer ?5 together with an indication that said image is to be rendered with the specified group effect.

17. The computer system of claim 11, wherein said specified group effect comprises a group effect selected from the group of group effects consisting of: uniform opacity; 30 a Gaussian blur; and an arbitrary function on the colour and opacity of the group. 2077137_1

18. A computer program product comprising a computer readable medium comprising a computer program recorded therein for rendering graphics primitives with a specified group effect applied only to a subset of said graphics primitives, said computer program product comprising: computer program code means for sequentially processing graphics commands for rendering said graphics primitives to identify ones of said graphics commands that said specified group effect applies to; computer program code means for storing said identified graphics commands in a command queue; and computer program code means for providing graphics commands from the command queue to a graphics renderer together with an indication that said graphics commands are to be rendered with the specified group effect.

19. The computer program product of claim 18, further comprising computer program code means for providing graphics commands that said specified group effect does not apply to directly to said graphics renderer without storing said graphics commands in said command queue.

20. The computer program product of claim 18, wherein graphics commands that said ) specified group effect does not apply to are non-sequential.

21. The computer program product of claim 18, further comprising one or more computer program code means from the group consisting of: computer program code means for modifying one or more graphics instructions in said 5 command queue to incorporate said specified group effect; and computer program code means for adding one or more graphics instructions to said command queue to incorporate said specified group effect.

22. The computer program product of claim 18, further comprising: 0 computer program code means for identifying a Create Image command; computer program code means for identifying a Create Surface command; and 2077137_1 - 2ti computer program code means for creating said command queue in response to identifying said Create Surface command, wherein said command queue is associated with a virtual surface.

23. The computer program product of claim 22, further comprising: computer program code means for identifying a Draw Image command in said command queue; computer program code means for determining said specified group effect; and computer program code means for providing said Draw Image command to said graphics renderer together with an indication that said image is to be rendered with the specified group effect.

24. The computer program product of claim 18, wherein said specified group effect comprises a group effect selected from the group of group effects consisting of: uniform opacity; a Gaussian blur; and an arbitrary function on the colour and opacity of the group. DATED 27 APRIL, 2009 CANON KABUSHIKI KAISHA PATENT ATTORNEYS FOR THE APPLICANT/NOMINATED PERSON SPRUSON & FERGUSON 2077619 1