AU2008246224A1 - Method, apparatus and system for rendering - Google Patents

Description

S&F Ref: 882425 AUSTRALIA PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT Name and Address Canon Kabushiki Kaisha, of 30-2, Shimomaruko 3 of Applicant: chome, Ohta-ku, Tokyo, 146, Japan Actual Inventor(s): David Nathan Riesel David Robert James Monaghan Ian Geoffrey Combes Address for Service: Spruson & Ferguson St Martins Tower Level 35 31 Market Street Sydney NSW 2000 (CCN 3710000177) Invention Title: Method, apparatus and system for rendering The following statement is a full description of this invention, including the best method of performing it known to me/us: 5845c(1 852414_1) METHOD, APPARATUS AND SYSTEM FOR RENDERING FIELD OF THE INVENTION The present invention relates generally to image processing and, in particular, to a method and apparatus for rendering a portion of a document on a rendering system. The present invention also relates to a computer program product including a computer readable medium 5 having recorded thereon a computer program for rendering a portion of a document on a rendering system. DESCRIPTION OF BACKGROUND ART A computer application typically provides a page, to a device for printing, in the form of a description of the page. The description is provided to device driver software of the printer J device in a page description language (PDL), such as Adobe@ PostScript@ or Hewlett Packard@ PCL. The PDL provides descriptions of objects to be rendered onto the page in a rendering (or Z-) order, as opposed to providing a raster image of the page to be printed. The page is typically rendered for printing and/or display by an object-based graphics system, also known as a Raster Image Processor (RIP). 5 Page contents vary in complexity. The complexity of a page is related to object density and amount of overlap within the page. For example, for pages containing text only the object density is low and there is usually no compositing to be performed. Such a page is not considered to be complex. In contrast, a page with hundreds of overlapping transparent bitmaps requires compositing and bitmap interpolation, and is considered to be complex. !0 Unfortunately, the complexity of a page is unknown until the entire page has been interpreted by the printer device. Due to varying and unpredictable complexity of pages, a printer device cannot typically guarantee that all pages can be rendered in real-time. Conventional printer devices typically spool an entire page of pixels in memory before sending rendered pixels to a printer engine. Spooling a page of pixels requires a large amount 25 of memory to store the pixel data, and sending the pixel data from the printer device to the printer engine consumes a large amount of bandwidth. The spooled pixels may be compressed, but this often requires a large amount of processing resources and introduces quality loss. In addition to the memory usage required for storing the spooled pixels, a large amount of memory is often required for the storage and processing of complex pages. 30 As such, printer devices with low memory may not be able to both store and process an entire page representation. - 1- 882425_Final(1848794vl) One method of ensuring that a printing system can render an original page representation more efficiently and with less memory is to transform the page representation into an intermediate page representation if the original page representation exceeds some complexity bounds. In general, the intermediate page representation is smaller in size than a 5 corresponding raster pixel representation. Generating the intermediate page representation occupies a printer controlling program processor resources. Rendering the intermediate page representation to pixels is left to a pixel rendering apparatus. The intermediate page representation is typically specified to be simple enough, so that the intermediate page representation can be rendered to pixels without requiring excessive processor resources or ) excessive memory. Several types of intermediate page representation formats are known. These known intermediate page representation formats have varying degrees of complexity. One method of generating an intermediate page representation involves dividing a page into regions and predicting time or resources required to render the region. A printer 5 controlling program, rather than a pixel rendering apparatus, is then used to render the region to pixels if the time or resources exceed some threshold. The intermediate page representation generated using such a method consists of a mix of PDL data and rasterised pixels. As a general rule, intermediate page representations that are simple to render by the pixel rendering apparatus, are difficult to generate. On the other hand, intermediate page 0 representations that are quick to generate, are generally complex and difficult to render. Often, it is extremely difficult to render an intermediate page representation in real time, especially if an associated input PDL is very complex. Using an appropriate intermediate page representation can improve the performance of a printing system in rendering a page. However, there are a number of limitations to using an ?5 intermediate page representation. For any intermediate page representation, there are a large number of medium complexity pages that could be efficiently processed with either a simple representation (e.g., PDL) or a more complex intermediate page representation. However it is extremely difficult to a priori predict the kind of intermediate page representation format most suitable for any given page. A printing system which is biased toward using a simple to 30 render intermediate page representation format may place a large workload on a printer controlling program, leaving a pixel rendering apparatus idle, and can stall the print engine. A printing system biased toward creating a fast method to generate intermediate page -2- 882425_Final(1 848794v1) representation puts a large workload on a pixel rendering apparatus, which can stall a printer controlling program. One method of ensuring that a printing system can efficiently render a page representation is to dynamically allocate resources between a printer controlling program or pixel rendering apparatus as required to render the page. If a page is exceedingly complex, additional processing resources and memory might be allocated to the pixel rendering apparatus to allow the pixel rendering apparatus to render the page. However, such a method does not necessarily optimise throughput of the printing system as a whole, as the printer controlling program and pixel rendering apparatus are competing for resources. If there are a large number of complex pages requiring rendering, both the printer controlling program and the pixel rendering apparatus will be heavily loaded, and dynamic resource allocation may lead to resource contention. Thus, a need clearly exists for a more efficient method of rendering page representations. SUMMARY OF THE INVENTION It is an object of the present invention to substantially overcome, or at least ameliorate, one or more disadvantages of existing arrangements. Disclosed are arrangements which seek to address the above problems by using a plurality of distinct formats for spooled intermediate page representations used for rendering pages from PDL to pixels. The disclosed arrangements overcome any imbalance between an intermediate page representation generator and a renderer by introducing a dynamic feedback in the form of intermediate page representation queue size into the intermediate page representation generator. The intermediate page representation generator uses the intermediate page representation queue size, together with an input job complexity estimate, to decide what 5 format of intermediate page representation to spool. The disclosed arrangements ensure both efficient resource allocation and optimal division of workload between a printer controlling program and a pixel rendering apparatus. According to one aspect of the present invention there is provided a method of rendering a portion of a document on a rendering system, said method comprising the steps of: 0 determining complexity of the portion of the document based on a representation of the document portion; determining workload balance of the rendering system; - 3- 882425_Final(1 848794vl) selecting one of a plurality of intermediate representation formats depending on the complexity of the portion of the document and the workload balance of the rendering system; generating an intermediate representation of the portion of the document according to the selected format; and rendering the portion of the document based on the generated intermediate page representation, wherein the workload balance of the rendering system is determined based on a queue of intermediate representations. According to another aspect of the present invention there is provided a rendering system for rendering a portion of a document, said system comprising: a memory for storing data and a computer program; and a processor coupled to said memory executing said computer program, said computer program comprising instructions for: determining complexity of the portion of the document based on a representation of the document portion; determining workload balance of the rendering system; selecting one of a plurality of intermediate representation formats depending on the complexity of the portion of the document and the workload balance of the rendering system; generating an intermediate representation of the portion of the document according to the selected format; and rendering the portion of the document based on the generated intermediate page representation, wherein the workload balance of the rendering system is determined based on a queue of intermediate representations. According to still another aspect of the present invention there is provided an 5 apparatus for rendering a portion of a document on a rendering system, said apparatus comprising: means for determining complexity of the portion of the document based on a representation of the document portion; means for determining workload balance of the rendering system; 0 means for selecting one of a plurality of intermediate representation formats depending on the complexity of the portion of the document and the workload balance of the rendering system; -4- 882425_Final(1848794v1) means for generating an intermediate representation of the portion of the document according to the selected format; and means for rendering the portion of the document based on the generated intermediate page representation, wherein the workload balance of the rendering system is determined 5 based on a queue of intermediate representations. According to still another aspect of the present invention there is provided a computer readable medium, having a program recorded thereon, where the program is configured to make a computer execute a procedure to render a portion of a document on a rendering system, said program comprising: 3 code for determining complexity of the portion of the document based on a representation of the document portion; code for determining workload balance of the rendering system; code for selecting one of a plurality of intermediate representation formats depending on the complexity of the portion of the document and the workload balance of the rendering 5 system; code for generating an intermediate representation of the portion of the document according to the selected format; and rendering the portion of the document based on the generated intermediate page representation, wherein the workload balance of the rendering system is determined based on 0 a queue of intermediate representations. Other aspects of the invention are also disclosed. Brief Description of the Drawings One or more embodiments of the invention will now be described with reference to the following drawings, in which: Z5 Figs. IA, lB and IC form a schematic block diagram of a computer system upon which arrangements described can be practised; Fig. 2 is a schematic block diagram of a conventional printing system, where a vectorised display list format is used for an intermediate page representation; Fig. 3 is a schematic block diagram of another conventional printing system, where a 30 fillmap format is used as the intermediate page representation; Fig. 4 is a schematic block diagram of a print system according to an embodiment of this invention; -5- 882425_Final(1848794v1) Fig. 5 is a schematic flow diagram illustrating a method of spool format determination used by the intermediate format determination module on Fig. 4; Fig. 6 is a schematic diagram of an exemplary fillmap spool generation module from fig. 4, where the computing resources handling the task of fillmap generation are dynamically allocated. Fig. 7 shows an exemplary page with some graphic objects on the page; Fig. 8 shows a fillmap representation of the page shown in Fig. 7; Fig. 9 describes data structures of a fillmap representation, such as shown in Fig 8, in more detail; Fig. 10 shows an example of a fill compositing sequence table, which is part of a fillmap representation; Fig. 11 is a flow diagram showing a method of rendering one or more pages of a document on the pixel rendering system of Fig. IA to I C; Fig. 12 is a flow diagram showing a method of generating an intermediate page representation, as executed in the method of Fig. 11; Fig. 13 is a flow diagram showing a method of generating a vectorised display list, as may be executed in the method of Fig. 12; and Fig. 14 is a flow diagram showing a method of generating the intermediate page representation in the form of the fillmap representation, as may be executed in the method of Fig. 12. DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION Where reference is made in any one or more of the accompanying drawings to steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention 5 appears. Figs. 1A to IC collectively form a schematic block diagram of a pixel rendering system 100, upon which the various arrangements described can be practiced. As seen in Fig. 1A, the system 100 is formed by a computer module 101, input devices such as a keyboard 102, a mouse pointer device 103, a scanner 126, a camera 127, and a 0 microphone 180, and output devices including a printing system 115, a display device 114 and loudspeakers 117. An external Modulator-Demodulator (Modem) transceiver device 116 may be used by the computer module 101 for communicating to and from a communications network 120 via a connection 121. The network 120 may be a wide-area network (WAN), - 6- 882425_Final(1848794v]) such as the Internet or a private WAN. Where the connection 121 is a telephone line, the modem 116 may be a traditional "dial-up" modem. Alternatively, where the connection 121 is a high capacity (eg: cable) connection, the modem 116 may be a broadband modem. A wireless modem may also be used for wireless connection to the network 120. The computer module 101 typically includes at least one processor unit 105, and a memory unit 106 for example formed from semiconductor random access memory (RAM) and semiconductor read only memory (ROM). The module 101 also includes a number of input/output (I/O) interfaces including an audio-video interface 107 that couples to the video display 114, loudspeakers 117 and microphone 180, an I/O interface 113 for the keyboard 102, mouse 103, scanner 126, camera 127 and optionally a joystick (not illustrated), and an interface 108 for the external modem 116. In some implementations, the modem 116 may be incorporated within the computer module 101, for example within the interface 108. The computer module 101 also has a local network interface 1 1 which, via a connection 123, permits coupling of the pixel rendering system 100 to a local computer network 122, known as a Local Area Network (LAN). As also illustrated, the local network 122 may also couple to the wide network 120 via a connection 124, which would typically include a so-called "firewall" device or device of similar functionality. The interface I I 1 may be formed by an Ethernet'i circuit card, a BluetoothTM wireless arrangement or an IEEE 802.11 wireless arrangement. ) The interfaces 108 and 113 may afford either or both of serial and parallel connectivity, the former typically being implemented according to the Universal Serial Bus (USB) standards and having corresponding USB connectors (not illustrated). Storage devices 109 are provided and typically include a hard disk drive (HDD) 110. Other storage devices such as a floppy disk drive and a magnetic tape drive (not illustrated) may also be used. An optical disk 5 drive 112 is typically provided to act as a non-volatile source of data. Portable memory devices, such optical disks (eg: CD-ROM, DVD), USB-RAM, and floppy disks for example may then be used as appropriate sources of data to the system 100. The components 105 to 113 of the computer module 101 typically communicate via an interconnected bus 104. Examples of computers on which the described arrangements can be 0 practised include IBM-PC's and compatibles, Sun Sparcstations, Apple MacTM or alike computer systems evolved therefrom. Methods described below may be implemented using the system 100 wherein the processes of Figs. 4 to 6, to be described, may be implemented as one or more software application -7- 882425_Final(1848794vI) programs 133 executable within the system 100. In particular, the steps of the described methods are effected by instructions 131 in the software that are carried out within the system 100. The software instructions 131 may be formed as one or more code modules, each for performing one or more particular tasks. The software may also be divided into two separate parts, in which a first part and the corresponding code modules performs the described methods and a second part and the corresponding code modules manage a user interface between the first part and the user. The software may be stored in a computer readable medium, including the storage devices described below, for example. The software is loaded into the system 100 from the computer readable medium, and then executed by the system 100. A computer readable medium having such software or computer program recorded on it is a computer program product. The use of the computer program product in the system 100 preferably effects an advantageous apparatus for implementing the described methods. The software 133 is typically stored in the HDD 110 or the memory 106. The software is loaded into the system 100 from a computer readable medium, and then executed by the system 100. Thus for example the software may be stored on an optically readable CD-ROM medium 125 that is read by the optical disk drive 112. A computer readable medium having such software or computer program recorded on it is a computer program product. The use of the computer program product in the computer system 100 preferably effects an advantageous apparatus for implementing the described methods. In some instances, the application programs 133 may be supplied to the user encoded on one or more CD-ROM 125 and read via the corresponding drive 12, or alternatively may be read by the user from the networks 120 or 122. Still further, the software can also be loaded into the system 100 from other computer readable media. Computer readable storage media 5 refers to any storage medium that participates in providing instructions and/or data to the system 100 for execution and/or processing. Examples of such storage media include floppy disks, magnetic tape, CD-ROM, a hard disk drive, a ROM or integrated circuit, USB memory, a magneto-optical disk, or a computer readable card such as a PCMCIA card and the like, whether or not such devices are internal or external of the computer module 101. Examples of 0 computer readable transmission media that may also participate in the provision of software, application programs, instructions and/or data to the computer module 101 include radio or infra-red transmission channels as well as a network connection to another computer or -8- 882425_Final(1848794vl) networked device, and the Internet or Intranets including e-mail transmissions and information recorded on Websites and the like. The second part of the application programs 133 and the corresponding code modules mentioned above may be executed to implement one or more graphical user interfaces (GUIs) to be rendered or otherwise represented upon the display 114. Through manipulation of typically the keyboard 102 and the mouse 103, a user of the system 100 and the application may manipulate the interface in a functionally adaptable manner to provide controlling commands and/or input to the applications associated with the GUI(s). Other forms of functionally adaptable user interfaces may also be implemented, such as an audio interface utilizing speech prompts output via the loudspeakers 117 and user voice commands input via the microphone 180. Fig. I B is a detailed schematic block diagram of the processor 105 and a "memory" 134. The memory 134 represents a logical aggregation of all the memory modules (including the HDD 109 and semiconductor memory 106) that can be accessed by the computer module 10 1 in Fig. IA. When the computer module 101 is initially powered up, a power-on self-test (POST) program 150 executes. The POST program 150 is typically stored in a ROM 149 of the semiconductor memory 106. A hardware device such as the ROM 149 is sometimes referred to as firmware. The POST program 150 examines hardware within the computer module 101 to ensure proper functioning, and typically checks the processor 105, the memory (109, 106), and a basic input-output systems software (BIOS) module 151, also typically stored in the ROM 149, for correct operation. Once the POST program 150 has run successfully, the BIOS 151 activates the hard disk drive 110. Activation of the hard disk drive I10 causes a bootstrap loader program 152 that is resident on the hard disk drive 110 to execute via the 5 processor 105. This loads an operating system 153 into the RAM memory 106 upon which the operating system 153 commences operation. The operating system 153 is a system level application, executable by the processor 105, to fulfil various high level functions, including processor management, memory management, device management, storage management, software application interface, and generic user interface. 0 The operating system 153 manages the memory (109, 106) in order to ensure that each process or application running on the computer module 101 has sufficient memory in which to execute without colliding with memory allocated to another process. Furthermore, the different types of memory available in the system 100 must be used properly so that each -9- 882425_Final(1848794v 1) process can run effectively. Accordingly, the aggregated memory 134 is not intended to illustrate how particular segments of memory are allocated (unless otherwise stated), but rather to provide a general view of the memory accessible by the system 100 and how such is used. The processor 105 includes a number of functional modules including a control unit 139, an arithmetic logic unit (ALU) 140, and a local or internal memory 148, sometimes called a cache memory. The cache memory 148 typically includes a number of storage registers 144 146 in a register section. One or more internal busses 141 functionally interconnect these functional modules. The processor 105 typically also has one or more interfaces 142 for communicating with external devices via the system bus 104, using a connection 118. The application program 133 includes a sequence of instructions 131 that may include conditional branch and loop instructions. The program 133 may also include data 132 which is used in execution of the program 133. The instructions 131 and the data 132 are stored in memory locations 128-130 and 135-137 respectively. Depending upon the relative size of the instructions 131 and the memory locations 128-130, a particular instruction may be stored in a single memory location as depicted by the instruction shown in the memory location 130. Alternately, an instruction may be segmented into a number of parts each of which is stored in a separate memory location, as depicted by the instruction segments shown in the memory locations 128-129. In general, the processor 105 is given a set of instructions which are executed therein. The processor 105 then waits for a subsequent input, to which the processor 105 reacts to by executing another set of instructions. Each input may be provided from one or more of a number of sources, including data generated by one or more of the input devices 102, 103, data received from an external source across one of the networks 120, 102, data retrieved from 5 one of the storage devices 106, 109 or data retrieved from a storage medium 125 inserted into the corresponding reader 112. The execution of a set of the instructions may in some cases result in output of data. Execution may also involve storing data or variables to the memory 134. The described methods use input variables 154 that are stored in the memory 134 in 0 corresponding memory locations 155-158. The described methods produce output variables 161 that are stored in the memory 134 in corresponding memory locations 162-165. Intermediate variables may be stored in memory locations 159, 160, 166 and 167. - 10- 882425_Final(1848794v1) The register section 144-146, the arithmetic logic unit (ALU) 140, and the control unit 139 of the processor 105 work together to perform sequences of micro-operations needed to perform "fetch, decode, and execute" cycles for every instruction in the instruction set making up the program 133. Each fetch, decode, and execute cycle comprises: (a) a fetch operation, which fetches or reads an instruction 131 from a memory location 128; (b) a decode operation in which the control unit 139 determines which instruction has been fetched; and (c) an execute operation in which the control unit 139 and/or the ALU 140 execute the instruction. Thereafter, a further fetch, decode, and execute cycle for the next instruction may be executed. Similarly, a store cycle may be performed by which the control unit 139 stores or writes a value to a memory location 132. Each step or sub-process in the processes of Figs. 4 to 6 is associated with one or more segments of the program 133, and is performed by the register section 144-147, the ALU 140, and the control unit 139 in the processor 105 working together to perform the fetch, decode, and execute cycles for every instruction in the instruction set for the noted segments of the program 133. The described methods may alternatively be implemented in dedicated hardware such as ) one or more integrated circuits performing the functions or sub functions of the methods. Such dedicated hardware may include graphic processors, digital signal processors, or one or more microprocessors and associated memories. The system 100 may be used for rendering computer graphic object images. The computer module 101 is connected to the printing system 115 via the network 122. The local network 5 122 may have multiple personal computers (not shown) connected to the network 122. Alternatively, the printing system 115 may be connected directly to the computer module 101 via the I/O interface 108. As seen in Fig. IC, the printing system 115 comprises a controller processor 170 for executing a controlling program 171 resident in memory 190. The printing system 115 also 0 comprises a pixel rendering apparatus 180 and a printer engine 195. The components 180, 190 and 195 typically communicate via an interconnected bus 175. In the arrangement of Figs. IA to I C, the pixel rendering apparatus 180 is in the form of an application-specific integrated circuit (ASIC) coupled via the bus 175 to the controller processor 170, and the - 11- 882425_Final(1848794v]) printer engine 195. However, the pixel rendering apparatus 180 may also be implemented in software executed in the controller processor 170. A method 1100 of rendering one or more portions of a document will now be described in detail with reference to Fig. 11. As described below, each portion of the document is a page of the document. Alternatively, each portion of the document may be a fraction of a page or one or more objects of a page. The method 1100 may be implemented as software resident in the memory 190 of the printing system 115 and being controlled in its execution by the controller processor 170. In the system 100, the application program 133 creates page-based documents where each page contains objects such as text, lines, fill regions, and image data. The application program 133 sends a high level description of a page (e.g., a PDL file) to the controlling program 171 being executed by the controller processor 170 of the printer system 160 via the network 122. The method 1100 begins at step 1101 where the controlling program 171 receives the high level description of the page. Then at the next step 1103, the controlling program 171 generates an intermediate representation for the page, and provides the intermediate representation to the pixel rendering apparatus 180. A method 1200 of generating an intermediate page representation, as executed at step 1103, will be described below with reference to Fig. 12. ) The controlling program 171 is also responsible for providing memory 190 for the pixel rendering apparatus 180, initialising the pixel rendering apparatus 180, and instructing the pixel rendering apparatus 180 to start rendering the page. At the next step 1105, the pixel rendering apparatus 180 uses the intermediate page representation to render the page to pixels. The output of the pixel rendering apparatus 180 is 5 colour pixel data, which may be used by the printer engine 195. The method 1200 of generating an intermediate page representation, as executed at step 1103, will now be described below with reference to Fig. 12. The method 1200 may be implemented as software, in the form of the controlling program 171, resident in the memory 190 of the printing system 115 and being controlled in its execution by the controller 0 processor 170. The method 1200 begins at step 1201, when the controlling program 171, under execution of the controller processor 170, receives the description of the page from the software application 133. At step 1201,'the controlling program 171 converts objects in the page - 12- 882425_Final(1848794vl) description into an intermediate page representation called a display list. At the next step 1203, the input display list is used by the controlling program 171 to generate a further intermediate page representation. The format of the intermediate page representation generated at step 1203 is selected from a set of possible intermediate page representations. The method 1200 continues at the next step 1205, where the controlling program 171, stores the intermediate page representation in a section of the memory 190 called an intermediate format queue 420 (see Fig. 4) which will described in detail below. In order to select the intermediate page representation for a particular page, the controlling program 171 performs the step of determining workload balance of the rendering system 100 based on a proportion of memory currently available in the intermediate format queue 420. The controlling program 171 also performs the step of determining the capability of the printer engine 195 to render the page in real time by the pixel rendering apparatus 180. The controlling program 171 also performs the step of determining complexity of the page based on the display list representation of the page. These measures are then used to determine the appropriate format of the intermediate page representation for the page description. A method 500 of determining the format of the intermediate page representation for the page description, as executed at step 1203, will be described below with reference to Fig. 5. The intermediate page representations generated at step 1203 may be in one of at least two formats being a vectorised display list format or a fillmap format. An intermediate page representation, in the form of a vectorised display list representation of a page, consists of a set of Y-sorted edges, each of which refers to a priority level. Each priority level has an associated level structure. Each level structure refers to one 5 or more fills. These fills are used to generate pixel colour information for the corresponding level. Each level structure also contains information such as a winding rule used to control level activation, and a raster operation used to combine colour data generated for the level with colour data generated for other levels on the page. A method 1300 of generating a vectorised display list, as may be executed at step 0 1203, will now be described with reference to Fig. 13. The method 1300 may be implemented as software, in the form of the controlling program 171, resident in the memory 190 and being controlled in its execution by the controller processor 170. The vectorised display list may be quickly generated from an input display list using relatively modest processing resources. - 13- 882425_Final(1848794v1) The method 1300 begins at step 1301, where the controlling program 171 creates a priority level and associated level structure for each object in the input display list, and initialised using object fills and object data associated with the object. At the next step 1303, each object path of the controlling program 171 is converted into a set of vectors each of which refers to a priority level of the object. Then at the next step 1305, the set of vectors for all objects on the page are placed in the vectorised display list. These vectors are first sorted by starting scanline (Y) and secondly by starting x-position (X). Generating the vectorised display list does not require a large amount of processing by the controller processor 170. The vectorised display list may be used to render the page. As each scanline of the page is processed in ascending Y-order, edge vectors active on the scanline are determined, and an x-position where the vector intersects with the scanline is determined. Each edge either activates or deactivates a referenced priority level associated with the edge. The set of active edges are processed in ascending x-order along the given scanline, and for each edge crossing a set of active priority levels is determined. If a priority level is active for a given pixel, then the priority level contributes to generating colour data for that pixel. The set of active priority levels is used to generate pixel colour information for all pixels between a current edge crossing and a next edge crossing. Rendering the vectorised display list can require a large amount of processing by the pixel rendering apparatus 180. An intermediate page representation in the form of a fillmap representation of a page comprises non-overlapping regions defined by pixel aligned edges. Each region refers to a fill compositing sequence, which is used to generate colour information for every pixel in the region. A fill compositing sequence is a sequence of Z-ordered entries. Each entry references level information which contains attributes such as a fill and a raster operation. A fill is used to determine an output pixel colour at any given location in the page. A raster operation 5 specifies how to mix the colour data produced by the fill with colour data produced by other fill compositing sequence entries. Fig. 7 shows an exemplary page 710 that contains three (3) objects - an opaque object with a ternary raster operation 720, an opaque low resolution source bitmap object 730, and a semi-transparent flat object 740. A tiled fillmap representation 810 of the example page 710 D is shown in Fig. 8. Fillmap tile 820 is at position "(1, 2)". The position, (1, 2), is expressed in (column, row) form. The contents of fillmap tile 820 are shown expanded on the right hand side of Fig. 8. Four regions 830, 840, 850 and 860 in fillmap tile 820 reference the fill compositing sequences with indices one, two, three, and four, respectively. - 14- 882425_Final(1848794v]) Level Information for each object 720, 730 and 740 in the example page 710 is shown in Fig. 9. Level Info #1 910 corresponds to the opaque object 720 requiring a ternary raster operation. Level Info #1 910 has a flat colour source fill 915, a bitmap pattern fill 917, and the raster operation 919 is a ternary operation labelled 'ROP3'. Level Info #2 920 corresponds to the opaque low resolution source bitmap object 730. Level Info #2 920 contains a bitmap source fill 925, and has the binary raster operation "copypen" 927. Level Info #3 930 corresponds to the semi-transparent flat object 740. Level Info #3 930 contains a flat source fill 935, and has the binary raster operation "over" 937. Fig. 10 shows a table of fill compositing sequences 1010 for the tiled fillmap 810. A method 1400 of generating the intermediate page representation in the form of the fillmap representation will now be described with reference to Fig. 14. The method 1400 may be implemented as software, in the form of the controlling program 171, resident in the memory 190 and being controlled in its execution by the controller processor 170. In order to generate the intermediate page representation in the form of the fillmap representation, the fill compositing sequence for each pixel in the page is determined. The method 1400 begins at step 1401, where the controlling program 171 scan-converts each object in the input display list. At the next step 1403, the regions where objects overlap are determined by the controlling program 171. Then at the next step 1405, the fill compositing sequence for the overlapping regions is determined. This results in a fillmap representation consisting of non-overlapping regions. Generating the fillmap representation can require a large amount of processing by the controller processor 170. Rendering the fillmap, requires relatively modest processing resources. To render the fillmap for each page pixel, the pixel rendering apparatus 180 determines the fill compositing sequence that is active for that pixel, and uses the referenced fill compositing sequence to 5 generate the pixel color information. Each page pixel belongs to a single region, which references a single fill compositing sequence. Rendering the fillmap does not require a large amount of processing by the pixel rendering apparatus 180. Fig 2 is a schematic block diagram of a conventional printing system 200, where a vectorised display list format is used as an intermediate page representation. The printing 0 system 200 comprises a page description input queue 210, a controller processor 240, a vectorised display list spool queue 220, a pixel rendering apparatus 280 and rendered pixels 230. Dashed arrows, such as arrow 201 represent the data flows in the printing system 200. Data from page description queue 210 is read by the controller processor 240. The controller - 15- 882425_Final(1848794v1) processor 240 first uses a display list generation module 245 to read an input page description and create an intermediate page representation in the form of a z-ordered input display list. The input display list is then transformed into vectorised display list spool data using a vectorised display list spool generating module 250. The vectorised display list spool data is placed in the vectorised display list spool queue 220. When the pixel rendering apparatus 280 determines that the vectorised display list spool queue 220 is not empty, the pixel rendering apparatus 280 reads the spool data from the queue 220. The pixel rendering apparatus 180 calls a vectorised display list rendering module 260 to render the spool data to pixels 230. Fig 3 is a schematic block diagram of another conventional printing system 300, where a fillmap is used as an intermediate page representation. The system 300 comprises a page description input queue 310, a controller processor 340, a fillmap spool queue 320, a pixel rendering apparatus 390 and rendered pixels 330. Dashed arrows, such as arrow 301 represent data flows on the printing system 300. The data from page description queue 310 is read by the controller processor 340. The controller processor 340 first uses a display list generation module 345 to read an input page description and create an intermediate page representation in the form of a z-ordered input display list. The z-ordered display list is transformed into fillmap spool data using a fillmap spool generating module 350. The fillmap spool data is placed in the fillmap spool queue 320. When the pixel rendering apparatus 390 determines that the fillmap spool queue 320 is not empty, the pixel rendering apparatus 390 reads the ) fillmap spool data from the queue 320. The pixel rendering apparatus 180 calls a fillmap rendering module 360 to render the spool data to pixels 330. Fig. 4 is a schematic block diagram showing control flows and data flows through the printing system 115 of Figs. IA to IC. Fig. 4 shows a page description input queue 410, the controller processor 170, an intermediate format spool queue 420, the pixel rendering 5 apparatus 180 and rendered pixels 430. As seen in Fig. 4, the printing system 115 comprises a display list generation module 440, an intermediate format determination module 450, a fillmap spool generation module 460 and a vectorised display list generation module 470. The printing system 115 also comprises a fillmap rendering module 480 and a vectorised display list rendering module 490. The modules 440, 450, 460, 470, 480 and 490 may be 30 implemented as software, in the form of the controlling program 171, resident within the memory 190 of the printing system 115 and being controlled in its execution by the controller processor 170. - 16- 882425_Final(1848794vl) Solid arrows of Fig. 4, such as arrow 402 represent the control flows and dashed arrows, such as arrow 401 represent the data flows on the printing system 115. Data from the page description queue 410 is read by the controller processor 170. The controller processor 170 transforms the page description data 410 into an intermediate page representation in the form of intermediate format spool data 420. The intermediate format spool data is read by the pixel rendering apparatus 180. The pixel rendering apparatus 180 processes the intermediate format spool data and produces rendered pixel data 430. The display list generation module 440 sequentially reads the page description input data 410 and buffers the read data in memory 190. The display list generation module 440 then determines when to start generating the intermediate page representation, in the form of a display list, from the data stored in the memory 190. The trigger to start generating the intermediate page representation may be one of the following: a) enough data has been gathered to rasterise a logical unit. The logical unit could be a number of pages constituting a print job, a single page, or a band within a single page; or b) while gathering the page description input data 410, a predetermined information quantity limit has been reached. Such a limit can be expressed by an amount of available memory in dedicated to the storage of the input display list, or number of objects in the input display list, or some other measure, or combination of measures. Once the display list generation module 440 decides to generate the intermediate page representation in the form of the display list, the module 440 passes control to the intermediate format determination module 450. The intermediate format determination module 450 monitors load on the pixel rendering apparatus 180 by checking the memory 5 available to the intermediate format spool queue 420, together with the number of pages in the intermediate format spool queue 420. If the amount of memory available to the intermediate format spool queue 420 is low, or the number of pages in the intermediate format spool queue 420 is high, then the pixel rendering apparatus 180 is deemed to be heavily loaded. Conversely, if the amount of memory available to the intermediate format spool queue 420 0 is high, or the number of pages in the intermediate format spool queue 420 is low, then the pixel rendering apparatus 180 is deemed to be lightly loaded. The intermediate format determination module 450 also determines the complexity of the display list generated by the display list generation module 440. The measure of data complexity can be based, for - 17- 882425_Final(1848794vl) example, on number of objects per page (or number of objects in a portion of a document), density of control points for paths in the input display list, estimated size of the vectorised display list (i.e., representation) per page (i.e., a document portion), number and size of bitmaps. Other complexity measures may be used. The complexity is determined based on the input display list representing the page description input data 410. The intermediate format determination module 450 uses the load on the pixel rendering apparatus 180 and data complexity measures to decide which intermediate format to use for generating the intermediate page representation. If the format of the intermediate page representation is determined to be the fillmap format, the fillmap spool generation module 460 is called to generate the fillmap. The fillmap is then placed in the intermediate format spool queue 420. Otherwise, the vectorised display list spool generation module 470 is called to generate the intermediate page representation in the form of a vectorised display list. The vectorised display list is then placed in the intermediate format spool queue 420. The generation of the intermediate page representation can stall in a case where the intermediate format spool queue 420 is full. In this instance, either the fillmap spool generation module 460 or the vectorised display list generation module 470, which is preparing the intermediate page representation for a separate page, waits until the pixel rendering apparatus 180 makes room in the intermediate format spool queue 420, as described later. The intermediate format spool queue 420 can contain page representations in the form of both fillmap data and vectorised display list data. The type of data in the intermediate format spool queue 420 is identified by a differentiating bit so that pixel sequential rendering apparatus 180 reading the intermediate format spool queue 420 can distinguish between fillmap spool data and vectorised display list spool data. When the pixel rendering apparatus 5 180 determines that the intermediate format spool queue 420 is not empty, the pixel rendering apparatus 180 reads the spool data from the intermediate format queue 420. If the spool data is determined from an associated differentiating bit to be in fillmap format, the fillmap rendering module 480 is called to render the spool data to pixels. Otherwise the vectorised display list rendering module 490 is called to render the spool data to pixels. The pixels are 0 output to the rendered pixel data 430. The method 500 of determining the format of an intermediate page representation, as used by the intermediate format determination module 450, will now be described in more detail. The method 500 may be implemented as software, in the form of the controlling program 171, - 18- 882425_Final(1848794vl) resident within the memory 190 and being controlled in its execution by the controller processor 170. The method 500 performs the step of selecting the appropriate intermediate page representation format depending on the complexity of the page and the workload balance of the rendering system 100. The method 500 has two possible return values. If the method 500 returns VECTORISEDDL, then the method 500 has determined that the appropriate format of the intermediate page representation to use is the vectorised display list page representation. If the method 500 returns FILLMAP, then the method 500 has determined that the appropriate format of the intermediate page representation to use is the fillmap representation. The method 500 begins at step 520, where the controlling program 171 determines complexity of the input display list. The complexity of the display list can vary from low complexity to high complexity. Suitable complexity measures include, for example, the number of objects per page, the density of control points for paths in the input display list, or the amount of memory taken by the input display list, number and size of bitmaps. Other complexity measures may be used. The rendering capabilities of the vectorised display list rendering module 490 and memory limit are considered. In a case where the display list rendering module 490 is unable to render the spool data in real time, then the complexity is set to be equal to the predetermined maximum vectorised display list complexity. Also, in the case when the display list is being converted to the page representation because the memory limit has been reached, the complexity is set to be equal to the predetermined maximum vectorised display list complexity. The maximum vectorised display list complexity will be discussed in more detail below. At the next step 530, the controlling program 171 compares the input display list 5 complexity with a predetermined minimum fillmap complexity. The predetermined minimum fillmap complexity is a predetermined system parameter indicating a complexity measure below which the fillmap page representation may be inefficient. For pages with complexity up to the predetermined minimum fillmap complexity, produced fillmap page representations are generally greater in size than the equivalent vectorised display list page representations, D and generally take longer to generate than the equivalent vectorised display list page representation. If the controlling program 171 determines at step 530 that the input display list complexity is no greater than a minimum fillmap complexity, then the method 500 proceeds to step 560. - 19- 882425_Final(1848794vl) At step 560, the controlling program 171 concludes the method 500 by returning a tag, VECTORISEDDL, indicating that the determined format for the intermediate page representation is vectorised display list. Otherwise, the process method 500 proceeds from step 530 to step 540. At step 540, the controlling program 171 compares the input display list complexity with predetermined maximum vectorised display list complexity. The maximum vectorised display list complexity is a predetermined system parameter indicating the complexity measure above which the vectorised display list intermediate page representation may be inefficient. For such complexity, produced fillmap intermediate page representations are generally smaller in size than an equivalent vectorised display list intermediate page representation, and generally require less memory to generate than an equivalent vectorised display list page representation. If the controlling program 171 determines, at step 540, that the input display list complexity is greater than the maximum vectorised display list complexity, then the method 500 proceeds to step 580. Otherwise, the method 500 proceeds to step 550. At step 580, the controlling program 171 returns a tag, FILLMAP, indicating that the determined format for the intermediate page representation is fillmap. If the controlling program 171 determines, at step 540, that the display list complexity is not greater than the predetermined maximum vectorised display list complexity, then the format of the intermediate page representation cannot be determined by the input display list complexity alone. In this case, the method 500 proceeds from step 540 to step 550. At step 550, the controlling program 171 determines the renderer load. In particular, at step 550, the intermediate format determination module 450 checks the current size of the intermediate format spool queue 420, and compares the current size of the intermediate format spool queue 420 with a predetermined threshold value represented by a tag, FMThreshold. The current 5 size of the intermediate format spool queue 420 is dependent on a number of intermediate page representation (i.e., fillmaps or vectorised display lists) previously stored in the queue 420. The threshold value, FM_Threshold, is a predetermined system parameter indicating the intermediate format spool queue 420 above which the pixel rendering apparatus 180 is deemed to be fully loaded. If the pixel rendering apparatus 180 is fully loaded, then the 0 fillmap format is preferable to the vectorised display list format, as the fillmap format requires less processing by the pixel rendering apparatus 180. Accordingly, the workload balance of the rendering system 100 is determined based on the intermediate format spool queue 420 containing one or more intermediate page representations. -20- 882425_Final(1848794v]) If, at step 570, the current size of the intermediate format spool queue 420 is greater than the predetermined threshold, FMThreshold, then the method 500 concludes at step 580 by returning FILLMAP. Otherwise, the method 500 concludes at step 560 by returning VECTORISEDDL. The controlling program 171 performs the step of generating the intermediate page representation according to the selected intermediate page representation format. When the intermediate format determination module 450 returns FILLMAP, the fillmap spool generation module 460 is called to generate the intermediate page representation. When the intermediate format determination module 450 returns VECTORISEDDL the vectorised display list spool generation module 470 is called to generate the intermediate page representation. The pixel rendering apparatus 180 in conjunction with the printer engine 195, under control of the controller processor 170, together perform the step of rendering a page based on the generated intermediate page representation. In the case where the controlling program 171 is executing on a controller processor 170 which consists of multiple processors, additional optimisation of computing resources is possible. In particular, computing resources may be divided between the fillmap spool generation module 460 and the vectorised display list spool generation module 470 and the fillmap spool generation module 460 and the vectorised display list spool generation module 470 may be operated on separate page representations, or partial page representations, in ) parallel. Equal division of resources will give optimal performance in the case where the two intermediate format spool generation modules 460 and 470 are equally loaded. If a majority of page representations require a fillmap intermediate page representation format, the computing resources may not be optimally utilised, as the computing resources used by the vectorised display list spool generation module 470 will be underutilised, and the 5 computing resources used by the fillmap spool generation module 460 may not be sufficient to generate fillmaps at a speed to maintain the pixel rendering apparatus 180 at full capacity. If a majority of page representations require a vectorised display list intermediate format, equal division of the computing resources will generally be optimally utilised, as although the computing resources used by the fillmap spool generation module 460 will be underutilised, 0 the computing resources used by the vectorised display list spool generation module 470 will be sufficient to generate vectorised display lists at a speed to maintain the pixel rendering apparatus 180 at full capacity. -21- 882425_Final(1848794v 1) Dynamic reallocation of processor resources between the fillmap spool generation module 460 described above and the vectorised display list generation module 470 may be used to re balance processor resources. The fillmap spool generation module 460 may be internally multi-threaded, allowing processor resources to be reassigned from the vectorised display list generation module 470 to the fillmap spool generation module 460 as required. Fig. 6 is a schematic block diagram showing the multi-threaded fillmap spool generation module 460 in more detail. The fillmap spool generation module 460 comprises a fillmap spool generation manager 610 and a fillmap spool merger module 620, which may be implemented as software, in the form of the controlling program 171, resident in the memory 190 and being controlled in its execution by the controlling processor 170. As seen in Fig. 6, fillmap generation processes 600 are responsible for processing the data from the display list generation module 440 and converting the data into fillmap data in spooled format. Fillmap generation processes 600 are allocated as needed by a fillmap spool generation manager 610. i The fillmap spool generation manager 610 is also responsible for distributing the input data 601 among processes 600, synchronizing the execution of processes 600, and controlling the execution of the fillmap spool merger module 620. The fillmap spool merger module 620 gathers partial fillmap data generated by each of the processes 600, and produces the final spool data that goes into the intermediate format spool queue 420. ) ' The task of optimal computing resource utilisation is carried out by the fillmap spool generation manager 610. For example, more than one process 600 may need to be allocated in order to generate fillmap spool data efficiently and the predetermined information quantity limit has been reached while gathering page description input data 210 within the display list generation module 440. In such a case, a logical rendering unit of information (e.g., page, 5 band etc) is split into at least two largely independent sub-units. The fillmap spool generation manager 610 assigns the sub-units to fillmap generation processes 600 as appropriate. The fillmap spool generation manager 610 then controls the execution of processes 600 and also controls how often the fillmap spool merger 620 is executed. Hence, where a single processor used by the fillmap spool generation module 460 is overloaded, processor resources 30 may be re-assigned from the vectorised display list spool generation module 470, allowing the fillmap spool generation module 460 to process pages without stalling the pixel rendering apparatus 180. Industrial Applicability -22- 882425_Final(1848794v1) It is apparent from the above that the arrangements described are applicable to the computer and data processing industries. The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive. In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including", and not "consisting only of'. Variations of the word "comprising", such as "comprise" and "comprises" have correspondingly varied meanings. - 23- 882425_Final(1848794v1)

Claims (13)

1. A method of rendering a portion of a document on a rendering system, said method comprising the steps of: determining complexity of the portion of the document based on a representation of the document portion; determining workload balance of the rendering system; selecting one of a plurality of intermediate representation formats depending on the complexity of the portion of the document and the workload balance of the rendering system; generating an intermediate representation of the portion of the document according to the selected format; and rendering the portion of the document based on the generated intermediate page representation, wherein the workload balance of the rendering system is determined based on a queue of intermediate representations.

2. The method according to claim 1, wherein the workload balance of the rendering system is determined based on a number of intermediate representations previously stored in said queue.

3. The method according to claim 1, wherein the complexity of the portion of the document is determined based on number of objects in the document portion.

4. The method according to claim 1, wherein the complexity of the portion of the document is determined based on density of control points for paths in an input display list. 5

5. The method according to claim 1, wherein the complexity of the portion of the document is determined based on estimated size of the representation of the document portion. 0

6. The method according to claim 1, wherein the complexity of the portion of the document is determined based on a display list representing the document portion. -24- 882425_Final(1848794vl)

7. The method according to claim 1, wherein at least one of the plurality of intermediate representation formats comprises non-overlapping regions defined by pixel aligned edges.

8. The method according to claim 1, wherein the generated intermediate representation is a vectorised display list.

9. The method according to claim 1, wherein the generated intermediate representation is a fillmap.

10. A rendering system for rendering a portion of a document, said system comprising: a memory for storing data and a computer program; and a processor coupled to said memory executing said computer program, said computer program comprising instructions for: determining complexity of the portion of the document based on a representation of the document portion; determining workload balance of the rendering system; selecting one of a plurality of intermediate representation formats depending on the complexity of the portion of the document and the workload balance of the rendering system; generating an intermediate representation of the portion of the document according to the selected format; and rendering the portion of the document based on the generated intermediate page representation, wherein the workload balance of the rendering system is determined based on a queue of intermediate representations. 5

11. An apparatus for rendering a portion of a document on a rendering system, said apparatus comprising: means for determining complexity of the portion of the document based on a representation of the document portion; 0 means for determining workload balance of the rendering system; means for selecting one of a plurality of intermediate representation formats depending on the complexity of the portion of the document and the workload balance of the rendering system; - 25- 882425_Final(1848794vl) means for generating an intermediate representation of the portion of the document according to the selected format; and means for rendering the portion of the document based on the generated intermediate page representation, wherein the workload balance of the rendering system is determined based on a queue of intermediate representations.

12. A computer readable medium, having a program recorded thereon, where the program is configured to make a computer execute a procedure to render a portion of a document on a rendering system, said program comprising: code for determining complexity of the portion of the document based on a representation of the document portion; code for determining workload balance of the rendering system; code for selecting one of a plurality of intermediate representation formats depending on the complexity of the portion of the document and the workload balance of the rendering system; code for generating an intermediate representation of the portion of the document according to the selected format; and rendering the portion of the document based on the generated intermediate page representation, wherein the workload balance of the rendering system is determined based on ) a queue of intermediate representations.

13. A method of rendering a portion of a document on a rendering system, said method being substantially as herein before described with reference to any one of the embodiments as that embodiment is shown in the accompanying drawings. 5 DATED this 17th Day of November 2008 CANON KABUSHIKI KAISHA Patent Attorneys for the Applicant SPRUSON&FERGUSON - 26- 882425_Final(1848794v1)