AU2011265381A1 - Method, apparatus and system for processing patches of a reference image and a target image - Google Patents

Abstract

A method of processing patches of a reference image (e.g., 1010) and a target image (e.g., 1050) is disclosed. A set of reference patches is accessed for the reference image. The reference patches are selected based on content of the reference image (1010). A corresponding set of target patches is identified for the target image (1050) based on 10 respective locations of the corresponding reference patches in the reference image (1010). A transform is determined for relating the reference image (1010) and the target image (1050) based on a shift between at least one target patch located in a first region of the target image (1050) and the corresponding reference patch. The corresponding reference patch is repositioned within the reference image (1010) based on an inverse of the 15 determined transform and the location of a further target patch located partially within the first region of the target image (1050) and partially within a second region of the target image (1050).

Description

S&FRef: P019410 AUSTRALIA PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT Name and Address Canon Kabushiki Kaisha, of 30-2, Shimomaruko 3 of Applicant: chome, Ohta-ku, Tokyo, 146, Japan Actual Inventor(s): Eric Wai-Shing Chong Address for Service: Spruson & Ferguson St Martins Tower Level 35 31 Market Street Sydney NSW 2000 (CCN 3710000177) Invention Title: Method, apparatus and system for processing patches of a reference image and a target image The following statement is a full description of this invention, including the best method of performing it known to me/us: 5845c(5848613_1) - 1 METHOD, APPARATUS AND SYSTEM FOR PROCESSING PATCHES OF A REFERENCE IMAGE AND A TARGET IMAGE FIELD OF INVENTION The present invention relates generally to alignment of images and, in particular, to identifying points and regions in images that are suitable for image alignment. The present invention also relates to a method and apparatus for determining a patch location 5 for relating a reference image and a target image. The present invention also relates to a computer program product including a computer readable medium having recorded thereon a computer program for determining a patch location for relating a reference image and a target image. DESCRIPTION OF BACKGROUND ART 10 There is a general need for measuring output quality of a printing system. Results from such quality measurement may be used to fine-tune and configure parameters of the printing system for improved performance. Traditionally, quality measurement has been performed in an offline fashion through manual inspection of an output print from the printing system. 15 With ever increasing printing speeds and volumes, the need for automated real time detection of print defects to maintain print quality has increased. Timely identification of print defects can allow virtually immediate corrective action, such as re printing, to be taken, which in turn reduces waste in paper and ink or toner, while improving efficiency. Some detection systems can operate independent of user or operator 20 input, and may be considered automatic detection systems. A number of automatic print defect detection systems have been developed. In some arrangements, the systems involve the use of an image acquisition device, such as a CCD (charge-coupled device) camera, to capture or scan an image of a document printout 5845437v1 (P019410_SpeciLodged) -2 (also referred to as an output print). The scan image is then compared to an image (referred to as the original image) of an original source input document. Discrepancies identified during the comparison can be flagged as print defects. Real-time print defect detection for high volume printing is an exceptionally 5 challenging problem because defect detection requires very accurate image alignment between the scan image and the original image, and a very high processing speed. Some existing print defect detection methods are unsuitable for real-time processing because the methods are too slow. Other methods are insufficiently accurate and are sensitive to noise and distortion. The noise and distortion either results in a high rate of false defect 10 detections, or a coarse detection method, where only large defects are detected. Sparse image alignment based on a set of local shift estimates between image patches in an original and scan images is a suitable image alignment method for real-time print defect detection. Such a method involves identifying a set of sparsely located image patches that are centred on strong image alignment structure in the original image, 15 determining a corresponding set of image patches in the scan image spatially, and estimating the spatial relationship between each image patch pair via shift estimation. A global transform, defining the spatial relationship, between contents in the original image and contents in the scan image is derived based on the set of shift estimates. The above sparse image alignment method assumes that an image patch with 20 strong image alignment structure in the original image and an image patch at the corresponding spatial location in the scan image have substantial common image structure, representing an overlap area between the patch pair. The overlap area is needed for determining the spatial relationship between contents in the patches. However, the overlap area is likely to be reduced by relative movements between an image acquisition device 5845437v1 (P019410_Speci_Lodged) -3 and a printout. As a result of the reduction in common image structure between patches, the accuracy of shift estimation between the patches is reduced. SUMMARY OF THE [NVENTION It is an object of the present invention to substantially overcome, or at least 5 ameliorate, one or more disadvantages of existing arrangements. Disclosed are arrangements, referred to as Adaptive Print Verification (APV) arrangements, which dynamically adapt an image alignment parameter according to local shift measurements between an original source document image and a digitally captured image of the printed source document, in order to establish correspondence between the 10 images. The arrangements enable accurate image comparison to be performed for the purpose of print defect detection in real-time. According to one aspect of the present disclosure there is provided a method of processing patches of a reference image and a target image, said method comprising: accessing a set of reference patches for the reference image, said reference patches 15 being selected based on content of the reference image; identifying a corresponding set of target patches for the target image based on respective locations of the corresponding reference patches in the reference image; determining a transform for relating the reference image and the target image based on a shift between at least one target patch located in a first region of the target 20 image and the corresponding reference patch; and repositioning the corresponding reference patch within the reference image based on an inverse of the determined transform and the location of a further target patch located partially within the first region of the target image and partially within a second region of the target image. 5845437vi (P019410_Speci_Lodged) -4 According to another aspect of the present disclosure there is provided an apparatus for processing patches of a reference image and a target image, said apparatus comprising: means for accessing a set of reference patches for the reference image, said 5 reference patches being selected based on content of the reference image; means for identifying a corresponding set of target patches for the target image based on respective locations of the corresponding reference patches in the reference image; means for determining a transform for relating the reference image and the target 10 image based on a shift between at least one target patch located in a first region of the target image and the corresponding reference patch; and means for repositioning the corresponding reference patch within the reference image based on an inverse of the determined transform and the location of a further target patch located partially within the first region of the target image and partially within a 15 second region of the target image. According to still another aspect of the present disclosure there is provided a system for processing patches of a reference image and a target image, said system comprising: a memory for storing data and a computer program; 20 a processor coupled to said memory for executing said computer program, said program comprising instructions for: accessing a set of reference patches for the reference image, said reference patches being selected based on content of the reference image; 5845437v1 (P019410_SpeciLodged) -5 identifying a corresponding set of target patches for the target image based on respective locations of the corresponding reference patches in the reference image; determining a transform for relating the reference image and the target 5 image based on a shift between at least one target patch located in a first region of the target image and the corresponding reference patch; and repositioning the corresponding reference patch within the reference image based on an inverse of the determined transform and the location of a further target patch located partially within the first region of the target image and partially 10 within a second region of the target image. According to still another aspect of the present disclosure there is provided a computer readable medium having software stored thereon for processing patches of a reference image and a target image, said program comprising: code for accessing a set of reference patches for the reference image, said 15 reference patches being selected based on content of the reference image; code for identifying a corresponding set of target patches for the target image based on respective locations of the corresponding reference patches in the reference image; code for determining a transform for relating the reference image and the target 20 image based on a shift between at least one target patch located in a first region of the target image and the corresponding reference patch; and code for repositioning the corresponding reference patch within the reference image based on an inverse of the determined transform and the location of a further target patch located partially within the first region of the target image and partially within a second 25 region of the target image. 5845437v1 (P019410_SpeciLodged) -6 Other aspects of the invention are also disclosed. BRIEF DESCRIPTION OF THE DRAWINGS At least one embodiment of the present invention will now be described with reference to the following drawings, in which: 5 Fig. 1 is a top-level flow-chart showing a method of determining unexpected differences in a printed page; Fig. 2 is a flow-chart showing a method of detecting defects, as executed in the method of Fig. 1; Fig. 3 is a diagrammatic overview of a print system on which the method of Fig. 1 10 may be practiced; Fig. 4 is a flow-chart showing a method of aligning image bands, as executed in the method of Fig. 2; Fig. 5 is a flow-chart showing a method of comparing image bands, as executed in the method of Fig. 2; 15 Fig. 6 is a flow-chart showing a method of comparing image tiles, as executed in the method of Fig. 5; Fig.7 shows a graphical view of how the steps of Fig. 2 may be performed in parallel; Fig. 8 is a flow-chart showing a method of processing target patches spread across 20 two consecutive bands, as executed in the method of Fig. 4; Fig. 9 shows different patches in a target image; Fig. 10A shows an example reference image; Fig. 10B shows an example of a target image; Fig. 11A shows an example of an expected image band; 25 Fig. 11B shows an example of a scan image band; 5845437v1 (P01941 OSpeciLodged) -7 Fig. 12A, 12B and 12C show an example where consecutive bands in a target image contain a partially overlapping region; Figs. 13A and 13B collectively form a schematic block diagram representation of an electronic device upon which described arrangements can be practised; 5 Fig. 14 shows the details of the print verification unit of Fig. 3; and Fig. 15 is a flow-chart showing a method of estimating shift as executed in the method of Fig. 4. DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION Where reference is made in any one or more of the accompanying drawings to 10 steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears. It is to be noted that the discussions contained in the "Background" section and that above relating to prior art arrangements relate to discussions of devices which may 15 form public knowledge through their use. Such discussions should not be interpreted as a representation by the present inventor(s) or the patent applicant that such devices in any way form part of the common general knowledge in the art. With reference to Fig. 1, an output print 163 of a print step 130 performed by a print system 300, as seen in Fig. 3, will not, in general, precisely reflect an associated 20 source input document 166. This is because the print step 130, through which the source input document 166 is processed to produce output print 163, introduces some changes to the source input document 166 by virtue of the physical characteristics of a print engine 329 and associated movement and operations inside the print system 300. Furthermore, if the source input document 166 is compared with a scan image 164 of the output print 163, 25 the physical characteristics of a scan step 140 by which the scan image 164 is formed also 5845437v1 (P019410_SpecLodged) contribute changes to the source input document 166. As a result of such cumulative changes, a direct comparison between the source input document 166 and the scan image 164 is unsuitable for assessing the quality of the output print 163. In order to effectively assess the quality of the output print 163, the changes may be accounted for before the 5 source input document 166 is compared to the scan image 164. In going from the source input document 166 to the scan image 164, each pixel in the source input document 166 is expected to undergo changes or transformations in both the intensity domain and in the spatial domain. Intensity transformations are mainly caused by physical characteristics of the 10 various processes through which the source input document 166 passes in producing the output print 163. Intensity transformations tend to produce expected differences and are dependent on operating conditions of the print system 300. Those operating conditions are often represented using a set of parameters in the form of a printer model. Spatial transformations are mainly caused by mechanical movements inside the 15 print system 300. Spatial transformations may severely affect pixel placement accuracy on a print medium and presents a major challenge to real-time print output checking. There may also be further differences between the scan image 164 and the source input document 166, which are not accounted for by the transformations that occur within the print step 130 and the scan step 140. Such further differences are referred to as 20 unexpected differences, and these can be amenable to corrective action. The unexpected differences are also referred to as "print defects". The disclosed Adaptive Print Verification (APV) arrangements discriminate between expected and unexpected differences by dynamically adapting to the transformations that occur within the print step 130 and the scan step 140. By generating 25 an "expected print result" in accordance with the transformations, it is possible to check 5845437v1 (P019410_SpeciLodged) -9 that the output meets expectations with a reduced danger of falsely detecting an otherwise expected change as a defect (known as "false positives"). In one of the disclosed APV arrangements, the output print 163 produced by the print step 130 of the print system 300 from the source document 166 is scanned to produce 5 digital representation 164 (hereinafter referred to as a "scan image") of the output print 163. In order to detect print errors in the output print 163, a set of parameters which model characteristics of the print mechanism of the print system 300 are firstly determined, and values for the parameters are determined based on operating condition data for at least a part of the print system 300. The operating condition data may be determined from the 10 print system 300 or from other sources, such as for example, external sensors adapted to measure environmental parameters such as humidity and/or temperature in which the print system 300 is located. The value associated with each of the parameters is used to generate, by modifying a rendering 160 of the source document 166, an expected digital representation of the output print 163. The expected digital representation takes into 15 account the physical characteristics of the print system 300, thereby effectively compensating for output errors associated with operating conditions of the print system (these output errors being expected differences) in the intensity domain. The generated expected digital representation is then aligned to the scan image 164 to remove any spatial transformations caused during printing or scanning. The aligned expected digital 20 representation is compared to the scan image 164 of the output print 163 in order to detect unexpected differences (i.e. differences not attributable to the physical characteristics of the print system 300 or spatial transformations caused during printing or scanning) these being identified as print errors in the output of the print system 300. Fig. 3 is a diagrammatic overview of the important components of the print 25 system 300 on which method 100 of Fig. 1 may be practiced. An expanded depiction is 5845437v1 (P019410_SpeciLodged) - 10 shown in Figs. 13A and 13B. In particular, Fig. 3 is a schematic block diagram of the print system 300 with which the arrangements may be practiced. The print system 300 comprises a central processing unit 301 connected to four chromatic image forming units 302, 303, 304, and 305. For ease of description, chromatic colourant substances are each 5 referred to simply as respective colour space -"colourant". In the example depicted in Fig. 3, an image forming unit 302 dispenses cyan colourant from a reservoir 307, an image forming unit 303 dispenses magenta colourant from a reservoir 308, an image forming unit 304 dispenses yellow colourant from a reservoir 309, and an image forming unit 305 dispenses black colourant from a reservoir 310. In the example of Fig. 3, there are four 10 chromatic image forming units, creating images with cyan, magenta, yellow, and black (known as a CMYK printing system). Printers with less or more chromatic image forming units and different types of colourants are also available. Central processing unit 301 communicates with the four image forming units 302 305 by a data bus 312. Using the data bus 312, the central processing unit 301 may receive 15 data from, and issue instructions to, (a) the image forming units 302-305, as well as (b) an input paper feed mechanism 316, (c) an output visual display and input control user interface 320, and (d) a memory 323 used to store information needed by the print system 300 during its operation. The central processing unit 301 also has a link or interface 322 to a device 321 that acts as a source of data to print. The data source 321 may, for example, 20 be a personal computer, the Internet, a Local Area Network (LAN), or a scanner, etc., from which the central processing unit 301 receives electronic information to be printed, this electronic information being the source document 166 in Fig. 1. The data to be printed may be stored in the memory 323. Alternatively, the data source 321 to be printed may be directly connected to the data bus 312. 5845437v1 (P019410_SpecLodged) - 11 When the central processing unit 301 receives data to be printed, instructions are sent to the input paper feed mechanism 316. The input paper feed mechanism 316 takes a sheet of paper 319 from an input paper tray 315, and places the sheet of paper 319 on a transfer belt 313. The transfer belt 313 moves in the direction of an arrow 314 (from right 5 to left horizontally in Fig. 3), to cause the sheet of paper 319 to sequentially pass by each of the image forming units 302-305. As the sheet of paper 319 passes under each image forming unit 302, 303, 304, 305, the central processing unit 301 causes the image forming units 302, 303, 304, or 305 to write an image to the sheet of paper 319 using a particular colourant of the image forming unit in question. After the sheet of paper 319 has passed 10 under all of the image forming units 302-305, a full colour image will have been placed on the sheet of paper 319. For the case of a fused toner printer, the sheet of paper 319 then passes by a fuser unit 324 that affixes the colourants to the sheet of the paper 319. The image forming units and the fusing unit 324 are collectively known as a print engine 329. The output print 163 15 of the print engine 329 may then be checked by a print verification unit 330 (also referred to as a print defect detector system). The sheet of paper 319 is then passed to a paper output tray 317 by an output paper feed mechanism 318. The printer architecture in Fig. 3 is for illustrative purposes only. Many different printer architectures may be adapted for use by the disclosed arrangements. In one 20 example, the arrangements may take the action of sending instructions to the print system 300 to reproduce the output print 163 if one or more errors are detected. Figs. 13A and 13B collectively form a schematic block diagram representation of the print system 300 in more detail. As seen in Figs. 13A and 13B, the print system 300 comprises embedded components, upon which methods to be described are desirably 25 practiced. The print system 300 in the arrangement of Figs. 13A and 13B is a printer in 5845437v1 (PO19410_SpeciLodged) - 12 which processing resources are limited. Nevertheless, one or more of the APV functional processes may alternately be performed on higher-level devices such as desktop computers, server computers, and other such devices with significantly larger processing resources, which are connected to the printer. 5 As seen in Fig. 13A, the print system 300 comprises an embedded controller 1302. Accordingly, the print system 300 may be referred to as an "embedded device." In the present example, the controller 1302 has the central processing unit (or "processor") 301 which is bi-directionally coupled to the memory (internal storage module) 323 (see Fig. 3). The memory 323 may be formed from non-volatile semiconductor read only 10 memory (ROM) 1360 and semiconductor random access memory (RAM) 1370, as seen in Fig. 20B. The RAM 1370 may be volatile, non-volatile or a combination of volatile and non-volatile memory. The print system 300 includes a display controller 1307 (which is an expanded depiction of the output visual display and input controls 320), which is connected to a 15 video display 1314, such as a liquid crystal display (LCD) panel or the like. The display controller 1307 is configured for displaying graphical images on video display 1314 in accordance with instructions received from the embedded controller 1302, to which the display controller 1307 is connected. The print system 300 also includes user input devices 1313 (which is an expanded 20 depiction of the output visual display and input controls 320) which are typically formed by keys, a keypad or like controls. In some arrangements, the user input devices 1313 may include a touch sensitive panel physically associated with the display 1314 to collectively form a touch-screen. Such a touch-screen may thus operate as one form of graphical user interface (GUI) as opposed to a prompt or menu driven GUI typically used with keypad 25 display combinations. Other forms of user input devices may also be used, such as a 5845437v1 (P019410_SpeciLodged) - 13 microphone (not illustrated) for voice commands or a joystick/thumb wheel (not illustrated) for ease of navigation about menus. As seen in Fig. 13A, the print system 300 also comprises a portable memory interface 1306, which is coupled to the processor 301 via a connection 1319. The portable 5 memory interface 1306 allows a complementary portable memory device 1325 to be coupled to the print system 300 to act as a source or destination of data or to supplement the memory (or internal storage module) 323. Examples of such interfaces permit coupling with portable memory devices such as Universal Serial Bus (USB) memory devices, Secure Digital (SD) cards, Personal Computer Memory Card International Association 10 (PCMIA) cards, optical disks and magnetic disks. The print system 300 also has a communications interface 1308 to permit coupling of the print system 300 to a computer or communications network 1320 via a connection 1321. The connection 1321 may be wired or wireless. For example, the connection 1321 may be radio frequency or optical. An example of a wired connection includes Ethernet. 15 Further, an example of wireless connection includes BluetoothTm type local interconnection, Wi-Fi (including protocols based on the standards of the IEEE 802.11 family), Infrared Data Association (IrDa) and the like. The source device 321 may, as in the present example, be connected to the processor 301 via the network 1320. The print system 300 is configured to perform some or all of the APV sub-steps in 20 the method 100 of Fig. 1. The embedded controller 1302, in conjunction with the print engine 329, the print verification unit 330 which are depicted by a special function unit 13 10, is provided to perform that method 100. The special function unit 1310 is connected to the embedded controller 1302. The APV methods described hereinafter may be implemented using the embedded 25 controller 1302, where the processes of Figs. 1-2, 4-5, 8, and 15 may be implemented as at 5845437v1 (P019410_SpeciLodged) -14 least one or more APV software application programs 1333 executable within the embedded controller 1302. The APV software application programs 1333 may be functionally distributed among the functional elements in the print system 300, as shown in the example in Fig. 14 5 where at least some code modules 1403 of the APV software application program 1333 are stored within a memory 1404 of the print verification unit 330. The print system 300 of Fig. 13A implements the described APV methods. In particular, with reference to Fig. 13B, the steps of the described APV methods are effected by instructions in the software application program 1333 that are carried out within the 10 controller 1302 possibly in conjunction with processor 1406. The software instructions may be formed as one or more code modules, each for performing one or more particular tasks. The software may also be divided into two separate parts, in which a first part and the corresponding code modules performs the described APV methods and a second part and the corresponding code modules manage a user interface between the first part and the 15 user. The software 1333 of the embedded controller 1302 is typically stored in the non volatile ROM 2060 of the internal storage module 323. The software 1333 stored in the ROM 1360 can be updated when required from a computer readable medium. The software 1333 can be loaded into and executed by the processor 301, possibly in 20 conjunction with the processor 1406. In some instances, the processor 301 may execute software instructions that are located in RAM 1370. Software instructions may be loaded into the RAM 1370 by the processor 301 initiating a copy of one or more code modules from ROM 1360 into RAM 1370. Alternatively, the software instructions of one or more code modules may be pre-installed in a non-volatile region of RAM 1370 by a 5845437v1 (P019410_SpeciLodged) - 15 manufacturer. After one or more code modules have been located in RAM 1370, the processor 301 may execute software instructions of the one or more code modules. The APV application program 1333 is typically pre-installed and stored in the ROM 1360 by a manufacturer, prior to distribution of the print system 300. However, in 5 some instances, the application programs 1333 may be supplied to the user encoded on one or more CD-ROM (not shown) and read via the portable memory interface 1306 of Fig. 13A prior to storage in the internal storage module 323 or in the portable memory 1325. In another alternative, the software application program 1333 may be read by the processor 301 from the network 1320, or loaded into the controller 1302 or the portable storage 10 medium 1325 from other computer readable media. Computer readable storage media refers to any storage medium that participates in providing instructions and/or data to the controller 2002 for execution and/or processing. Examples of such storage media include floppy disks, magnetic tape, CD-ROM, a hard disk drive, a ROM or integrated circuit, USB memory, a magneto-optical disk, flash memory, or a computer readable card such as 15 a PCMCIA card and the like, whether or not such devices are internal or external of the print system 300. Examples of computer readable transmission media that may also participate in the provision of software, application programs, instructions and/or data to the print system 300 include radio or infra-red transmission channels as well as a network connection to another computer or networked device, and the Internet or Intranets 20 including e-mail transmissions and information recorded on Websites and the like. A computer readable medium having such software or computer program recorded on it is a computer program product. The second part of the APV application programs 1333 and the corresponding code modules mentioned above may be executed to implement one or more graphical user 25 interfaces (GUIs) to be rendered or otherwise represented upon the display 1314 of Fig. 5845437v1 (P019410_SpeciLodged) - 16 13A. Through manipulation of the user input device 1313 (e.g., the keypad), a user of the print system 300 and the application programs 1333 may manipulate the interface in a functionally adaptable manner to provide controlling commands and/or input to the applications associated with the GUI(s). Other forms of functionally adaptable user 5 interfaces may also be implemented, such as an audio interface utilizing speech prompts output via loudspeakers (not illustrated) and user voice commands input via the microphone (not illustrated). Fig. 13B illustrates in detail the embedded controller 1302 having the processor 301 for executing the APV application program 1333, possibly in conjunction with the 10 processor 1406, and the internal storage 323 and/or memory 1404. The internal storage 323 comprises read only memory (ROM) 1360 and random access memory (RAM) 1370. The processor 301 is able to execute the APV application programs 1333 stored in one or both of the connected memories 1360, 1370 and 1404. When the controller 1302 is initially powered up, a system program resident in the ROM 1360 is executed. The code 15 modules of application program 1333 permanently stored in the ROM 1360 are sometimes referred to as "firmware". Execution of the firmware by the processor 301 may fulfil various functions, including processor management, memory management, device management, storage management and user interface. The processor 301 typically includes a number of functional modules including a 20 control unit (CU) 1351, an arithmetic logic unit (ALU) 1352 and a local or internal memory comprising a set of registers 1354 which typically contain atomic data elements 1356, 1357, along with internal buffer or cache memory 1355. One or more internal buses 1359 interconnect these functional modules. The processor 301 typically also has one or more interfaces 1358 for communicating with external devices via system bus 1381, using 25 a connection 1361. 5845437v1 (P019410_SpeciLodged) -17 The APV application program 1333 includes a sequence of instructions 1362 through 1363 that may include conditional branch and loop instructions. The program 1333 may also include data, which is used in execution of the program 1333. This data may be stored as part of the instruction or in a separate location 1364 within the ROM 5 1360 or RAM 1370. In general, the processor 301 is given a set of instructions, which are executed therein. This set of instructions may be organised into blocks, which perform specific tasks or handle specific events that occur in the print system 300. Typically, the APV application program 1333 waits for events and subsequently executes the block of code 10 associated with that event. Events may be triggered in response to input from a user, via the user input devices 1313 of Fig. 13A, as detected by the processor 301. Events may also be triggered in response to other sensors and interfaces in the print system 300. The execution of a set of the instructions may require numeric variables to be read and modified. Such numeric variables are stored in the RAM 1370. The disclosed 15 methods use input variables 1371 that are stored in known locations 1372, 1373 in the RAM 1370. The input variables 1371 are processed to produce output variables 1377 that are stored in known locations 1378, 1379 in the RAM 1370 and/or the memory 1404. Intermediate variables 1374 may be stored in additional memory locations in locations 1375, 1376 of the RAM 1370. Alternatively, some intermediate variables may only exist 20 in the registers 1354 of the processor 301. The execution of a sequence of instructions is achieved in the processor 301, possibly in conjunction with the processor 1406, by repeated application of a fetch-execute cycle. The control unit 1351 of the processor 301 maintains a register called the program counter, which contains the address in ROM 1360 or RAM 1370 of the next instruction to 25 be executed. At the start of the fetch execute cycle, the contents of the memory address 5845437v1 (P019410_SpeciLodged) - 18 indexed by the program counter is loaded into the control unit 1351. The instruction thus loaded controls the subsequent operation of the processor 301, causing for example, data to be loaded from ROM memory 1360 into processor registers 1354, the contents of a register to be arithmetically combined with the contents of another register, the contents of a 5 register to be written to the location stored in another register and so on. At the end of the fetch execute cycle the program counter is updated to point to the next instruction in the system program code. Depending on the instruction just executed this may involve incrementing the address contained in the program counter or loading the program counter with a new address in order to achieve a branch operation. 10 Each step or sub-process in the processes of the APV methods described below is associated with one or more segments of the application program 1333, and is performed by repeated execution of a fetch-execute cycle in the processor 301 or similar programmatic operation of other independent processor blocks in the print system 300. Components of the print verification unit 330, such as the processor 1406, 15 memory 1404 and print verification I/O unit 1405, are functionally connected via a bus 1412, and are connected to the other processing units of the print system 300 via the data bus 312. The components work in a similar manner to corresponding components 301, 323 and 320 shown in Fig. 3. Accordingly, the components 1406, 1404 and 1405, will not be described in detail below. Furthermore, in one arrangement, the functions performed by 20 the processor 1406, memory 1404 and print verification I/O unit 1405 may instead be performed by the central processing unit 301, memory 323 and I/O unit 320, respectively. However, the processor 1406, memory 1404 and print verification I/O unit 1405 may be separate components within the print verification unit 330 as shown in Fig. 14. Returning to Fig. 1, the top-level flow chart shows method 100 useful for 25 determining if a page contains unexpected differences. In particular, Fig. 1 provides a 5845437v1 (PO19410_SpeciLodged) -19 high-level overview of method 100 of performing colour imaging according to one arrangement executing on the print system 300 including the print verification unit 330 of Fig. 14. Fig. 14 shows the details of the print verification unit 330 of Fig. 3. In one 5 arrangement, the print verification module 330 may form part of the special function module 1310 of Fig. 13A. The unit 330, which performs the noted verification process, comprises an image inspection device in the form of an image capture system 1408. The image capture system 1408 may be used to assess the quality of output prints by detecting unexpected print differences generated by the print engine 329 which performs printing 10 step 130 in the arrangement of Fig. 1. The source input document 166 to the print system 300 is, in the present example, a digital document expressed in the form of a page description language (PDL) script, which describes the appearance of document pages. Document pages typically contain text, graphical elements (line-art, graphs, etc.) and digital images (such as photographs). The source input document 166 may also be referred 15 to as a source image, source image data and so on. In a rendering step 120, the source document 166 is rendered using a software rasteriser module (not shown), under control of the processor 301 executing one or more code modules of the APV software application 1333. The source document 166 is rendered by processing the PDL script, to generate a two-dimensional bitmap image 160 of 20 the source document 166. In another implementation, the rasteriser may be implemented in hardware (e.g., 1402) or a combination of hardware and software. The rasteriser may also generate alignment information (also referred to as "alignment hints"). The alignment information may take the form of a list 162 of regions of the bitmap image 160 with intrinsic alignment structure. The regions will be referred to as "alignment patches" 25 below. Each alignment hint corresponds to an alignment patch (or region), which is 5845437v1 (P019410_SpeciLodged) -20 preferably a square patch, with a patch centre (x,y), a patch strength value indicating the amount of intrinsic alignment structure within the patch (e.g. high frequency components, edges, corners, etc.), and a patch size corresponding to the width and height of the alignment patch. The list 162 of alignment hints is sorted in the y-direction, as this is 5 typically the transport direction 314. The rendered bitmap image 160 and the associated list of alignment hints 162 may be temporarily stored in the printer memory 323. Upon completing processing in the rendering step 120, the rendered bitmap image 160 is sent to colour print step 130. The colour print step 130 uses the print engine 329, and produces output print 163 by forming a visible image on a print medium such as the 10 paper sheet 319 using the print engine 329. The rendered bitmap image 160 in the memory 323 is transferred in synchronism with (a) a sync signal and clock signal (not shown) required for operating the print engine 329, and (b) a transfer request (not shown) of a specific colour component signal or the like, via the bus 312. The rendered bitmap image 160 together with the generated alignment data 162 is also sent (a) to memory 1404 of the 15 print verification unit 330 via the bus 312 and (b) print verification unit I/O unit 1405, as seen in Fig. 14, for use in a subsequent defect detection step 150. The output print 163, which is on the paper sheet 319 in the described example, generated by the colour print step 130 is scanned at an image capturing step 140. The output print 163 may be scanned using, for example, the image capture system 1408. In 20 particular, at step 140, the processor 301 may forward a signal (e.g., via processor 1406) to the image capture system 1408 in order to scan the print 163. The image capture system 1408 may be a colour line scanner for real-time imaging and processing. However, any image capturing device that is capable of digitising and producing high quality digital copy of printouts may be used. Such may include, in some arrangements, an imaging device 25 such as a digital camera. 5845437v1 (P019410_SpeciLodged) - 21 In one arrangement shown in Fig. 14, the image capture system 1408 may be configured to capture an image of the output print 163 from the sheet 319 on a scan-line by scan-line basis, or on a band-by-band basis, where each band comprises a number of scan lines. The captured digital image 164 is sent to print defect detection step 150. A method 5 200 of detecting defects in the output print 163, as executed at step 150, will be described in more detail below with reference to Fig. 2. The method 200 may be implemented by APV Application Specific Integrated Circuit (ASIC) 1407 and/or APV software application program 1403, upon receipt of a signal from processor 1406. The method 200 aligns and compares the bitmap image 160 and the scan image 164 using alignment data 10 162 from rendering step 120 in order to locate and identify print defects in the output print 163. Upon completion, the print defect detection step 150 outputs a defect map 165 indicating defect types and locations of all detected defects within the output print 163. The defect map 165 may be used to make a decision on quality of the output print 163 in a 15 decision step 170. A decision signal 175 is generated by the processor 1406. The decision signal 175 may then be used to trigger an automatic reprint or alert the user to potential issues with the output print 163. In one arrangement, the decision signal 175 is set to "1" (error present) if there are more than ten (10) pixels marked as defective in the defect map 165, or is set to "0" (no error) otherwise. Alternatively, the defective pixel number may be 20 set by the user. Fig. 7 and Fig. 14 show how, in one arrangement, the print step 130, the scanning step 140 and the defect detection step 150 may be arranged in a pipeline. In such an arrangement, a region 1411, such as a band of the rendered bitmap image 160, is printed by the print system engine 329 to form a corresponding region of the output print 163 on the 25 paper sheet 319. When the printed region 1411 of the output print 163 moves to a position 5845437v1 (P019410_SpeciLodged) -22 1412 under the image capture system 1408, the printed region 1411 is scanned according to the scanning step 140 using image capture system 1408 to form part of the scanned image 164. The scan of printed region 1412, as a band of scan-lines, is sent to the print defect detection step 150 for alignment and comparison with corresponding rendered 5 region (band 1411) of the bitmap image 160 that was sent to the print engine 329. Fig. 7 shows a graphical view of how the steps of Fig. 2 (to be described) may be performed in parallel. In particular, Fig. 7 shows that as page 319 moves in feed direction 314, a first region of the rendered bitmap image 160 is printed 710 by the print system engine 329 to form a first region of the output print 163. As a second region is being 10 printed 715 on the printed page 163, the first region is scanned 720 by the scanning step 140 using the scanner 1408 to form a first region of the scanned image 164. As a third region is being printed 745, the first scanned region is sent 730 to the print defect detection step 150 for alignment and comparison with the corresponding first rendered region that was sent to the print system engine 329, and the second printed region is scanned 725. 15 Subsequent regions of the rendered bitmap image 160 may then be processed in the same manner as shown in Fig. 7. Thus, the pipeline arrangement allows all three processing stages (print, scan, detect) to occur concurrently after the first two regions. Returning to Fig. 1, it is advantageous, during rasterisation in the step 120, to perform image analysis on the rendered bitmap image 160 in order to identify the 20 alignment regions for the list 162. The identified alignment regions provide valuable alignment hints to the print defect detection step 150 as shown in Fig. 1. Accurate registration of the bitmap image 160 and the printout scan image 164 enables image quality metric evaluation to be performed on a pixel-to-pixel basis. One of the advantages of such a method is that precise image alignment may be performed without the need to embed 5845437v1 (P019410_SpeciLodged) - 23 special registration marks or patterns explicitly in the source input document 166 and/or the bitmap image 160. The image analysis performed in the rendering step 120 to determine the alignment hints may be based on Harris corners, or other useful image features. Each entry 5 in the list 162 may be described by a data structure comprising four data fields. The four data fields are configured for storing an x-coordinate of a centre of a region (corresponding to the spatial location of a corner), a y-coordinate of the centre of the region, corner strength of the region, and patch size of the region, respectively. In other arrangements, feature points in the bitmap image 160 may be determined 10 using Gradient Structure Tensor or Scale-Invariant Feature Transform (SIFT). Further, alignment hint generation may be performed on a down-sampled image for speed purposes, in which case processing parameters are adjusted accordingly. Fig. 2 is a flow-chart showing a method 200 of detecting defects, as executed at step 150 of Fig. 1, in more detail. The method 200 may be implemented by the APV ASIC 15 1407 and/or one or more code modules 1403 of the software application program 1333 resident in the memory 1404. The method 200 processes bands (or regions) of the bitmap image 160 and the scan image 164. A band of the scan image 164 corresponds to a band 1412 of the page 319 scanned by image capture system 1408. A band of the scan image 164, for example, is a 20 number of consecutive image lines stored in the memory buffer 1404. In one arrangement, a height of each band may be two-hundred and fifty-six (256) scanlines and the width of each band may be the width of the bitmap image 160. For example, for an A4 original document 160 at 300dpi, the width of each band is two-thousand, four hundred and ninety (2490) pixels. Image data within the memory buffer 1404 is updated continuously in a 25 "rolling buffer" arrangement where a fixed number of scanlines are acquired by scanning 5845437v1 (P019410_SpeciLodged) - 24 sensors 1408 in scanning step 140. The fixed number of scanlines may be stored in the memory buffer 1404 by flushing an equal number of scanlines out of the memory buffer 1404 in a first-in-first-out (FIFO) manner. In one arrangement, the number of scanlines acquired at each scanner sampling instance is two-hundred and fifty six (256). In an 5 alternative arrangement, the number of scanlines acquired at each scanner sampling instance is less than the height of the scan band memory buffer, thereby creating an overlapping data content region between any two consecutive scan bands. As seen in Fig. 2, the method 200 begins at a scanline band retrieval step 210 where the memory buffer 1404 is filled by the processor 1406 with a band of image data 10 ("scan band") from the scan image 164. The scan band is fed to the processor 1406 after execution of scanning step 140. In the example shown in Fig. 2, the scan band is down sampled in a down-sampling step 230. For example, the scan band may be down-sampled using a separable Burt-Adelson filter, to reduce the amount of data to be processed, to thereby output a scan band 235 which is a band of the scan image 164. However, step 230 15 is optional and may be excluded. Alternatively, capturing system 1408 with a low resolution image sensor may be used to acquire a low resolution scan band 235. As seen in Fig. 2, in parallel to step 210, a band of the bitmap image 160 at the corresponding resolution and location as the scan band 235, is obtained by the processor 1406 in an original image band and alignment data retrieval step 220. The list 162 of 20 alignment hints generated in rendering step 120 for image alignment is passed to the processor 1406 by the processor 301 for processing at step 220. Once the corresponding original image band 226 has been extracted in step 220, a model of a print and capture process (hereafter referred to as a "print/scan model") is applied by the processor 1406 to original image band 226 at a model application step 225. At step 225, the processor 1406 25 applies a set of transforms to the original image band 226, in accordance with the 5845437v1 (P019410_SpeciLodged) -25 print/scan model, to change the original image band 226. The transforms produce an image band 227 representing an expected output of a print and scan process, referred to as the "expected image" band 227. The print/scan model may include many smaller component models. 5 Returning to Fig. 2, the scan band 235 and the expected image band 227 are then processed by the processor 1406 at a band alignment step 240. In particular, at step 240, the processor 1406 performs image alignment of the scan band 235 and the expected image band 227 using the list of alignment patches (i.e. alignment hints) 162. As step 225 does not change the coordinate system of the original image band 226, spatially aligning the 10 coordinates of the scan band 235 to the expected band 227 is equivalent to aligning the coordinates to the original image band 226. At step 240, the processor 1406 establishes pixel-to-pixel correspondence between the scan band 235 and the expected image band 227 prior to a comparison process in step270. In order to perform real-time print defect detection, a fast and accurate image 15 alignment method is desirable. A block based correlation technique, where correlation is performed for every block in a regular grid, is inefficient. Furthermore, such block based correlation does not take into account whether or not a block contains image structure that is intrinsically alignable. Inclusion of unreliable correlation results can affect overall image alignment accuracy. The described methods overcome the above disadvantages of 20 block based correlation by using a patch-based sparse image alignment technique that accurately estimates a geometrical transformation between images using alignment patches. A method 400 of aligning image bands, as executed at step 240, will be described in greater detail below with reference to Fig. 4. In another arrangement, the method 200 may exclude the original image band step 25 220, so that the whole bitmap 160 is processed at the print/scan application step 225. In 5845437v1 (P019410_SpeciLodged) -26 this instance, the alignment step 240 accepts the whole bitmap 160 with print/scan model applied (the bitmap 160 thus becomes an expected image), instead of the expected image band 227. In such an arrangement, at the alignment step 240, the processor 1406 attempts to align a region within the expected image to the scan band 240 by extracting a number of 5 alignment patches from the expected image and an equal number of corresponding target image patches from the scan image band 235. The extracted alignment patches and target image patches may be used for shift estimation in order to estimate a transformation between the expected image and the scan image 164. In another arrangement, the print/scan model application step 225 may be applied in the method 100 following the 10 rendering step 120. In decision step 250, a test is performed by the processor 1406 as directed by the APV ASIC 1407 and/or the APV code modules 1403 of the software application program 1333 to determine if any geometric errors indicating a misalignment condition (e.g. excessive shift, skew, etc.) were detected in step 240. Step 250 will be described in further 15 detail below with reference to Fig. 4. If the result of the test at step 250 is Yes, then the method 200 moves to a defect map output step 295. Otherwise, the method 200 continues at a band content comparison step 270. As a result of processing in step 240, the scan band 235 and the expected image band 227 are accurately aligned with pixel-to-pixel correspondence. The aligned image 20 bands are further processed at comparing step 270, where the processor 1406 as directed by the APV ASIC 1407 and/or the APV code modules 1403 of the software application program 1333, compares the contents of the scan band 235 and the expected image band 227 to locate and identify print defects. A method 500 of comparing image bands, as executed at step 270, will be described in greater detail with reference to Fig. 5. 5845437v1 (P019410_SpeciLodged) - 27 Following step 270, a check is made by the processor 1406 at a decision step 280 to determine if any print defects were detected in step 270. If the result of step 280 is No, then the method 200 continues at decision step 290. Otherwise processing continues at step 295. 5 At decision step 290, the processor 1406 determines if there are any new scanlines from image capture system 1408 to be processed. If the result of step 290 is Yes, then the method 200 returns to step 210 where the existing band in memory buffer 1404 is rolled (i.e., the top two-hundred and fifty-six (256) scanlines of the band are removed and remaining scanlines in the memory buffer 1404 are moved up by two-hundred and fifty-six 10 (256) lines, with the final two-hundred and fifty-six (256) lines replaced by newly acquired scanlines from step 140. If the result of step 290 is No, then the method 200 continues at step 295, where defect map 165 is updated. Step 295 concludes the method 200, and control returns to step 170 in Fig. 1. Returning to Fig. 1, a decision is made in decision step 170 as to acceptability of 15 output print 163. The defect map 165 may for example be displayed on the video display 1314 for checking by an operator to manually make a decision regarding appropriateness of printed page 163. Sparse image alignment methods, as used in the described methods, use a small number of selected image patches to determine local shift measurements between a pair of 20 input images. In this instance, local shift measurements are assumed to be consistent with a global motion model that maps one input image to the other. By measuring local shifts, image transformation between the bitmap image 160 and the scan image 164 may be modelled. Furthermore, as described above, a band-based arrangement is used to perform alignment step 240. Rather than aligning the whole bitmap image 160 to the whole scan 5845437v1 (P019410_SpeciLodged) -28 image 164, alignment step 240 is performed using image bands extracted from the bitmap image 160 and the scan image 164. Figs. 11A and I1B show the detail of two bands which, according to one example, may be processed in alignment step 240 of Fig. 2. In particular, Fig. 11A shows an 5 example of expected image band 227. Further, Fig. 11B shows an example of scan image band 235. Relative positions of an example alignment patch 1 1 10 in the expected image band 227 and a corresponding image patch 1120 in the scan image band 235 are shown in Figs. 11A and 11B, respectively. In alignment step 240, shift estimation may then be performed, by the processor 1406 as directed by the APV ASIC 2107 and/or the code 10 modules 1403 of software application program 1333, on the two patches 1110 and 1120 to determine a translation that relates the two patches 1110 and 1120. A next pair of patches, referenced as 1130 and 1140 in Figs. 11A and 11B, respectively, may then be selected from expected image band 227 and scan image band 235. The patch 1130 is another alignment patch and the patch 1140 is the corresponding region as determined by the 15 spatial transformation between the two images 160 and 164. Shift estimation may then be repeated between the new pair of patches 1130 and 1140. All the alignment patches 162 within the expected image band 227 may be processed in a similar manner to the patches (e.g., 1110, 1130) of Fig. 11A. Image alignment may be performed between a "reference image" and a "target 20 image" in order to determine a transformation that relates contents within the reference image and the target image geometrically. Any image feature in the reference image may be mapped to a corresponding spatial location with a corresponding image feature in the target image. Similarly, any image feature in the target image may be mapped to a corresponding spatial location with a corresponding image feature in the reference image. 5845437v1 (P019410_SpeciLodged) -29 In the method 100, the bitmap image 160 may be referred to as a reference image. Further, the scan image 164 may be referred to as a target image. Similarly, in the method 200, the expected image band 227 may be referred to as a reference image band. Further, the scan band 235 may be referred to as a target image 5 band. Thus, alignment patches 1110 and 1130 in the reference image band 227 may be referred to as reference patches. Similarly, alignment patches 1120 and 1140 in the target image band 235 may be referred to as target patches. The method 400 of aligning image bands, as executed at step 240, will be described in greater detail below with reference to Fig. 4. The method 400 may be 10 executed by the processor 1406 as directed by the APV ASIC 1407 and/or one or more code modules 1403 of the software application program 1333 resident in the memory 1404. The method 400 operates on two images: the scan image band (or target image band) 235 and the expected image band (or reference image band) 227, and makes use of the alignment hint data 162 derived in the step 120. 15 The method 400 begins at processing step 405, where previously partially extracted patches are processed by the processor 1406. A method 800 of processing target patches across two consecutive image bands, as executed at step 405, will be described in detail below with reference to Fig. 8. Then, from an iterating step 410, the list 162 of alignment hints is iterated by 20 checking each hint progressively to identify suitable alignment hints for further processing. The list 162 may be stored in the memory buffer 1404 by the processor 1406 in conjunction with the processor 301. In a mapping step 417, a current alignment hint 415, which is a reference patch in the reference image band 227, is mapped to the coordinate system of the target (scan) 25 image 164. If the processor 1406 determines in a decision step 420 that the current 5845437v1 (P019410_SpecLodged) - 30 alignment hint 415 when mapped to the target image coordinate system is within the target image band 235, then the method 400 moves to step 440. Otherwise, the method 400 continues to a partial target patch extracting step 432. The current alignment hint (or reference patch) 415 identified in step 410 is 5 described by a data structure which may be configured within the memory buffer 1404. The data structure comprises four data fields for storing: (i) x coordinate of the hint location in a coordinate space of reference image (original image bitmap) 160, (ii) y coordinate of the hint location in the coordinate space of reference image (original 10 image bitmap) 160, (iii) corner (or feature) strength of the reference patch, and (iv) patch size of the reference patch. In extraction step 440, a patch 441 of size S, x S, centred at the current reference patch location 415 is extracted from reference image band (expected image band) 227 and 15 stored within the memory 1404 by the processor 1406. At the same time, a patch 442 of size S, x S, centred at corresponding hint location in the target image band (scan image band) 235 is extracted and stored within the memory 1404 by the processor 1406. The corresponding hint location (i.e., target patch) in the target image band 235 is determined in step 417 using a current estimate transformation derived from a previous 20 alignment operation on a previous document image or band to transform the x and y coordinates of the current reference patch 415 to a corresponding location (x and y coordinates) in the target image band (scan image band) 235. In accordance with the method 400, the target patches that are completely or partially located within the currently processed target band 235 are considered. At step 25 420, if the processor 1406 determines that the considered target patch is partially located 5845437v1 (P019410_SpeciLodged) -31 within the current band, then the method 400 continues to step 432, which extracts part of the target patch that overlaps the current target image band. The partially extracted patch is stored within the memory 1404 for further processing in the step 405 when a next target image band is processed. 5 In shift operation step 445, the processor 1406 performs a shift estimation operation between the reference patch 441 and corresponding target patch 442 in the target image band 235. In one arrangement, "phase only" correlation (hereinafter referred to as "phase correlation") is used to perform the shift estimation operation. Alternatively, other shift estimation methods, such as gradient-based shift estimation, may be used. A method 10 1500 of estimating shift, as executed at step 445, will now be described with reference to Fig. 15. The method 1500 may be executed by the processor 1406, as directed by the APV ASIC 1407 and/or one or more code modules 1403 of the software application program 1333 resident in the memory 1404. 15 The method 1500 begins at step 1510, where the processor 2106 applies a window function, such as a Hanning window, to each of the two patches 441 and 442. The two windowed patches are then padded with zeros to twice the patch width and height. The padded patches are then phase correlated in step 1510. The phase correlation of step 1510 produces a raster array 1530 of real values. 20 In a following peak detection step 1520, the location of a highest peak is determined within the raster array 1530 and stored within the memory 1404, with the location of the highest peak being relative to the centre of the alignment hint patch. A confidence factor for the peak may also be determined. The confidence factor may be defined as height of the detected peak relative to height of a second peak. The output of 5845437v1 (P019410_SpeciLodged) - 32 the shift estimation step 445 is a displacement vector 446 formed of the highest peak location and the confidence factor. After step 445, the method 400 moves to a vector displacement storage step 450, where the location of the peak, the confidence, and the centre of the reference (alignment 5 hint) patch may then be stored in a memory buffer 1404. If it is determined by the processor 1406 in a following decision step 460 that more alignment hints exist, then the method 400 returns to step 410, where a next reference patch (alignment hint) 415 is selected. Otherwise the method 400 continues to a transform determination step 470. 10 In an alternative arrangement, binary correlation may be used in place of phase correlation. The output of the phase correlations performed in accordance with the method 1500 is a set of displacement vectors Dn that represents the transformation that is required to map pixels of the reference image band (expected image band) 227 to the target image 15 band (scan image band) 235. In determining step 470, the processor 1406 determines a transformation from the displacement vectors determined at step 445. The transformation may be stored in memory 1404. In one arrangement, the transformation is an affine transformation with a set of linear transform parameters [(b]1, b 2 , b, b.., Ax,Ay), that relates the displacement 20 vectors in the Cartesian coordinate system in accordance with Equation [I], as follows: h b n [1] where (x., Yn) are alignment patch centres and Ci.Y-) are affine transformed points. In addition, points (xn- 'n) are displaced by the displacement vectors Dn to determine displaced points (X-.n) as follows in Equation [2]: 25 n- n)=(Kn-Yn)+ D [2] 5845437v1 (P019410_SpeciLodged) - 33 A suitable affine transformation may be determined by minimising error between displaced coordinates (-in. ) and affine transformed points (n-i n) by changing affine transform parameters [(b]n,,bj 2 ,b,b 2 2 ,Ax,AY). The error to be minimised is the Euclidean norm measure E as follows in Equation [3]: IN 5 n=1 [3] A minimising solution suitable for minimising the error is shown in Equations [4], below: bln b2

M-

1 n xn by2: \ )[4] 10 with the following relationships of Equations [5]: =(Sxx 5;y S;, =Zn~ ~ Sxy3 Syy 53, = xnyn ynyOn y)n S,. 5 S IXn Dn 1 -SySY + SSy)& - SSxy +Sxs ,S&sxy -S. Syy@ - SS ,5& - SX5 + SSx&S Szy - SxxSY@SySY - SxSyy&SXSY 15 [5] and the relationships of Equation [6], below: 5845437v1 (P019410_SpeciLodged) -34 IM = -SsxSXY + 25 S'YS - 5 Sy. - s s, 5 .y + SS,,syy [6] where the sums are carried out over all displacement vectors with a peak confidence greater than a threshold Pmin. In one arrangement, the threshold Pmin is equal to 2.0. Following step 470, the set of linear transform parameters 5 [(bjn,bu,bj,.bjAxAy) is examined by the processor 1406 in a geometric error detection step 480 to identify geometric errors such as rotation, scaling, shearing and translation. The set of linear transform parameters [(bll, b 1 2 , b 2 1 ,b b 22 ,A, A)when considered without the translation is a 2x2 matrix as in Equation [7], below: = b 1 i b 2 1 ) A = C1 b~l)[7] 10 The matrix of Equation [7] may be decomposed into individual transformations assuming a particular order of transformations as shown in Equation [8] below: A -li b _21 0O 1 0 +cos6 -[sin6 aba 0sy yG' 1 -+ sin6 + cos6 [8 where scaling is defined in accordance with Equation [9] as follows: 15 where 5, and SY specify the scale factor along the x-axis and y-axis, respectively. Shearing is defined in accordance with Equation [10]: (1 0) (1 hN) (hY 1)or\0 1 ) [10] where Ax, and hy specify the shear factor along the x-axis and y-axis, respectively. Rotation is defined in accordance with Equation [11], as follows: (+cos6 -sin6\ 20 (+sin6 +cos9) [11] where 6 specifies the angle of rotation. 5845437v1 (P019410_SpeciLodged) -35 The parameters sx, Sr, hy, and 9 may be determined from the above matrix coefficients by the following Equations [12], [13], [14] and [15]: s = b2+b [12] det(A) S, = S, [13] h = b-jb 12 + b-lb 22 5 det(A) [14] tan

=

bu, [15] In one arrangement, the allowable horizontal or vertical displacement magnitude Amxis four (4) pixels for images at 300dpi, and scale factor range (sm.s) is (0.98, 1.02). In such an arrangement, allowable shear factor magnitude hma is 0.01, and 10 allowable angle of rotation 0mx. is 0.1 degrees. However, suitable alternative parameters, allowing for greater translation or rotation, may also be used. If the derived transformation obtained in step 470 satisfies the above affine transformation criteria, then the scan band 235 is deemed to be free of geometric errors in a following decision step 490, and the method 400 continues at an expected image to scan 15 space mapping step 492. Otherwise the method 400 concludes. In step 492, the set of registration parameters is used to map the expected image band 227 to the scan image space. In particular, RGB value at coordinate (xs.y 3 ) in transformed image band is the same as RGB value at coordinate (Xr. Yr) in the expected image band 227, where coordinate (x,.,Yr) is determined by an inverse of the linear 20 transformation represented by the registration parameters in Equation [16], as follows: x,. = b 1 b 2 [(+b 2 -b 2 1 g x,-Ax 'rbb - b 2 b \.-bu 2 +bu y - [16] 5845437v1 (P019410_SpeciLodged) - 36 For coordinates (Xr- Yr) that do not correspond to integer pixel positions, an interpolation scheme (e.g., bi-linear interpolation) may be used to calculate the RGB value (x, y) from neighbouring values. Following step 492, the method 400 concludes. In an alternative arrangement, in step 492 the set of registration parameters may 5 be used to map the scan image band 235 to the original image coordinate space. As a result of such mapping in the step 492, the reference image band (expected image band) 227 and the target image band (scan image band) 235 are aligned. The method 500 of comparing image bands, as executed at step 270, will be described in greater detail with reference to Fig. 5. The method 500 may be executed by 10 the processor 1406, as directed by the APV ASIC 1407 and/or one or more code modules 1403 of the software application program 1333 resident in the memory 1404. In accordance with the method 500, the processor 2106 operates on two image bands, being the target image band (scan image band) 235, and aligned reference image band (expected image band) 502. 15 Processing of the method 500 operates in a tile raster order, in which tiles are made available for processing from top-to-bottom and left-to-right in the band, one at a time. As seen in Fig. 5, the method 500 begins at step 510, where a Q x Q pixel tile is selected by the processor 1406 from each of the two bands 502, 235, with the tiles having corresponding positions in the respective bands 502, 235. The two tiles, namely an aligned 20 expected image tile 514 and a scan tile 516, may be stored within the memory buffer 1404 and then processed in following step 520. In one arrangement, Q is equal thirty-two (32) pixels. At comparison step 520, the processor 1406, as directed by the APV ASIC 1407 and/or the code modules 1403 of the software application 1333 performs a comparison 25 between the tiles 514 and 516 to identify print defects.. A method 600 of comparing tiles 5845437v1 (P019410_SpeciLodged) - 37 of the image bands 502 and 235, as executed at step 520, will be described in greater detail with reference to Fig. 6. The method 600 may be executed by the processor 1406, as directed by the APV ASIC 1407 and/or one or more code modules 1403 of the software application program 5 1333 resident in the memory 1404. The method 600 begins at selection step 610, where a scan pixel to be checked is selected by the processor 1406 from the scan tile 516. In a next determining step 620, the minimum difference in a neighbourhood of the scan pixel selected at step 610 is determined by the processor 1406. Any suitable distance metric, D, may be used to determine the difference at step 620. 10 In one arrangement, the distance metric, D, may be a colour difference metric, DRGB(p,q). The colour difference metric, DRGB(p,q), may be used to determine colour difference for a single pixel. The colour difference metric used may be a Euclidean distance in RGB space, which for two pixels p and q is expressed in Equation [17], as follows: 15 DRGB p,9 - P ~(p, -[1]q,Y +(p -q) where p, Pg IPb are red, green, and blue components of pixel p, and likewise components q,, q,,q for pixel q. In an alternative arrangement, the distance metric, D, used at step 620 is a Delta E distance metric, D,(p, q), which may be determined in accordance with Equation [18], as 20 follows: DA(p,q)= (pL* - qL)2 +(pa* qa) +(pb. qb) [18] The Delta E distance, DAE(p,q), is defined using the L*a*b* colour space, which has a known conversion from the sRGB colour space. In one arrangement, the RGB values provided by image capture devices, such as the scanner 1408, may be sRGB values. 5845437v1 (P019410_SpeciLodged) - 38 The minimum distance between a scan pixel p, at location xy and nearby pixels in aligned expected image pe, may be determined using the chosen distance metric D according to Equation [9], as follows: Dmi.(P,[x,yD= mi (D(p,[x,y],p,[x',y'D) [19] x'=x-KB..x+Kg y'=y-K 8 ..y+KH 5 where KB is approximately half the neighbourhood size. In one arrangement, KB, may be selected as one (1) pixel, giving a 3x3 neighbourhood. In a next update step 630, a tile defect map, configured within memory buffer 1404, is updated by the processor 1406 at location xy of the map based on the value of Dmin. A pixel is determined to be defective if the Dmin value of the pixel is greater than a 10 certain threshold, Ddret. In one arrangement where the Delta E distance metric, DAF, is used to determine Dmin, the threshold Ddefect is set as ten (10). In next decision step 640, if there are any more pixels left to process in the scan tile, the method 600 returns to the step 610. Otherwise, if no pixels are left to process, the method 600 is completed and control returns to a step 530 in the method 500 of Fig. 5. 15 Following execution of step 520, the tile-based defect map created in step 630 is stored in the defect map 165 in band defect map updating step 530. In a following decision step 540, if the processor 1406 determines that print defects existed when updating the band defect map in step 530, then the method 500 concludes as no further processing is necessary. Otherwise, the method 500 proceeds to step 550. 20 In one arrangement, the defect map 165 is a 2-dimensional map, which allows a user to determine where defects occurred in the output print 163. At a next decision step 550, if the processor 1406 determines that there are remaining tiles to be processed, then the method 500 continues at step 510 where a next set of tiles are selected. Otherwise, the method 500 concludes. 5845437v1 (P019410_SpeciLodged) - 39 Returning to the alignment method 400 of Fig. 4, in order to determine the alignment between the reference image 160 and the target image 164, the contents of target patches extracted at step 440 and of corresponding reference patches need to overlap. The accuracy of determining the alignment between the reference image 160 and the target 5 image 164 is related to the amount of overlap between the target patches and corresponding reference patches, with accuracy increasing as the overlap increases. The APV ASIC 1407 also requires an estimation of the transform between the reference and the target coordinate systems in order to determine appropriate target patch locations. By processing the target image 164 band by band, the transform determined for a 10 band becomes a best estimated transform of a following band. The locations of the target patches in the processed band may be determined using the best estimated transform as determined at the time that the processed band is processed. As shown in Fig. 9, the location of each target patch (e.g., 930, 940) in a target image 900 may be determined by applying a best estimated transform to a corresponding 15 reference patch location at the time that the target patch is extracted (e.g., as at step 440). In the example of Fig. 9, some of the target patches (e.g., 930) are entirely located within a band. For example, the target patch 930 is entirely located in the band 910. Some other target patches are spread across two consecutive bands. For example, as seen in Fig. 9, the target patch 940 is spread across the bands 910 and 920. An upper part 950 of the target 20 patch 940 is partially extracted at step 432 when the band 910 is processed, and a lower part 960 of the same target patch 940 is extracted at step 830 when the band 920 is processed. Both parts 950 and 960 of the target patch 940 need to be extracted and combined together in order to form the entire target patch 940 before the target patch 940 may be processed. As a consequence, the target patch 940 may be used in the alignment 5845437v1 (P01941OSpeciLodged) - 40 method 400 only when processing the band 920. However, an initial location of the target patch 940 was determined when the band 910 started being processed. Target patches that are spread across consecutive bands, such as the target patch 940, may result in a sub-optimal image alignment. For example, as illustrated in Figs. 10A 5 and 10B, when band 1080 of target image 1050 is processed, the APV ASIC 1407 determines that a reference patch, P(),at location 1020 in reference image 1010 has a corresponding target patch, P(0, at location 1060. The location 1060 of the target patch P() is determined by applying, for example, the best estimated transform B known at the beginning of the processing of the band 1080 to location 1020 of the reference patch Pr(0. 10 Having determined the location 1060 of the target patch P(0, the processor 1406 determines that the target patch P() is spread across the bands 1080 and 1090. The upper part 1062 of the target patch is located in the band 1080, while the lower part 1064 of the target patch is located in the following band 1090. Since the target patch 1060 is not entirely located within the band 1080, the target patch 1060 cannot be 15 used to determine the alignment of the band 1080. The alignment of the band 1080 is determined using other target patches, such as target patch 1055, which is entirely located within the band 1080. In accordance with the example of Figs. 10A and 10B, the alignment of the band 1080 produces a new transform T mapping locations in the reference image 1010 to the target image 1050. The transform T is expected to be a refinement of the best 20 estimated transform B known at the beginning of the processing of the band 1080. The newly determined transform T may be applied to location 1020 of the reference patch Pr() in order to determine a better location 1070 of a corresponding target patch P(O. The application of the transform T also increases the overlapping contents of the reference patch Pr(Land corresponding target patch P(z) for the alignment of the band 25 1090. 5845437v1 (P019410_SpeciLodged) -41 However, in a pipeline processing arrangement, for improved processing speed and efficiency as shown in Fig. 7, the memory buffer 1404 for storing current scan band data is updated continuously. In accordance with the example of Figs.10A and 10B, the band 1080 is overwritten by the band 1090 after the band 1080 is processed. Thus, refined 5 target patch location 1070 for the reference patch r(i' is no longer available, and the target patchP(0) remains at the same location 1060. In order to perform image alignment, for example, of the reference 1010 and the target image 1050, lower part 1064 of the target patch may be extracted according to step 830 of the method 800 described below to form a complete extracted target patch P4( at location 1060. 10 As the improved transform T is available at the beginning of processing at the band 1090, the transform T may be used to acquire a patch pair with greater correspondence. In accordance with the example of Figs. 1OA and 10B, the method 800 retains the target patch Pr(Z) at location 1060 and determines a refined corresponding reference patch (i) to a new location 1030 in the reference image 1010. The reference 15 patch Pr() is determined by applying the inverse of the transform T to the location 1060 of the target patch P(0. Refining the reference patch P(O from location 1020 to location 1030 improves correspondence between the reference patch Pr() and the target patch P(0), thereby improving shift estimation accuracy and stability. However, there is a limit to how much the refinement is allowed to move the reference patch Pr(O) from original location 20 1020 to a refined location 1030, because the original location 1020 for reference patch &() was selected based on the amount of image alignment structure. Moving the reference patch P( too far away from the location 1020 may reduce the effectiveness the reference patch 4(. In one arrangement, the refined location 1030 of the reference patch P,4O) may not be more than half the patch size away from original location 1020. In another 25 arrangement, both the target patch P() and corresponding reference patch Pr() are 5845437v1 (P019410_SpeciLodged) - 42 repositioned within certain constraints in order to increase overlapping contents between the target patch Pt(0 and reference patch P() while retaining suitable contents for shift estimation step 445 of the method 400. The band-based alignment step 240 of the method 200 takes special care of the 5 target patches, such as target patch Pt() at location 1060 that are spread across two consecutive bands (e.g., bands 1080 and 1090). The upper part of such patches (e.g., 1062) is extracted from a band (e.g., 1080) and is then combined with the lower part of the same patches (e.g., 1064) when a following band (e.g., 1090) is processed. The method 800 of processing target patches across two consecutive image bands, 10 as executed at step 405, will now be described with reference to Fig. 8. The method 400 may be executed by the processor 1406 as directed by the APV ASIC 1407 and/or one or more code modules 1403of the software application program 1333 resident in the memory 1404. The method 800 will be described by way of example with reference to the reference image 1010 and the target image 1050 of Figs. IA and 10B, respectively. Prior 15 to execution of the method 800 the processor 1406 may be used for accessing a set of reference patches (e.g., 1020, 1035) for the reference image 1010, the reference patches being selected based on content of the reference image 1010. As described below, the method 800 may be used for refining the locations of the reference patches based on a corresponding set of target patches (e.g., 1060) for the target image 1050 and a best 20 estimated transform. In accordance with the method 800, between the processing of two consecutive bands (e.g. bands 1080 and 1090), the APV ASIC 1407 stores within the memory 1404 a list of partially extracted patches 810 which were extracted at step 432 when a previous target image band was processed. The APV ASIC 1407 also stores and a best estimated 25 transform 820. When processing the first band 1080 of the target image 1050, the list 810 5845437v1 (P019410_Speci_Lodged) - 43 of partially extracted patches is empty and the best estimated transform 820 is set to an initial value that may be a default value based on physical characteristics of the print system 300. The best estimated transform for relating the reference image and the target image is determined during execution of the method 400 based on a shift between at least 5 one target patch located in a first band (or region) of the target image and the corresponding reference patch. Then at decision step 880, the processor 1406 determines if there is any partially extracted patch in the list 810 which has not been processed yet. If there is no partially extracted patch in the list 810 to process, then the method 800 proceeds to step 410 in 10 Fig. 4. Otherwise, if there is a partially extracted patch in the list 810 to process, then the method 800 proceeds to step 825. At accessing step 825, the processor 1406 accesses a previously partially extracted target patch (e.g. the upper part 1062 of target patch 1060) from the memory 1404. Then at extraction step 830, the processor 1406 extracts lower part (e.g., 1064) of the previously 15 partially extracted target patch at location 1062. The previously extracted upper part (e.g., 1062) and the newly extracted lower part 1064 are combined together by the processor 1406 to form the complete image of the target patch P(i) at location 1060. In order to increase the overlap between the contents of the target patch and corresponding reference patch, at determining step 840, the processor 1406is used for repositioning the location of 20 the corresponding reference patch Pr (i at location 1020 within the reference image 1010 to a new refined location 1030 by applying the inverse of the best estimated transform known at the current time to the target patch location 1060. As seen in Fig. 10B, the patch at target patch location 1060 is located partially within the first band (or region) 1080 and partially within the second band (or region) 1090 of the target image 1050. 5845437v1 (P019410_SpeciLodged) -44 In one arrangement, the best estimated transform is expressed as an affine transform characterised by parameters [(b 1 , b 1 zb 1 , b 2

;

2 ,Ax,Ay) of the affine transform. In accordance with the example of Figs. 10A and 10B, the inverse affine transform is applied to the location(xs YS) of the target patch 1060 at step 840 in order to determine a 5 better location(xyr. Yr) of corresponding reference patch 1030, in accordance with Equation [20]: X rb22+bi X - Ax) v,)"~ bb, -bub?\.-bn +b1) y,-AY) [20] Then at shift estimation step 850, the processor 1406 performs a shift estimation between the extracted target patch at location 1060 and newly repositioned reference patch 10 at location 1030, in accordance with the method 1500 as described above with reference to Fig. 15. At storing step 860, the processor 1406 stores a resulting displacement vector for further processing at step 470. Then the method 800 returns to step 880, where the processor 1406 determines if there is any other partially extracted patch to be processed. 15 Otherwise the method 800 concludes. Processing returns to step 410. After determining step 470, the processor 1406 assigns the transform T to the best estimated transform B of the next band 1098 to be processed. In this instance, the list of partially extracted patches determined at step 432, stored within the memory 1404 becomes the list 810 of previously partially extracted patches. 20 In one arrangement, both a target patch (e.g., target patch 1280) of a target image 1250 (see Fig. 12A), which is spread across two consecutive bands 1260 and 1270, and a corresponding reference patch (e.g., reference patch 1220) may be repositioned within certain constraints. In such an arrangement, the reference patch 1220 and the target patch 1280 are both repositioned in order to increase the overlapping contents of the target patch 25 1280 and the reference patch 1220 while retaining suitable contents for the shift estimation 5845437v1 (P019410_SpeciLodged) - 45 step 445. In such an arrangement, while processing the band 1260, the APV ASIC 1407 may be configured to determine, based on the best estimated transform known at the current time, that the reference patch 1220 in the reference image 1210 has a corresponding target patch 1280 in the target image 1250. Since the target patch 1280 is spread across the 5 two consecutive bands 1260 and 1270, only the upper part 1281 of the target patch 1280 is extracted while processing the band 1260. In this instance, the upper part 1281 is stored within the memory 1404 for further processing with next band 1270. The alignment of the band 1260 produces a new transform from the reference to the target coordinate systems, and based on the new transform, the APV ASIC 1407 may 10 determine a better location 1290 of the reference patch 1220. At the time that the better location 1290 is determined, the band 1260 that has just been processed is not available within the memory 1404 anymore. In this instance, it is only possible to extract target patches in an area made of the union of currently processed band 1270 and previously partially extracted patch 1281. The APV ASIC 1407 may be configured for repositioning 15 the target patch corresponding to the reference patch 1220 from a latest best position 1290 to a nearest position 1291, as seen in Fig. 12B. Repositioning the target patch corresponding to the reference patch 1220 in such a manner allows the target patch to be completely located within the boundaries made of the union (or overlap area) of the currently processed band (or first region) 1270 and 20 previously partially extracted patch 1281. Depending on relative positions of the previously partially extracted patch 1281, the new best target patch location 1290 and currently processed band 1270, the nearest position 1291 may be entirely in the currently processed band 1270 as illustrated in Fig. 12B. In this instance, the previously partially extracted patch 1281 may be discarded. Alternatively, the nearest position 1291 may be 25 overlapping the previously partially extracted patch 1281 as shown in Fig. 12C. In all 5845437v1 (P019410_SpeciLodged) -46 cases, the corresponding reference patch 1220 is repositioned to a better position 1230 by applying the inverse of the best estimated transform known at that time to the location of the newly determined position 1291 of the target patch. Consecutive bands in the target image 1250 may contain a partially overlapping 5 region. Referring to Fig. 12A, if consecutive bands 1260 and 1270 overlap, the target patch 1290 may overlap within the area made of the union of the currently processed band 1270 and the previously partially extracted patch 1281. Such an overlap enables the target patch 1290 to be repositioned to a closer and more convenient location 1291 in order to minimise the repositioning of the corresponding reference patch 1220 to the position 1230. 10 INDUSTRIAL APPLICABILITY The arrangements described are applicable to the computer and data processing industries and particularly industries in which printing is an important element. The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and 15 spirit of the invention, the embodiments being illustrative and not restrictive. In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including", and not "consisting only of'. Variations of the word "comprising", such as "comprise" and "comprises" have correspondingly varied meanings. 20 5845437v1 (P019410_SpeciLodged)

Claims (7)

1. A method of processing patches of a reference image and a target image, said method comprising: 5 accessing a set of reference patches for the reference image, said reference patches being selected based on content of the reference image; identifying a corresponding set of target patches for the target image based on respective locations of the corresponding reference patches in the reference image; determining a transform for relating the reference image and the target image 10 based on a shift between at least one target patch located in a first region of the target image and the corresponding reference patch; and repositioning the corresponding reference patch within the reference image based on an inverse of the determined transform and the location of a further target patch located partially within the first region of the target image and partially within a second region of 15 the target image.

2. The method according to claim 1, further comprising determining a further transform relating the reference image and target image based on a shift between the further target patch and the corresponding repositioned reference patch. 20

3. The method according to claim 1, further comprising repositioning a further target patch within the boundaries of an overlap area between the first region and the second region. 5845437v1 (P019410_SpeciLodged) - 48

4. An apparatus for processing patches of a reference image and a target image, said apparatus comprising: means for accessing a set of reference patches for the reference image, said reference patches being selected based on content of the reference image; 5 means for identifying a corresponding set of target patches for the target image based on respective locations of the corresponding reference patches in the reference image; means for determining a transform for relating the reference image and the target image based on a shift between at least one target patch located in a first region of the 10 target image and the corresponding reference patch; and means for repositioning the corresponding reference patch within the reference image based on an inverse of the determined transform and the location of a further target patch located partially within the first region of the target image and partially within a second region of the target image. 15

5. A system for processing patches of a reference image and a target image, said system comprising: a memory for storing data and a computer program; a processor coupled to said memory for executing said computer program, said 20 program comprising instructions for: accessing a set of reference patches for the reference image, said reference patches being selected based on content of the reference image; identifying a corresponding set of target patches for the target image based on respective locations of the corresponding reference patches in the 25 reference image; 5845437v1 (P019410_SpeciLodged) - 49 determining a transform for relating the reference image and the target image based on a shift between at least one target patch located in a first region of the target image and the corresponding reference patch; and repositioning the corresponding reference patch within the reference 5 image based on an inverse of the determined transform and the location of a further target patch located partially within the first region of the target image and partially within a second region of the target image.

6. A computer readable medium having software stored thereon for processing 10 patches of a reference image and a target image, said program comprising: code for accessing a set of reference patches for the reference image, said reference patches being selected based on content of the reference image; code for identifying a corresponding set of target patches for the target image based on respective locations of the corresponding reference patches in the reference 15 image; code for determining a transform for relating the reference image and the target image based on a shift between at least one target patch located in a first region of the target image and the corresponding reference patch; and code for repositioning the corresponding reference patch within the reference 20 image based on an inverse of the determined transform and the location of a further target patch located partially within the first region of the target image and partially within a second region of the target image. 5845437v1 (P019410_SpeciLodged) -50

7. A method of processing patches of a reference image and a target image, said method being substantially as herein before described with reference to any one of the embodiments as that embodiment is shown in the accompanying drawings. 5 DATED this 19th Day of December 2011 CANON KABUSHIKI KAISHA Patent Attorneys for the Applicant SPRUSON&FERGUSON 5845437v1 (P019410_Speci_Lodged)