AU2005202715B2 - A Method for Detecting and Selecting Good Quality Image Frames from Video - Google Patents

Description

S&FRef: 723504

AUSTRALIA

PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT Name and Address of Applicant Actual Inventor(s): Address for Service: Invention Title: Canon Kabushiki Kaisha, of 30-2, Shimomaruko 3-chome, Ohta-ku, Tokyo, 146, Japan James David Clark, George Politis, Peter William Mitchell Ilbery Spruson Ferguson St Martins Tower Level 31 Market Street 'Sydney NSW 2000 (CCN 3710000177) A Method for Detecting and Selecting Good Quality Image Frames from Video Associated Provisional Application Details: [33] Country:

AU

[31] Appl'n No(s): 2004903394 [32] Application Date: 22 Jun 2004 The following statement is a full description of this invention, including the best method of performing it known to me/us:- 5845c A METHOD FOR DETECTING AND SELECTING GOOD QUALITY IMAGE FRAMES FROM VIDEO Field of the Invention The present invention relates to image analysis and applications based upon this analysis, particularly based upon the detection of motion, image blur arising from imperfect focus, and image exposure.

Background Amateur "home" video footage often contains scenes of poor quality. Thus, for example, a significant amount of home video footage contains out-of-focus subject matter, over-exposed and under-exposed scenes, or excess camera motion. This is often the case even when the capture device is equipped with modem day intelligent camera features that simplify and enhance the filming process. Poor quality scenes not only degrade the overall integrity of the video subject matter, but can exacerbate the difficulty of searching for good quality images or frames from within the video footage. Extraction of good quality frames from a long sequence of video footage can be performed manually. However since current storage media can archive 3 or more hours of video footage, this is laborious and time consuming.

Summary of the Invention According to a first aspect of the invention, there is provided a method of determining a quality value of a frame, the method comprising the steps of: dividing the frame into a plurality of tiles: determining attributes of each said tile based upon pixel values of the tile and (ii) pixel values of a corresponding tile of a preceding frame; and 200605 723504 12-MAR-2008 17:59 12. AR. 008 7:59SPRUSON FERGUSON 92615486 NO. 8475 P. 00 -2o establishing the quality value of the frame bytesting the tile attributes of the frame against pre-determined criteria; and defining the quality value of the frame depending upon results of said testing; wherein: the attributes comprise a motion definition state and a motion magnitude, these being determined by: o classifying the tile as having a motion defined state if one of the range of luminance values of pixels in the tile or in the corresponding tile of the preceding ci frame exceeds a pre-deterndned luminance threshold; and (ii) the difference between a midrange pixel luminance value for the tile and the mid-range pixel luminance value for the corresponding tile of the preceding frame exceeds a pre-determined mid-range difference threshold; and determining, if the tile is classified as having a motion defined state by the step said motion magnitude dependent upon a difference in pixel value distributions between a pixel value histogram of the die and a pixel value histogram of the corresponding tile in the previous frame; and wherein: the attributes further comprise a focus definition state and a focus magnitude, these being determined by classifying the tile as having a focus undefined state if the intensity range of pixels in the tile is too small, or there is too much motion in the tile or (ii) the number of defined difference quotients in the tile is less than a threshold; and determining, if the tile is not classified as having a Locus unadefined state by the step the focus magnitude depending upon the largest difference quotient values of pixel lumninance of the tile- 1155043_1/ 723504 050308 COMS ID No: ARCS-18271 1 Received by IP Australia: Time 18:14 Date 2008-03-12 12 MAR, 2008 18:00 SPRUSON FERGUSON 92615486 NO. 8475 P. 6 00 -3- O According to another aspect of the invention, there is provided an apparatus for determining a quality value of a frame, the apparatus comprising: means for dividing the frame into a plurality of tiles: means for determining attributes of each said tile based upon pixel values of the V tile and (ii) pixel values of a corresponding tile of a preceding frame; and means for establishing the quality value of the frame comprising O means for testing the tile attributes of the frame against pre-determined Ocriteria; and ci means for defining the quality value of the frame depending upon results of said testing; wherein: the attributes comprise a motion definition state and a motion magnitude, these being determined by: means for classifying the tile as having a motion defined state if one of the range of luminance values of pixels in the tile or in the corresponding tile of the preceding frame exceeds a pre-determined luminance threshold; and (ii) the difference between a mid-range pixel luminance value for the tile and the mid-range pixel luminance value for the corresponding tile of the preceding frame exceeds a pre-determined mid-range difference threshold; and means for determining, if the tile is classified as having a motion defined state by the step said motion magnitude dependent upon a difference in pixel value distributions between a pixel value histogram of the tile and a pixel value histogram of the corresponding tile in the previous frame; and wherein: the attributes further comprise a focus definition state and a focus magnitude, these being determined by: 1155043_1 723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 12-MWAR2008 18:00 SPRUSON FERGUSON 92615486 NO. 8475 P. 7 00 -4- 0 0 means for classifying the tile as having a focus undefined state if the intensity range of pixels in the tile is too small, or there is too much motion in the tile or (ii) the number of defined difference quotients in the tile is less than a threshold; and means for determining, if the tile is not classified as having a focus undefined state by the step the focus magnitude depending upon the largest difference R quotient values of pixel luminance of the tile.

SAccording to another aspect of the invention, there is provided a computer program o product having a computer readable medium having a computer program recorded therein for CN directing a processor to execute a method of determining a quality value of a frame, said computer program product comprising: code for dividing the frame into a plurality of tiles: code for determining attributes of each said tile based upon pixel values of the tile and (ii) pixel values of a corresponding tile of a preceding frame; and code for establishing the quality value of the frame comprising: code for testing the tile attributes of the frame against pre-determined criteria; and code for defining the quality value of the frame depending upon results of said testing; wherein: the attributes comprise a motion definition state and a motion magnitude, these being determined by: code for classifying the tile as having a motion defined state if one of(i) the range of luminance values of pixels in the tile or in the corresponding tile of the preceding frame exceeds a pre-determined luminance threshold; and (ii) the difference between a mid-range pixel luminance value for the tile and the mid-range pixel luminance 1155043_1/ 723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 12-MAR-2008 18:00 12. AR. %8 1:UUSPRUGON FERGUSON 92615486 NO. 8475 P. 8 00 4a o value for the corresponding file of the preceding frame exceeds a pre-determined rnic-range difference threshold; and code for determining, if the tile is classified as having a motion defined state by the step said motion magnitude dependent upon a difference in pixel value distributions between a pixel value histogram of the tile and a pixel value histogram of the R corresponding tile in the previous frame; and wherein: o the attributes fuirther comprise a focus definition state and a focus magnitude, these o being determined by: ci code for classifying the tile as having a focus undefined state if the intensity range of pixels in the tile is too small, or there is too much motion in the tile or (hi) the number of defined difference quotients in the tile is less than a threshold; and code for determining, if the tile is not classified as having a focus undefined state by the step the focus magnitude depending upon the largest difference quotient values of pixel luminance of the tile, According to another aspect of the invention, there is provided a video fr-ame selected from a sequence of video frames dependent upon a quality value of the frame, the frame being selected by a method of determining a quality value of a video frame, said method being applied to each frame in the sequence, said method comprising, in regard to a particular frame in the sequence, the steps of;.

dividing the frame into a plurality of tiles: determining attributes of each said tile based upon pixel values of the tile and (ii) pixel values of a corresponding tile of a preceding frame; and establishing the quality value of the frame by: testing the tile attributes of the frame against pro-determined criteria; and 11550431/ 723504 050308 COMS ID No: ARCS-18271 1 Received by IP Australia: Time (I-Pm) 18:14 Date 2008-03-12 12, MAR, 2008 18:00 SPRUSON FERGUSON 92615486 NO. 8475 P. 9 00 -4b ti w e defining the quality value of the frame depending upon results of said testing; wheein: the attributes comprise a motion definition state and a motion magnitude, these being determined by: classifying the tile as having a motion defined state if one of the 0 range of luminance values of pixels in the tile or in the corresponding tile of the preceding frame exceeds a pre-detennined luminance threshold; and (ii) the difference between a midcirange pixel luminance value for the tile and the mid-range pixel luminance value for the corresponding tile of the preceding frame exceeds a pre-determined mid-range difference threshold; and detemining if the tile is classified as having a motion defined state by the step said motion magnitude dependent upon a difference in pixel value distributions between a pixel value histogram of the tile and a pixel value histogram of the corresponding tile in the previous frame; and wherein: the attributes further comprise a fbcus definition state and a focus magnitude, these being determined by: classifying the tile as having a focus undefined state if the intensity range of pixels in the tile is too small, or there is too much motion in the tile or (ii) the number of defined difference quotients in the tile is less than a threshold; and determining, if the tile is not classified as having a focus undefined state by the step the focus magnitude depending upon the largest difference quotient values of pixel luminance of the tile.

11550431 723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 12 MAR 2008 18:01 SPRUSON FERGUSON 92615486 NO. 8475 P. 00 -4c 0 oAccording to another aspect of the invention, there is provided a method of estimating cxl quality of an image in a video, the video comprising a plurality of images, each image comprising a plurality of pixels, said method comprising the steps of: cxl arranging, for a currnt image of said video, pixels of the current image into a plurality of tiles, each said tile containing a plurality of pixels; determining the number of the tiles of the current image having a defined attribute; cxl oand oestimating quality of the current image based on the number of said tiles C'l having the attribute as a proportion of the number of tiles in the current image; wherein: said attribute is one of a motion attribute, a focus attribute, and an exposure attribute; said motion attribute classifies a tile as either motion-defined or motion-undefined; and wherein: determining the number of tiles of the current image having said motion attribute comprises the steps of: obtaining, for each colour channel of tile pixels, a respective range of sample values, and; setting a motion attribute of said tile of said current image to indicate that the tile is motion-undefined itsaid colour channel range of said tile of said current image does not exceed a colour channel range threshold value, and; said colour channel range of a corresponding tile of a previous image does not exceed said colour channel range threshold value, and; the absolute difference between a colour channel mid-range value of said tile of said current image and a corresponding colour channel mid-range value of said 1155043_1/723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 12 MAR 2008 18:01 SPRUSON FERGUSON 92615486 NO. 8475 P. 11 00 -4d 0 o0 corresponding tile of said previous image does not exceed a colour channel mid-range difference threshold value; otherwise setting said motion attribute of said tile of said current image to indicate that said tile is motion-defined, According to another aspect of the invention, there is provided a method of selecting an image from a video comprising a plurality of images, each image comprising a plurality of o pixels, and said method comprising the steps of: performing, for each image of said plurality of images, the steps of: ci arranging pixels of the image into a plurality of tiles, each said tile containing a plurality ofpixels; determining the number of the tiles of the image having a defined attribute; and estimating quality of the image based on the number of said tiles having the attribute as a proportion of the number of tiles in the image; and selecting an image from said video in accordance with the estimated quality wherein: said attribute is one of a motion attribute, a focus attribute, and an exposure attribute; said motion attzibute classifies a tile as either motion-defined or motion-undefined; and wherein: determining the number of tiles of the current image having said motion attribute comprises the steps of: obtaining, for each colour channel of tile pixels, a respective range of sample values, and; 11550431 723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 MAR. 2008 18:01 SPRUSON FERGUSON 92615486 NO. 8475 P. 12 0 4e setting a motion attribute Of said tile of said current image to indicate that the 0 tile is motion-undefined if: said colour channel range of said tWe of said current image does not c'i exceed a colour channel range threshold value, and; said colour channel range of a correspondiug tile of a previous image does not exceed said colour channel range threshold value, and; the absolute difference between a colour channel mid-range value of V) said tile of said current image and a corresponding colour channel mid-range value of said corresponding tile of said previous image does not exceed a colour channel mid-range difference threshold value; otherwise setting said motion attribute of said tile of said current image to indicate that said tile is motion-defined.

Other aspects of the invention are disclosed.

11550431 /723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 Brief Description of the Drawings One or more embodiments of the present invention will now be described with reference to the drawings, in which: Fig. 1 is functional block diagram of a system for selecting good quality frames from a video data stream; Fig. 2 is a schematic block diagram of a general purpose computer upon which the disclosed method can be practiced; Fig. 3 shows the correspondence between sub-processes of the disclosed method and the associated drawings in this description; Fig. 4 shows a schematic block diagram of a system for detecting motion, focus and quality in video; Fig. 5 shows a flow diagram of a method of calculating statistical data, motion data and focus data for each tile of tiled video frame image data in the motion and focus analysis unit 122 in Fig. 4; Fig. 6 shows a schematic diagram of a tiled video frame image comprising sample data from the luminance channel Y; Fig. 7 shows a flow diagram of a method of calculating the statistical data of a tile at step 206 in Fig. Fig. 8 shows a flow diagram of a method of calculating the motion data of a tile at step 210 in Fig. Fig. 9 shows a flow diagram of a method of calculating the focus data of a tile at step 212 in Fig. Fig. 10 shows a flow diagram of a method of calculating image quality data of a tiled image in the image quality analysis unit 126 in Fig. 4; 200605 723504 -6- Fig. 11 shows an alternative embodiment of a flow diagram of a method of calculating image quality data of a tiled image in the image quality analysis unit 126 in Fig. 4; and Fig. 12 shows a flow diagram of a method of classifying a tile at step 220 in Fig. Description of Preferred Embodiment Where reference is made in any one or more of the accompanying drawings to steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears.

It is to be noted that the discussions contained in the "Background" section and that above relating to prior art arrangements relate to discussions of processes or devices which form public knowledge through their respective publication and/or use. Such should not be interpreted as a representation by the present inventor(s) or patent applicant that such processes or devices in any way form part of the common general knowledge in the [he described arrangements assess individual frames in a video sequence of frames of image data on the basis of the quality of the frames. Each frame is divided into tiles of pixels, and the quality of the frame is assessed by estimating the amount of motion and the degree of focus within each tile, as well as the histogram of pixel luminance values within the frame. As will be described in relation to equations and provided that certain criteria are met, a tile can be analysed for motion attributes to yield a motion magnitude MMAG, and for focus attributes to yield a focus magnitude FMAG.

Turning to the above-mentioned criteria, if the range of pixels in a particular,tile is too small the amount of detail discernable within that tile is too small to enable a meaningful analysis to be performed. In this case, meaningful estimates of motion magnitude MMAG or 200605 723504 focus magnitude FMAG cannot be made. Furthermore, meaningful estimates of focus magnitude cannot be made if there is too much motion detected in a particular tile. In such cases, an estimation of focus blur is not meaningful as additional effects will be introduced by motion blurring.

Tiles that do not have a meaningful motion or focus magnitude are classified as being "motion-undefined" or "focus-undefined" respectively. Conversely, tiles that do have a meaningful motion or focus magnitude are classified as being "motion-defined" or "focusdefined" respectively. A tile which is both "motion-defined" and "focus-defined" is classified as "known". If the motion magnitude is below a threshold of acceptable motion, and the focus magnitude is above a threshold of acceptable focus, the tile is classified as "good", and otherwise it is classified as "bad".

As will be described later, a frame is analysed to determine whether it has acceptable exposure, being neither under- or over-exposed. This analysis uses an exposure metric, EM.

A frame is classified as being of "acceptable" or "unacceptable" quality.

Classification of a frame as "acceptable" depends on three criteria: first, the proportion of known tiles must exceed a threshold; second, the proportion of good tiles must exceed a threshold; and third, the exposure metric must exceed a threshold. Additionally, each frame is assigned a quality metric, QM, that measures the overall quality of the frame. The quality metric is calculated from the exposure metric, EM, the proportions of "good" and "bad" tiles, and the focus magnitudes of the tiles that are "good".

Video frame images of the video stream 110 can be ranked by comparing their respective quality metric values QM. Unacceptable images, those images that do not 200605 723504 meet the aforementioned criteria, may be discarded as soon as the frame fails one of the three criteria. This reduces the processing required to search video data for good frames.

Fig. 1 is functional block diagram of an embedded system in a video camera for selecting good quality frames from a video data stream. An optical signal 1101 is captured by an optical/electronic conversion module 1103 and converted to an electronic data stream 110 which is also referred to as the video data stream 110. The raw captured video footage contained in the stream 110 is stored in an onboard image store 1105 which can be removable, or fixed in which case the images can be downloaded to an external computer system 1000 (see Fig. 2) via a data port (not shown). A reference numeral 1106 depicts the stored raw captured video footage.

The disclosed arrangement for identifying good quality frames in an image stream processes the data stream 110 in a quality analysis module 120. The module 120 analyses the data stream 110 in real time, and outputs quality information at 1112 to a frame identification/selection module 1113. According to one arrangement the frame identification/selection module 1113 identifies, on the basis of the quality information 1112, good quality images which are then stored according to an arrow 1111 in the image store 1105 as depicted by 1109. According to another arrangement, the frame identification/selection module 1113 generates and stores, on the basis of the quality information 1112, meta-data pointers to good quality frames in the data stream 110. The module 1113 stores the pointers according to an arrow 1110 in the image store 1105. A reference numeral 1108 depicts the stored meta-data pointers. An arrow 1107 depicts the fact that the pointer meta-data 1108 points to images in the stored raw footage 1106.

According to another arrangement, the stored image data 1106 can be provided according to an arrow 1114 to the quality analysis module 120 for off-line post processing, 200605 723504 -9instead of performing the identification of good quality images on the fly using the data stream 110.

The disclosed arrangement in Fig. 1 enables the operator of the video camera 1100 to operate a control (not shown) for an appropriate length of time in order to capture desired subject matter. In one arrangement, the captured sequence of images 110 is buffered in the memory store 1105 and subsequently processed (as depicted by the arrow 1114) to identify and select the "best" quality frame in that captured sequence 1106. In an alternate arrangement, the desired still image can be identified in the captured video data bit stream 1106, and the meta-data pointer can be inserted into metadata 1108 associated with the captured video data bit stream 1106. The pointer would point, as depicted by the arrow 1107, to the appropriate position in the stored data 1106 that corresponds to the video frame that is considered to be of good quality. The pointer(s) can be accessed from the video metadata 1109 for subsequent post-processing.

The disclosed arrangements increase the probability of capturing a better shot or image of the desired subject matter by providing a selection of candidate frames that can be processed for relative quality to thus identify the best frame capturing the desired :subject matter.

Fig. 2 shows an alternate arrangement which performs post-processing of the raw video footage 1106 from Fig. 1. In this arrangement the method of identifying good quality images is implemented as software, such as an application program, executing within -a computer system 1000. In particular, the method steps of method of identifying good quality images, as described in relation to Figs. 3, 5, and 7-11 are effected by instructions in the software that are carried out by the computer. The instructions may be formed as one or more code modules, each for performing one or more particular tasks. The software,may 200605 723504 also be divided into two separate parts, in which a first part performs the methods for identifying good quality images and a second part manages a user interface between the first part and the user. The software may be stored in a computer readable medium, including the storage devices described below, for example. The software is loaded into the computer from the computer readable medium, and then executed by the computer. A computer readable medium having such software or computer program recorded on it is a computer program product. The use of the computer program product in the computer preferably effects an advantageous apparatus for identifying good quality images.

The computer system 1000 comprises a computer module 1001, input devices such as a keyboard 1002 and mouse 1003, output devices including a printer 1015, a display device 1014 and loudspeakers 1017. A Modulator-Demodulator (Modem).transceiver device 1016 is used by the computer module 1001 for communicating to and from a communications network 1007, for example connectable via a telephone line 1006 or other functional medium. The modem 1016 can be used to obtain access to the Internet, and other network systems, such as a Local Area Network (LAN) or a Wide Area Network (,WAN), and may be incorporated into the computer module 101 in some implementations.

The computer module 1001 typically includes at least one processor unit 1005, and a memory unit 1006, for example formed from semiconductor random access memory (RAM) and read only memory (ROM). The module 1001 also includes an number of input/output interfaces including an audio-video interface 1007 that couples to the video display 1014 and loudspeakers 1017, an I/O interface 1013 for the keyboard 1002 and mouse 1003 and optionally a joystick (not illustrated), and an interface 1008 for the modem, 1016 and the printer 1015. In some implementations, the modem 1016 may.be incorporated within the computer module 1001, for example within the interface 1008. A 200605 723504 -11storage device 1009 is provided and typically includes a hard disk drive 1010 and a floppy disk drive 1011. A magnetic tape drive (not illustrated) may also be used. A CD-ROM drive 1012 is typically provided as a non-volatile source of data. The components 1005 to 1013 of the computer module 1001, typically communicate via an interconnected bus 1004 and in a manner that results in a conventional mode of operation of the computer system 1001 known to those in the relevant art. Examples of computers on which the described arrangements can be practised include IBM-PC's and compatibles, Sun Sparcstations or like computer systems evolved therefrom.

Typically, the application program for identifying good quality images, and the media item files associated with the raw captured video footage 1106, are resident on the hard disk drive 1010 and read and controlled in its execution by the processor 1005.

Intermediate storage of the program and any data fetched from the computer memory 1009 or the network 1007 may be accomplished using the semiconductor memory 1006, possibly in concert with the hard disk drive 1010. In some instances, the application program for identifying good quality images may be supplied to the user encoded on a CD-ROM, 1021 :or a floppy disk 1020 and read via the corresponding drive 1012 or 1011, or alternatively may be read by the user from the network 1007 via the modem device 1016. Still further, the software for identifying good quality frames can also be loaded into the computer system 1000 from other computer readable media. The term "computer readable medium"t as used herein refers to any storage or transmission medium that participates in providing instructions and/or data to the computer system 1000 for execution and/or processing.

Examples of storage media include floppy disks, magnetic tape, CD-ROM, a hard disk drive, a ROM or integrated circuit, a magneto-optical disk, or a computer readable card suchlas a PCMCIA card and the like, whether or not such devices are internal or external of the 200605 723504 computer module 1001. Examples of transmission media include radio or infra-red transmission channels as well as a network connection to another computer or networked device, and the Internet or Intranets including e-mail transmissions and information recorded on Websites and the like.

The method of identifying good quality frames may alternatively be implemented in dedicated hardware such as one or more integrated circuits performing the functions or sub functions of identifying good quality images. Such dedicated hardware may include graphic processors, digital signal processors, or one or more microprocessors and associated memories.

*ig. 3 shows the correspondence between sub-processes of the disclosed method and the associated drawings in this description. Given the sequence of video frames 110, a particular frame 903 is considered, as depicted by a dashed arrow 902, and is subjected, as depicted by a dashed arrow 904, to an analysis process that is initially to be described in relation to Fig. 5. The frame 903 is tiled, in a manner to be described in relation to Fig. 6, and a tile 906 is processed, as depicted by a dashed arrow 907, to produce a histogram of pixel values, a tile of difference quotients, and maximum and minimum pixel values as will be described in relation to Fig. 7. The disclosed method then proceeds, as depicted by a dashed arrow 909, to perform tile classification in relation to "motion" attributes, as will;be described in relation to Fig. 8. The disclosed method then proceeds, as depicted by a dashed arrow 911, to perform tile classification in relation to "focus" attributes, as will be described in relation to Fig. 9. The disclosed method then proceeds, as depicted by a dashed arrow 913, to perform tile classification as "good", "bad", or "unknown", as will be described in relation to Fig. 12. The aforementioned tile-based processes are repeated for all or most tiles 200605 723504 -13in the frame 903, after which the disclosed method proceeds, as depicted by a dashed arrow 915, to determine frame based parameters as will be described in relation to Fig. Fig. 4 shows a functional block diagram of the quality analysis module 120 which is used to detect attributes relating to motion, focus and quality in the video stream 110. The video sequence 110, which comprises frames of image data, is analysed in the quality analysis module 120. This analysis produces image quality data 130 which is output from the module 120 for each frame of the video 110. The input video frame image data 110 comprises an array of image pixel values for each frame. Each pixel of a frame in a typical video sequence has a number of colour components, each component being represented by 8 bits of data. In the described arrangement, each pixel is represented by one 8-bit value, typically based upon the luminance channel of the video colour space which may comprise YUV or YCbCr.

The data for the chrominance channels V or Cb, Cr) are typically discarded in this arrangement. Retaining only the luminance component of a pixel ensures that the most important information luminance) is used for analysing motion and focus. Another benefit of this approach is that the quality processing of each frame of the input data 110 is simplified and fast, and the buffer requirements on the quality analysis module 120 are reduced. i Each frame of the video frame image data 110 is passed to a motion and focus analysis unit 122. In the unit 122 the frame is divided into tiles, and motion and focus data 124 is determined for each tile of the frame. The motion and focus data 124 for the frame is further processed in an image quality analysis unit 126. In the unit 126, the image quality data 130 is determined for the frame, based on the amount and distribution of the focus and motion information among the tiles within the frame.

200605 723504 Fig. 5 shows a flow diagram of a process 218 that determines statistical data, motion data and focus data for each tile of a current frame of the video frame image data 110. The motion and focus analysis unit 122 of Fig. 4 performs the process 218 for each frame it is given. The process 218 begins at a step 200. A following step 202 divides the current frame of the video data 110 into a plurality of tiles in a manner described in relation to Fig. 6. The step 204 acquires a current tile for processing. A following step 206 determines statistics for the current tile as is described in more detail in regard to Fig. 7. A following step 222 stores the statistics of the current tile, for later use.

If a subsequent testing step 208 determines that the frame currently being processed is the first frame in a sequence of frames of the video frame image data 110, then the process 218 proceeds to a decision step 214. This test relates to the fact that comparison with previous frames is performed in the disclosed method. If the frame being considered is the first frame of the video data stream 110, or alternately the first frame of a particular video shot in the stream, then comparison cannot be performed. If the step 214 determines that the current tile is not the last tile in the current frame, then the process 218 proceeds.back to the step 204. If, however, the step 214 determines that the current tile is the last tile, in the current frame, then the process 218 terminates at a step 216.

Returning to the step 208, if it is determined that the frame currently being analysed by the motion and focus analysis unit 122 is not the first frame in a sequence of frames of the video frame image data 110, then the process 218 proceeds to a step 224 that retreives the-tile statistics of the corresponding tile in the previous video frame image. A following step 210 determines motion data for the current tile as is described in more detail in regard to Fig. 8.

The process then proceeds to a step 212 that determines focus data for the current tile as described in more detail in regard to Fig. 9. After the tile focus data is calculated by the step 200605 723504 212, the process then proceeds to a step 220 that classifies the tile as "good", "bad", or "unknown", as described in more detail in regard to Fig. 12. After the tile classification is performed by the step 220, the process proceeds to a step 214.

If it is determined by the step 214 that the current tile is not the last tile in the current frame, then the method is directed to the step 204. Otherwise, if it is determined at the step 214 that the current tile is the last tile in the current frame, then the process 218 terminates at the step 216.

Fig. 6 is a schematic diagram of a tiled video frame 300 made up only from the luminance channel Y. The width and height of the video frame image 300, in units of pixels, are W, and H, respectively. Each tile 301 of the frame 300 comprises a fixed size rectangular array of pixels having a width WT pixels and a height HT pixels. The frame 300 is divided into N. tiles across the width of the frame 300 and M tiles down the height of the frame,300, where N L(WI -3)/WTJ, [1] M=L(H -3)/HTJ, [2] and is the floor operator. The floor operator applied to a number x Lxj yields the largest integer less than or equal to x. Some pixels near the edges of the frame are not a part of any tiles, but instead are part of the frame border. Borders 302 and 305 of width two pixels are left between the top and left edges of the frame and the tiles in the frame. Borders 303 and 304 of width not less than one pixel are left between the bottom and right edges of the frame and the tiles in the frame.

ig. 7 shows a flow diagram of the process 206 in Fig. 5 for determining statistical data for the current tile. The process 206 begins at a step 400. A following step 402 generates a histogram with Nb bins for the luminance values of the tile pixels. In the typical 200605 723504 -16case, luminance values are represented as 8-bit numbers, so that the possible luminance values lie in the range 0 to 255. Thus for 0 <i Nb, the value of the i-th histogram bin is the number of pixels in the tile that have luminance value in the range from 255i Nb to 255(i Nb 1.

A following step 404 determines difference quotients (Dv)ij and (DH)ij, representing a measure of image focus, for each 0 <i HT and 0 WT as follows: (Dv)ij f(Si- 2 Si-j, Sij, Si+lj) [3] (DH)ij f(Sij-2, Sij., Sij, Sij+l) [4] where Sij is the value of the pixel with a displacement of i pixels vertically and j pixels horizontally from the top-left pixel in the current tile, and f(xo, xl, x2, x 3 is a function whose value is undefined if for any i 0, 1, 2, or 3, xi is greater than a threshold TH; otherwise, if jxi x21 is less than a threshold Tw the value of f(xo, xi, x2, x 3 is 0; otherwise, the value is given by f(xo, x, x 2 3 I 2 [4A] Ix 0 X+ I+x 1 -x 2 +x 2 x 3 Use of TH discards difference quotients in the vicinity of saturated pixels, where focus determination is unreliable. Use of Tw discards difference quotients which may be large in value due to the presence of image noise.

Note that the values of the difference quotients (Dv)ij and (DH)ij will depend on the values of pixels outside the current tile, which may lie in neighbouring tiles, or in the border.

Note also that it can be shown that the values of (Dv)ij and (DH)ij will always lie between 0 and 1.

At a following step 406, the maximum Smax and minimum Smin pixel luminance values of the current tile are obtained as follows: Smax Max (Sij) 200605 723504 -17- Smin Min (Sij) [6] where Smax and Smin are respectively the maximum and minimum pixel luminance values from Sij.

The process 206 terminates at a following step 408.

[Fig. 8 shows a flow diagram of the process 210 for calculating the motion data of the current tile at the step 210 in Fig. 5. The process 210 begins at a step 500. A subsequent decision step 502 tests whether the luminance ranges of the current tile and the corresponding previous tile are not "too small". For the luminance ranges not to be too small, the Sample Value Range (SVR) for either of the current tile and the corresponding tile of the previous video frame must exceed a predefined luminance threshold RT. This is expressed mathematically as follows: SVRc SCmax SCmin RT OR SVRP SPmax SPmin RT [8] where: SVR and SVR P are the respective pixel value ranges for the current tile of the current and previous video frames; SCmax and SPmax are the respective maximum pixel luminance values for the current tile of the current and previous video frames; and SCmin and SPmin are the respective minimum pixel luminance values for the current tile of the current and previous video frames.

If the step 502 determines that the luminance range of one of the corresponding "current" tiles of the current video frame and the previous video frame is not "too small", i.e., at least one of the equations and is satisfied, then the process 210 proceeds to a step 504 that classifies the current tile of the current frame as having "defined" motion, as 200605 723504 -18being "motion defined". Otherwise, if the luminance range of one of the corresponding "current' tiles of the current video frame and the previous video frame is found to be too small, neither one of the Equations and are satisfied, then the process 210 proceeds to a step 508. The step 508 determines if the absolute value of the difference of Mid-Range pixel Values (MRV) for the corresponding "current" tiles of the current and previous frame is not "too small". For the absolute value of the difference of Mid-Range pixel Values not to be too small, the absolute value in Equation [10] must exceed the predetermined mid-range difference threshold MRT as follows: IMRVc MRVPI MRT where: MRVc (Sc.a SCmin) 2 [11]

MRV

P (SPmax SPmin) 2 [12] and:" MRVC and MRV P are the respective Mid-Range pixel Values for the current tile of the current and previous video frames; S max and SPmax are the respective maximum pixel luminance values for the current tile of the current and previous video frames; and SCmin and SPmin are the respective minimum pixel luminance values for the current tile of the current and previous video frames.

If the absolute value of the difference between the mid-range values of the corresponding current tiles is not too small, if the Equation [10] is satisfied, then the process 210 proceeds to the step 504. Otherwise, if at the step 508 the Equation [10] is not satisfied then the process 210 proceeds to a step 510 that classifies the current tile of the current video frame as having "undefined" motion. In this case it is not possible to determine 200605 723504 -19whether any motion has occurred, because the subject matter in that tile contains no detail.

Even if the subject moved, there may be no discernable change. The process 210 is then directed to the terminating step 512.

A step 504 classifies the current tile of the current frame as having "defined" motion, as being "motion defined". The motion magnitude MMAG for the current tile is then determined by a following step 506 as follows: N J N c -H? MMAG= H-H [9] where: MMAG is the motion magnitude for the current tile; Nb is the number of histogram bins, as previously described in reference to Fig. 7; and HP and HF are the i-th histogram bins of the current and previous video frame histograms.

The process 210 then proceeds to terminating step 512.

Apart from calculating a measure of motion, Equation has an additional advantage that MMAG will increase when lighting changes, due to either a change in scene brightness or camera exposure. This will have the effect that frames during which lighting is changing are more likely to be discarded.

Fig. 9 shows a flow diagram of the process 212 for determining focus data of the current tile at the step 212 in Fig. 5. The process 212 begins at a step 600. A following step 602 determines if either the Sample Value Range (SVR) of the current tile is less than the predetermined luminance threshold RT (see Equations and or (ii) there is too much motion in the current tile. The requirement "too much motion" requires that the current tile 200605 723504 be "motion defined" AND the also that motion magnitude MMAG (see equation exceed a pre-determined motion blur threshold MBT. Thus for the answer in step 602 to be YES, either the following equation [13] must be satisfied OR the current tile must have defined motion according to the step 504 in Fig. 8 AND Equation [14] must be satisfied: SVR SCmax SCmin RT [13] OR Nb MMAG= H Hj MBT [14] i where: SVRC is the sample value range of the current tile; MMAG is the motion magnitude of the current tile of the current frame relative to the corresponding "current" tile of the process frame; and MBT is the predetermined motion blur threshold.

If the answer in step 602 is NO, then a following step 604 determines if the number of difference quotients, (Dv)ij and (DH)ij, that are defined is less than a threshold TM.

If the answer in step 604 is NO, a following step 606 classifies the current tile as having "defined" focus, as being "focus defined". Following step 606, a step 608 calculates the focus magnitude FMAG for the present tile to be the average of the TN largest difference quotients in the present tile, where TN is a constant parameter, i.e., 1 TN FMAG Xn TN n=l where the Xn are the largest TN values of (Dv)ij and (DH)i,j. Since (Dv)ij and (DH)ij have values between 0 and 1, FMAG also has a value between 0 and 1.

The process 212 is then directed to a termination step 612.

200605 723504 J 1 -21- Returning to the steps 602 and 604, if the answers to either of these steps is YES, then the process 212 proceeds to a following step 610, which attributes the current tile with "undefined" focus. The process 212 then proceeds to a termination step 612.

In the described arrangement, tiles that contain "too much" motion are classified as "focus-defined" because the focus calculations for each tile should result from focus blur only, and not motion blur. Motion may lead to high difference quotients which may falsely indicate sharp not blurred) image features.

Fig. 12 shows a flow diagram of the process 220 for classifying the current tile as either "good", "bad", or "unknown" at the step 220 in Fig. 5. The process 220 begins at a step 1200. A following step 1202 checks whether the tile has "defined" motion. If the result is NO, a following step 1214 classifies the current tile as "unknown" quality. The process is then directed to a terminating step 1212.

Returning to the step 1202, if the result is YES, then the process proceeds to a following step 1204. The step 1204 checks whether the tile has too much motion, by checking whether the motion magnitude MMAG exceeds the predetermined motion blur threshold MBT. (See Equation Thus, for the step 1204 to be YES, Equation [16] must be satisfied:

MMA

G MBT [16] If the step 1204 is YES, a following step 1216 classifies the tile as "bad" quality.

The process is then directed to a terminating step 1212.

Returning to the step 1204, if the result is NO, then the process proceeds to a following step 1206. The step 1206 checks whether the tile has "defined" focus. If the result is NO, a following step 1218 classifies the tile as "unknown" quality. The process 'is then directed to a terminating step 1212.

200605 723504 -22- Returning to the step 1206, if the result is YES, then the process proceeds to a following step 1208. The step 1208 checks whether the tile has an acceptably large focus magnitude FMAG. This occurs when the focus magnitude is larger than a predetermined focus blur threshold FBT, that is, when FMA >FBT [17] If the step 1208 is NO, then a following step 1220 classifies the tile as "bad" quality.

The process is then directed to a terminating step 1212.

Returning to the step 1208, if the result is YES, then the process proceeds to a following step 1210 which classifies the tile as "good" quality. The process is then directed to a terminating step 1212.

eturning to Fig. 4, the motion and focus data 124 that is generated by the motion and focus analysis unit 122, according to the processes described in relation to Figs. 5, 7-9, and 12, is further analysed by the image quality analysis unit 126. In the described arrangement, the data 124 typically comprises, for each tile in the current frame, a classification of each tile as being good, bad, or unknown; and the focus magnitude value FMAG, if it is defined.

Fig. 10 shows a flow diagram of a process 701 for calculating image quality data 130 for the current video frame using the motion and the focus data 124 that is produced by the motion and focus analysis unit 122 in Fig. 4. The process 701 is performed by the image quality analysis unit 128. The process 701 begins at a step 700. A following step 702 determines the number of "good" tiles in the current frame. This number is hereinafter referred to as the Good Tile Count, NGOOD. Similarly, the step 702 also determines ;the number of "bad" tiles in the current frame. This number is hereinafter referred to as the Bad Tile Count, NBAD. Additionally, the step 702 also determines the number of"unknown" tiles 200605 723504 -23in the current frame. This number is hereinafter referred to as the Unknown Tile Count,

NUNKNOWN.

A following step 704 determines, for the current frame, an exposure metric value, EM. The exposure metric is intended to measure the extent to which the frame is suitably exposed, in other words, whether the brightness and contrast levels are suitable. This is acheived by calculating the entropy of the image luminance values. The step calculates a luminance histogram fy(y) for the current frame, and uses fy(y) to set the exposure metric value to be a measure of the entropy of the luminance of the current frame. In regard to a luminance histogram counts the number of pixels with each luminance value. The pixel luminance values are typically 8-bit integers, integers in the range from 0 to 255. Thus, the luminance histogram fy(y) is defined for y 0, 1, 255 as follows: fy(y) the number of pixels whose luminance value is y. [18] In regard to an entropy measure of the luminance channel can be calculated iby: summing the values of fy(y) log(fy(y)) [18A] for each y 0, 1, 255; dividing this number by the number of pixels in the frame, ny say; subtracting the result from log(ny); and diving by log 256. The exposure metric, EM, .is set to the result of this calculation. Thus, EM 1- log(ny)- log(f(y)) [19] log 256 nY y=0 It can be shown that the value of EM will always lie between 0 and 1.

Many other exposure metrics are known in the art, and could be used here. For example, the exposure metric EM could be calculated as the deviation from a uniform histogram, thus: 200605 723504 255 EM= fy fa, [19A] y=0 where fav is the average frequency ny 256.

A following step 706 classifies the current frame as either "acceptable" or "unacceptable". There are three criteria that must be met if the frame is to be considered "acceptable". If the frame does not meet all of the criteria, then it is classified "unacceptable". The criteria are as follows.

First, the ratio of the number of "unknown" tiles to the total number of tiles must be less than a predetermined threshold TUNKNOWN, i.e., N UNKNOWN TUN N 19B] N +N +UNKNOWN NUNKNOWN+ NGOOD BAD Second, the ratio of the number of "bad" tiles to the number of known tiles tiles that are not "unknown") must be less than a predetermined threshold TBAD, i.e.,

N

NBAD TBAD [19C] NGOOD NBAD Third, the exposure metric, EM, must exceed a predetermined threshold TEM, i.e., EM TEM [19D] If all three of the criteria are satisfied, the frame is classified as "acceptable".

Conversely, if any of the three criteria are unsatisfied, the frame is classified "unacceptable".

A following step 708 calculates a quality metric, QM, for the current frame. The quality metric consists of three parts, first a exposure metric, EM, calculated in step 704, second a motion metric, MM, described below, and third a focus metric, FM, also described below.

200605 723504 As a motion metric for the frame, MM, the step 708 calculates the ratio of the number of "good" tiles, NGOOD, to the number of known tiles the number of tiles that are not "unknown"). Thus, MM NGOOD NGOO +N N GOOD N

BAD

The value of MM will always lie between 0 and 1.

As a focus metric for the frame, FM, the step 708 calculates the average of the focus magnitudes, FMAG, of the "good" tiles. Thus, FM= 1 NX i [21] NGOOD i=1 where the Xi are the focus magnitudes, FMAG, of each of the "good" tiles. Note that FM will always have a value between 0 and 1.

After the step 708 has calculated the values of MM and FM, the step 708 calculates the quality metric, QM, for the current frame, which is a weighted average of the exposure metric, the motion metric, and the focus metric. Thus, QM wEEM w MM wFMFM [22] where WEM, WMM, and WFM are positive weights whose sum is 1. Typically, WEM, WMM and WFM will all equal 1/3. The value of QM will always lie between 0 and 1.

Returning to Fig. 10, following completion of the step 708, the process 701 terminates at a step 710.

Returning to Fig. 4, the image quality data 130 that is output from the quality analysis module 120 has, for each frame, a classification of either "acceptable" or "unacceptable, and also a quality metric value, QM, between 0.0 and 1.0. A value of 0.0 represents a frame with the lowest or worst quality metric value QM or rating. A value of represents a frame with the highest or best quality metric value QM or rating.1 200605 723504 -26- In an alternative arrangement, sub-regions of some or all the video frames are analysed for "good" quality. In relation to operations requiring a comparison with data from previous frames, this approach involves comparing a current sub-region of a current frame with a corresponding sub-region of a previous frame.

For a given frame, the sub-region that contains the "best" quality rating (QR)R is selected as a "good" quality region of the particular image. Preferably, the sub-regions are rectangular and are an integer number of tiles in height and width, and each image sub-region tile corresponds with a video frame image tile. Defining the "best" quality rating for a subregion typically requires balancing the quality metric for that sub-region (QM)R against the size of the selected sub-region This can be expressed, in one example as follows: (QR)R WQMR X (QM)R WsR x (S)R [23] where WQMR and WsR are weighting factors establishing the relative importance of the subregion quality and the sub-region size. The value of (QR)R is then maximised for different sub-region rectangle sizes and positions within the corresponding video frame image. For example, the maximum value of (QR)R could be found by exhaustive search.

in an alternative arrangement, the method of analysing the quality of a frame from the video frame image data 110 not only utilises the data 124 generated by the motion and focus analysis unit 122 of the preferred arrangement, but also utilises additional information relating to the spatial distribution of motion and/or focus defined tiles in each frame. The alternative arrangement operates in substantially the same manner as the arrangement described in relation to Fig. 4, differing only in the method for calculating the image quality data, 130 in the image quality analysis unit 126.

Fig. 11 shows an alternative flow diagram of a method 801 for calculating image quality data of a tiled frame in the image quality analysis unit 126 in Fig. 5. Fig. 11 shows 200605 723504 -27the method 801 which is an alternate method to the method 701 in Fig. 10. The method 801 begins at a step 800. A next step 802 determines, across the current video frame, the spatial distribution of tiles that have been classified as having "defined" motion and focus. In this manner, an array of classification values indicating the "defined" status of each tile, together with a parameter indicating the relative location of that tile within the image, is obtained for each frame of the video frame image data 110. At a next step 804, the respective motion and focus magnitude values MMAG and FMAG of each tile are incorporated into the array. The array is then buffered for subsequent processing. The aforementioned array process is performed for each frame of the video frame image data 110.

A following step 806 determines the motion metric MM, the focus metric FM, and the exposure metric EM, for each frame. In this arrangement, a user may specify a subregion of the frame as being especially significant. For example, the user may specify:the top-left quarter of the video frame. The motion, focus, and exposure metrics for the current frame are modified so that in Equations and only the tiles that lie inside the user-specified sub-region are considered. The frame is then classified as acceptable or unacceptable, and a quality metric is calculated as described before. The process 801 then proceeds to a terminating step 808.

In another arrangement, in calculating the focus metric FM for the current frame, the Equation [21] is modified so that the focus magnitude FMAG of each tile is weighted depending on the distance of the tile in question to the centre of the current frame.. For instance, a tile at the centre of the frame can have a focus magnitude weighting of whereas a tile on the border of the frame can have a weighting of 0.0, with all tiles located between the border and centre of the frame being given an appropriate weighting value for the focus magnitude.

200605 723504 -28- In another arrangement, an alternative image quality metric QM is calculated by the step 806. The Equation [22] is modified so that the quality metric value QM is defined as a function of the distribution of the defined tiles and the distribution and value of the associated motion and focus magnitude values. One definition of the quality metric value QM for this alternate arrangement is to assign a weighting to the respective motion and focus metric values MM and FM, and also to include the dependence of the spatial distribution of the defined tiles within the image, as follows: QM Ws X (WM x (1.0 MM) WF x FM). [24] where, Ws is a weighting value derived from the spatial distribution of the defined tiles, and the other variables have their previous definitions. Thus, for instance, the weighting value W, can be defined to indicate the amount of clustering that occurs between defined motion and/or focus tiles within the video frame image. If there is a large number of defined tiles grouped into a sub-region of the frame, and the remaining sub-regions of the frame contains only undefined tiles, then the weighting and hence the quality metric, can be assigned a higher value.

In another alternative arrangement, each tile of a frame is assigned an "exposure' attribute. This attribute is assigned to a tile if that tile is undefined AND the mid-range luminance value MRV (see Equations [11] and of the tile falls within a predefined range of values. Over-exposed tiles typically exhibit a significant amount of saturated colour, and as a result, many pixel luminance values are close to white (a value of 255 for.an 8-bit sample). Under-exposed tiles typically exhibit a significant amount of darkness, and as a result, many pixel luminance values close to black (a value of 0 for an 8-bit sample). In this arrangement, the quality metric value QM of each frame is dependent on the "exposure" attribute, along with the motion and focus metric and defined tile values.

200605 723504 -29- In yet another alternative arrangement using the "exposure" attribute, the exposure attribute can be calculated based on statistics of the histogram values of some or all of the tiles in the frame. Thus, for example, corresponding bins of each tile histogram of a frame can be summed, and the resultant histogram analysed to ensure that there is a "significant" amount of data present in each bin. Too much data present in too few histogram bins would indicate a poor dynamic range of colour within the video frame. The exposure attribute need not be limited to calculations of luminance values only, but could be performed on other colour channels as well as, or in place of, the luminance component.

Suitable values for the aforementioned parameters are: WT 16 HT= 16 TN TM TH 235 Tw RT= 16 Nb= 16 MBT 0.3 FBT 0.55 TEM 4 4 TUNKNOWN 0.61 TBAD 0.78 200605 723504 Industrial Applicability It is apparent from the above that the arrangements described are applicable to the video processing industry.

The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive.

In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including", and not "consisting only of'. Variations of the word "comprising", such as "comprise" and "comprises" have correspondingly varied meanings.

200605 723504

Claims (7)

1. A method of determining a quality value of a frame, the method comprising the ci steps of: dividing the frame into a plurality of tiles: determining attributes of each said tile based upon pixel values of the tile and (ii) ci opixel values of a corresponding tile of a preceding frame; and ci oestablishing the quality value of the frame by: Ci testing the tile attributes of the frame against pre-derermined criteria; and defining the quality value of the frame depending upon results of said testing; wherein: the attributes comprise a motion definition state and a motion magnitude, these being determined by: classifying the tile as having a motion defined state if one of the 1s range of luminance values of pixels in the tile or in the corresponding tile of the preceding ftame exceeds a pre-detenined luminance threshold; and (ii) the difference between a mid- range pixel luminance value for the tile and the mid-range pixel luminance value for the corresponding tile of the preceding frame exceeds a pre-deterinined mid-range difference threshold; and determining, if the tile is classified as having a motion defined state by the step said motion magnitude dependent upon a difference in pixel value distributions between a pixel value histogram of the tile and a pixel value histogram of the corresponding tile in the previous frame; and wherein: 11550431 723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12

12-MAR2008 18:02 SPRUSON FERGUSON 92615486 NO. 8475 P. 14 00 -32- 0 o the attributes further comprise a focus definition state and a focus magnitude, these being determined by: classifying the tile as having a focus undefined state if the intensity range of pixels in the tile is too small, or there is too much motion in the tile or (ii) the number of defined difference quotients in the tile is less than a threshold; and determining, if the tile is not classified as having a focus undefined state Sby the step the focus magnitude depending upon the largest difference quotient values of o pixel luminance of the tile. 2. A method according to claim 1, wherein if the tiles of the frame do not meet the criteria of the testing step, the defining step defines a default quality value for the frame. 3. A method according to claim 1, wherein a said pixel value comprises the luminance value of the pixel which is represented in one of YUV and YCbCr video colour space. 4. A method according to claim 1, wherein: the frame is one of a sequence of video frames; the method is applied to each frame in the sequence; and the best frame in the sequence is identified as that frame having the best quality value. A method according to claim 3, wherein; the attributes comprise a motion definition state and a motion magnitude, these being determined by: 11550431 723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 12.MAR. 2008 18:09 q P IQ n f) cc9 in 1^ r,, v, rci uo Yzbl4DI b NO. 8475 P. 00 -33- S(a) classifying the tile as having a motion defined state if one of the c range of luminance values of pixels in the tile or in the corresponding tile of the preceding frame exceeds a pre-determined luminance threshold; and (ii) the difference between a mid- crange pixel luminance value for the tile and the mid-range pixel luminance value for the corresponding tile of the preceding frame exceeds a pre-determined mid-range difference Sthreshold; and 0 determining, if the tile is classified as having a motion defined state by g the step said motion magnitude dependent upon a difference in pixel value distributions between a pixel value histogram of the tile and a pixel value histogram of the corresponding tile in the previous frame; and wherein: the attributes further comprise a focus definition state and a focus magnitude, these being determined by: classifying the tile as having a focus undefined state if the intensity range of pixels in the tile is too small, or there is too much motion in the tile or (ii) the nurnbe of defined difference quotients in the tile is less than a threshold; and determining, if the tile is not classified as having a focus undefined state by the step the focus magnitude depending upon the largest difference quotient values of pixel luminance of the tile. 6. A method according to claim 5, wherein the criteria are; that the number of tiles in the frame having a motion defined state exceed a pre- defined motion defined threshold; 1155043_1/ 723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 12-MAR-2008 18:02 12. MA. 2%313:Q2SRUSON FERGUSON 92615436 N.37 .1 NO, 8475 P. 16 00 -34- o that the ratio of tiles in the frame having a motion defined state whose motion magnitude exceeds a pro-defned motion threshold, to (ii) the total numnber of tiles in the fr-ame having a motion defined state, exceeds a proe-defined motion defined threshold; that the number of tiles in the frame having a focus defined state exceeds a pro- defined focus defined threshold; and R that the average of the largest focus magnitude values of tiles in the frame having o a focus defined state exceeds a pro-defined focus defined threshold. Cl7. A method according to claim 6, wherein if the tiles of the frame meet all the criteria of the testing step, the quality value depends upon the ratio and the average determined in the steps and (di) of claim 6. 8. A method according to claim 1, wherein: the quality value is determined based upon a sub-region of the frame; and the dividing step divides the sub-region into the plurality of tiles. 9. A method according to claim 8, wherein the defining step is dependent upon a weight establishing the importance of the sub-region in the frame, and a weight establishing the size of the sub-region relative to the frame. A method according to claim 6, wherein the criterion is dependent upon the spatial location of tiles in the frame wherein: the criterion is that the ratio of tiles in a pre-defined region of the frame having a motion defined state whose motion magnitude exceeds a pro-defined motion 1155043_11723504 050308 COMS ID No: ARCS-i 82711 Received by IP Australia: Time 18:14 Date 2008-03-12 12. MAR. 2008 i8:03 SPRUSON FERGUSON 92615436 NO. 8475 P. 17 00 o threshold, to (ii) thie total number of tiles in the frame having a motion defined state, exceeds a pre-defined motion defined threshold, 11. A method according to claim 6, wherein the criterion is dependent upon the spatial location of tiles in the frame wherein: the criterion is that the average of the largest focus magnitude values of tiles in o the frame having a focus defined state exceeds a pre-defined focus defined threshold, o wherein said focus magnitude values are weighted according to the relative distance of Ci respective tiles from the centre of the fr-ame. 12. A method according to claim 7, wherein the quality value futher depends upon a spatial distribution of the tiles having one of a motion defined state and a focus defined state.

13. An apparatus for determining a quality value of a frame, the apparatus comprising: means for dividing the frame into a plurality of tiles: means for determining attributes of each said tile based upon pixel values of the tile and (ii) pixel values of a corresponding tile of a preceding frame; and means for establishing the quality value of the fr-amle comprising: means for testing the tile attributes of the frame against pre-determined criteria; and means for defining the quality value of the frame depending upon results of said testing; wherein: the attributes comprise a motion definition state and a motion magnitude, these being determined by: 11550431 /723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 12-MAR-2008 18:03 12. AR. QQ3 3:U3SPRUSON FERGUSON 92615436 N.37 .1 NO. 8475 P. 18 0c' -36- o means for classifying the tile as having a motion defined state if one of the range of luminance values of pixels in the tile or in the corresponding tile of the preceding frame exceeds a pro-determined luminance threshold; and (ii) the difference between a mid-range pixel luminance value for the tile and the mid-range pixel luminance value forT the corresponding tile of the preceding frame exceeds a pre-determined mid-range difference threshold; and o means for determining, if the tile is classified as having a motion o defined state by the step said motion magnitude dependent upon a difference in pixel CA value distributions between a pixel value histogram of the tile and a pixel value histogram of the corresponding ie in the previous frame; and wherein: the attributes further comprise a focus definition state and a focus magnitude, these being determined by: means for classifying the tile as having a focus undefined state if the intensity range of pixels in the tile is too small, or there is too much motion in the tile or (Hi) the number of defined difference quotients in the tile is less than a threshold; and means for determining, if the tile is not classified as having a focus undefined state by the step the focus magnitude depending upon the largest difference quotient values of pixel luminance of the tile.

14. A computer program product having a computer readable medium having a computer program recorded therein for directing a processor to execute a method of determining a quality value of a frame, said computer program product comprising: code for dividing the frame into a plurality of tiles: 11550431 1723504 050308 COMS ID No:ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 12-MAR2008 18:03 SPRUSON FERGUSON 92615486 NO. 8475 P. 19 00 -37- 0 o code for determining attributes of each said tile based upon pixel values of the tile and (ii) pixel values of a corresponding tile of a preceding frame; and code for establishing the quality value of the frame comprising: code for testing the tile attributes of the frame against pre-determined criteria; and code for defining the quality value of the frame depending upon results of o said testing; wherein: the attributes comprise a motion definition state and a motion magnitude, these C being determined by: code for classifying the tile as having a motion defined state if one of(i) the range of luminance values of pixels in the tile or in the corresponding tile of the preceding frame exceeds a pre-determined luminance threshold; and (ii) the difference between a mid-range pixel luminance value for the tile and the mid-range pixel luminance value for the corresponding tile of the preceding frame exceeds a pre-determined mid-range difference threshold; and code for determining, if the tile is classified as having a motion defined state by the step said motion magnitude dependent upon a difference in pixel value distributions between a pixel value histogram of the tile and a pixel value histogram of the corresponding tile in the previous frame; and wherein: the attributes further comprise a focus definition state and a focus magnitude, these being determined by: code for classifying the tile as having a focus undefined state if the intensity range of pixels in the tile is too small, or there is too much motion in the tile or (ii) the number of defined difference quotients in the tile is less than a threshold; and 11550431/ 723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 12-MAR-2008 IB:04 SPRUSON FERGUSON 92615486 NO. 8475 P. 00 -38- S(d) code for determining, if the tile is not classified as having a focus undefined state by the step the focus magnitude depending upon the largest difference Squotient values of pixel luminance of the tile.

15. A video frame selected from a sequence of video frames dependent upon a quality value of the frame, the frame being selected by a method of determining a quality value of a C video frame, said method being applied to each frame in the sequence, said method Scomprising, in regard to a particular frame in the sequence, the steps of: C dividing the frame into a plurality of tiles: determining attributes of each said tile based upon pixel values of the tile and (ii) pixel values of a corresponding tile of a preceding frame; and establishing the quality value of the frame by: testing the tile attributes of the frame against pre-determined criteria; and defining the quality value of the frame depending upon results of said testing; wherein: the attributes comprise a motion definition state and a motion magnitude, these being determined by: classifying the tile as having a motion defined state if one of the range of luminance values of pixels in the tile or in the corresponding tile of the preceding frame exceeds a pre-determined luminance threshold; and (ii) the difference between a mid- range pixel luminance value for the tile and the mid-range pixel luminance value for the corresponding tile of the preceding frame exceeds a pre-determined mid-range difference threshold; and 1155043_1 /723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 12-MAR-2008 18:04 12. AR. 003 3:04SPRUSON FERGUSON 92615436 NO. 3475 P. 21 00 -9 deternining, if the tile is classified as having a motion defined state by ci the step said motion magnitude dependent upon a difference in pixel value distributions between a pixel value histogram of the tile and a pixel value histogramn of the corresponding tile in the previous frame; and wherein-, s the attributes fuirther comprise a focus definition state and a Locus magnitude, these being determined by: classifying the file as having a focus undefined state if the intensity range of pixels in the file is too small, or there is too much motion in the, file or (ii) the number of defined difference quotients in the tile is less than a teshold; and determining if the tile is not classified as having a focus undefined state by the step the focus magnitude depending upon the largest difference quotient values of pixel luminance of the tile.

16. A method of estimating quality of an image in a video, the video comprising a plurality of images, each image comprising a plurality of pixels, said method comprising the steps of: arranging, for a current image of said video, pixels of the current image into a plurality of tiles, each said tile containing a plurality of pixels; determining the number of the files of the current image having a defined attribute; and estimating quality of the current image based on the number of said tiles having the attribute as a proportion of the number of tiles in the current image; wherein: said attribute is one of a miotion attribute, a focus attribute, and an exposure attribute; 1155043_1/723504 050308 COMS ID No: ARCS-18271 1 Received by IP Australia: Time 18:14 Date

2008-03-12 12-MAR-2008 18:04 12. AR. U08 8:U4SPRUSON FERGUSON 92615436 N.87 .2 NO.8475 P. 22 00 -40 o said motion attribute classifies a tile as either motion-defined or motion-undefined; and wherein: determining the number of tiles of the current image having said motion attribute comprises the steps of: obtaining, for each colour channel of tile pixels, a respective range of sample values, and; 0 setting a motion attribute of Maid tile of said curent image to indicate that the o tile is motion-undefined if- Cl said colour channel range of said tile of said current image does not 1O exceed a colour channel range threshold value, and; said colour channel range of a corresponding dile of a previous image does not exceed said colour channel range threshold value, and; the absolute difference between a colour channel mid-range value of said tile of said current imnage and a corresponding colour channel mid-range value of said is corresponding tile of said previous image does not exceed a colour channel mid-range difference threshold value; otherwise setting said motion attribute of said tile of said current image to indicate that said tile is motion-defined. 17- A method of selecting an image from a video comprising a plurality of images, each image comprising a plurality of pixels, and said method comprising the steps of, performing, for each image of said plurality of images, the steps of: arranging pixels of the image into a plurality of tiles, each said file containing a plurality of pixels; 1155043_1/1723504 050308 COMS ID No: ARCS-18271 1 Received by IP Australia: Time 18:14 Date 2008-03-12 12. MAR. 2008 18:05 SPRUSON FERGUSON 92615486 NO. 8475 P. 23 00 -41- 0 o determining the number of the tiles of the image having a defined attribute; and estimating quality of the image based on the number of said tiles having the attribute as a proportion of the number of tiles in the image; and selecting an image from said video in accordance with the estimated quality; wherein: o said attribute is one of a motion attribute, a focus attribute, and an exposure attribute; o said motion attribute classifies a tile as either motion-defined or motion-undefined; C and wherein: determining the number of tiles of the current image having said motion attribute comprises the steps of: obtaining, for each colour channel of tile pixels, a respective range of sample values, and; setting a motion attribute of said tile of said current image to indicate that the tile is motion-undefined if: said colour channel range of said tile of said current image does not exceed a colour channel range threshold value, and; said colour channel range of a corresponding tile of a previous image does not exceed said colour channel range threshold value, and; the absolute difference between a colour channel mid-range value of said tile of said current image and a corresponding colour channel mid-range value of said corresponding tile of said previous image does not exceed a colour channel mid-range difference threshold value; otherwise 1155043_1/ 723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 12 MAR, 2008 18:05 SPRUSON FERGUSON 92615486 NO, 8475 P. 24 O0 -42- 0 setting said motion attribute of said tile of said current image to indicate that said tile is motion-defined. 18. A method according to claim 16 or claim 17 wherein said characteristic is dependent on luminance. 19. A method according to claim 16 or claim 17 wherein said tiles of said current image ocomprise a predetermined number of pixels Np. 20. A method according to claim 17 wherein said motion characteristic establishes a motion magnitude in said tile if said tile is determined to be motion-defined. 21. A method according to claim 17 wherein said focus characteristic classifies a tile as either focus-defined or focus-undefined. 22. A method according to claim 17 wherein said focus characteristic establishes a focus magnitude in said tile if said tile is detemzined to be focus-defined. 23. A method according to claim 22 wherein determination of the focus characteristic of a tile comprises the steps of: generating, for each sample of a colour channel, a difference quotient for said sample value in the horizontal and vertical directions; and 11550431 723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12 12. MAR. 2008 18:05 SPRUSON FERGUSON 92615436 NO. 8475 P. 00 -43- o defining said focus magnitude of said tile to be equal to the maximumi absolute value of said difference quotient values of said tile if said tile of said current image is focus- defined. 24. A method according to claims 16 wherein determining said proportion comprises the steps of- o determining the respective numbers of tiles in the image having respective good, bad and unknown classifications; c-i determining a quality metric value for said image; and classifing the frame as being one of acceptable and unacceptable dependent upon (i) the respective uwmbers of tiles in the inage having good, bad and unknown classification and (Hi) the quality metric value. A method according to claim 24 wherein said quality metric value is an image motion metric value defined as the number of motion-defined tiles wit a motion magnitude greater than a mnotion magnitude threshold value, normalised against the total number of mnotion-defined tiles. 26. A method according to claim 24 wherein said quality metric value is an image focus metric value defined as the avenage of the focus magnitudes of focus-defined tiles with a focus magnitude greater than a focus magnitude threshold value. 27. A meteod according to claim 24 wherein said quality metric value is an image exposure metric value dependent upon the entropy of the luminance values of the frame. 11 55043_1 723504 050308 COMS ID No: ARCS-18271 1 Received by IP Australia: Time 18:14 Date 2008-03-12 12, MAR. 2008 18:05 SPRUSON FERGUSON 92615486 NO. 8475 P. 26 00 -44- 0 0 28. A method according to claims 16 wherein said quality of the current image is defined as the weighted contribution from motion, focus and exposure characteristic values. 29. A method according to claim 28 wherein said quality of current image is assigned a normalised value of 0.0 for the worst quality image, 1.0 for the best quality image, oand corresponding intermediate values for images of intennediate quality, Ci 30. A method according to claim 29 wherein said quality of a current image is classified as acceptable if any of the following criteria arc satisfied: the proportion of known tiles exceeds a first threshold value; the proportion of good tiles exceeds a second threshold value; and the exposure metric exceeds a third threshold value. 31. A method according to claim 17 wherein: the estimated quality of each image is associated with a corresponding quality metric value; and said selecting of said image comprises the steps of: ranking quality metric values for said each image of the plurality of images; and selecting the image with the highest rank as the best image. DATED this l2 Day of March 2008 CANON KABUSHIKI KAISEA Patent Attorneys for the Applicant SPRUSON&FERGUSON 1155043_1/723504 050308 COMS ID No: ARCS-182711 Received by IP Australia: Time 18:14 Date 2008-03-12