AU2008264229B2 - Partial edge block transmission to external processing module - Google Patents

Description

S&F Ref: 880833 AUSTRALIA PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT Name and Address Canon Kabushiki Kaisha, of 30-2, Shimomaruko 3 of Applicant: chome, Ohta-ku, Tokyo, 146, Japan Actual Inventor(s): Peter Jan Pakulski Address for Service: Spruson & Ferguson St Martins Tower Level 35 31 Market Street Sydney NSW 2000 (CCN 3710000177) Invention Title: Partial edge block transmission to external processing module Associated Provisional Application Details: [33] Country: [31] Appl'n No(s): [32] Application Date: AU 2008249180 24 Nov 2008 The following statement is a full description of this invention, including the best method of performing it known to me/us: 5845c(1 9099421) -1 PARTIAL EDGE BLOCK TRANSMISSION TO EXTERNAL PROCESSING MODULE RELATED APPLICATION This application claims priority from Australian Patent Application No. 2008249180 entitled "Multiple JPEG coding of same image", filed on 24 November, 2008 in the name of Canon Kabushiki Kaisha, the entire contents of which are incorporated herein by s reference. TECHNICAL FIELD The present disclosure relates generally to video analytics and, in particular, to the selection and transmission of information from a camera to a client for further processing. DESCRIPTION OF BACKGROUND ART 10 Video cameras, such as Pan-Tilt-Zoom (PTZ) cameras, are omnipresent nowadays, and are often utilised for surveillance purposes. The cameras typically capture more data (video content) than human viewers can process. Automatic analysis of video content is therefore needed. One step often used in the processing of video content is the segmentation of video is data into foreground objects and a background scene, or background. Such segmentation allows for further analysis of the video content, such as detection of specific foreground objects, or tracking of moving objects. Such further analysis may, for example, result in sending an alert to a security guard, perhaps upon detection of a foreground object or tracking an object entering or leaving a predefined area of interest. 20 Commonly, the video content is available in a compressed format. For efficiency, it is possible to perform the foreground/background segmentation in the compressed domain. Where the compressed video stream is available as a stream of JPEG files, for example, the segmentation can be done in the Discrete Cosine Transform (DCT) domain of the JPEG file to avoid a need to decode the file completely. The segmentation in such a case is 25 performed on a per-block basis. To save transmission bandwidth and storage space, the video content is not necessarily sent in full from a camera for processing and analysis. The video content may 10066 I .00oC M03 -2 be sent as a reduced set of frames, or at a lower resolution, or not at all. When this happens, the analysis results can be lacking in support data and context. One goal of analysing the compressed data can be to compensate for the compression used. An example of such processing is to refine a boundary between a foreground and a s background to a level of detail finer than the DCT blocks on which the boundary was initially calculated. Two aspects of video content analysis are of particular interest. First is the detection of abandoned objects. An example of an abandoned object is an object, such as an item of luggage, that has been brought into a scene that is being monitored over a sequence of 1o video frames and wherein the object is subsequently left in the scene. Second is the detection of object removal, Object removal relates to detecting that an object, which was previously considered part of the background of a scene that is being monitored over a sequence of video frames, has been removed from the scene. A common approach to foreground/background segmentation is background 15 subtraction. Both the removal of objects which previously formed part of the background, and the insertion of new, stationary foreign objects, result in an image region where the image is different from the remembered background, but is otherwise not changing. A difficulty is that the two types of change are not easily distinguishable from each other. It is possible to differentiate between the two events through examination of the image data at 20 the boundaries of the detected regions of change, A consistent boundary around the region of change indicates a high probability that an object has been inserted, whereas an inconsistent boundary around the region of change indicates a high probability that an object has been removed, as the region boundary is not tracing out an object boundary. Thus, a need exists to provide an improved method for image processing. 2$ SUMMARY It is an object of the present invention to overcome substantially, or at least ameliorate, one or more disadvantages of existing arrangements. According to a first aspect of the present disclosure, there is provided a computer-implementable method for processing a video frame defined by a plurality of n0 visual elements, wherein each of the visual elements is associated with a plurality, n, of coefficients. The method detects at least one object in the video frame, with each of the detected objects being associated with a region of at least one visual element. For each of I~)04t6 JX)C ~OS33,spu~ -3 the regions, the method determines a set of boundary visual elements and identifies, for each boundary visual element in the set of boundary visual elements for the region, a subset of dominant coefficients selected from the plurality of coefficients associated with the boundary visual element, The subset includes a number, m, of dominant coefficients, 5 wherein 0 < m <n. The method then transmits to an external processing device the subset of dominant coefficients. According to a second aspect of the present disclosure, there is provided a computer readable storage medium having recorded thereon a computer program for processing a video frame defined by a plurality of visual elements, each of the visual elements being 10 associated with a plurality, n, of coefficients. The computer program product includes: code for detecting at least one object in the video frame, each of the detected objects being associated with a region of at least one visual element; code for processing each region to: determine a set of boundary visual elements; and is identify, for each boundary visual element in the set of boundary visual elements for the region, a subset of dominant coefficients selected from the at least one coefficient associated with the boundary visual element, wherein the subset includes a number, m, of dominant coefficients, wherein 0 < m < n; and code for identifying the dominant coefficients as a set of image processing 20 information associated with the video frame. According to a third aspect of the present disclosure, there is provided a method for obtaining a refined boundary of a region of an image captured by an image capture device, wherein the image is represented by a plurality of blocks and each of the blocks is associated with a plurality, n, of coefficients. The method includes the steps of: 25 (a) at a first processing module on the image capture device: detecting at least one object in the video frame, each of the detected objects being associated with a region of at least one block; determining a set of boundary blocks for each of the regions associated with the detected objects; and 30 identifying a subset of dominant coefficients for each of the boundary blocks, wherein the subset includes a number, m, of dominant coefficients, wherein 0 < m <n; 19%68r6 LDOC 8&i533-speci -4 (b) transmitting the subset of dominant coefficients; and (c) at a second processing module on an external processing device: receiving the transmitted subset of dominant coefficients; and processing the subset of dominant coefficients to obtain a refined 5 representation of the boundary of each of the regions associated with the detected objects. According to a fourth aspect of the present disclosure, there is provided a method for classifying a region of an image captured by an image capture device as one of an abandoned object event and an object removal event, wherein the image is represented by a 10 plurality of blocks, each of the blocks being associated with a plurality, n, of coefficients. The method includes the steps of: (a) at a first processing module on the image capture device: detecting at least one object in the video frame, each of the detected objects being associated with a region of at least one block; 15 determining a set of boundary blocks for each of the regions associated with the detected objects; and identifying a subset of dominant coefficient for each of the boundary blocks, wherein the subset includes a number, m, of dominant coefficients, wherein 0<m < n; 20 (b) transmitting the subset of dominant coefficients; and (c) at a second processing module on an external processing device: receiving the transmitted subset of dominant coefficients; and processing the subset of dominant coefficients to classify each of the regions associated with the detected objects as a one of an abandoned object 25 event and an object removal event, According to another aspect of the present disclosure, there is provided an apparatus for implementing any one of the aforementioned methods. According to another aspect of the present disclosure, there is provided a computer program product including a computer readable medium having recorded thereon a 30 computer program for implementing any one of the aforementioned methods. Other aspects of the invention are also disclosed.

-5 BRIEF DESCRIPTION OF THE DRAWINGS One or more embodiments of the invention will now be described with reference to the following drawings, in which: Fig. 1 is a scheatic block diagram of an electronic system on which one or more Described arrangements for detecting abandoned object and object removal events can be practised; Fig. 2 is a schematic representation illustrating a video scene in which an object is abandoned; Fig. 3 is a schematic representation illustrating a video scene in which an object is 10 removed; Fig. 4 is a schematic representation illustrating a region of change detected by a camera for both abandoned object and object removal events; Fig. 5 is a schematic flow diagram illustrating a method of distinguishing abandoned object and object removal events, according to one embodiment 15 of the present disclosure; Fig. 6 is a schematic flow diagram illustrating a method of traversing boundary blocks of a detected region of change; Fig. 7 is a schematic flow diagram illustrating a method of iterating successive boundary blocks of a detected region of change, as used in the method of 20 Fig. 6; Fig. S is a schematic flow diagram illustrating a method of determining whether a DOT block is a boundary DoT block, as used in the method of Fig. 7; Fig. 9 is a schematic representation illustrating an example of a detected region of change with boundary DOT blacks highlighted; 25 Fig. 10 is a schematic representation illustrating predicted edge characteristics of a detected region of change, according to one embodiment of the present disclosure; Fig. 11 is a schematic representation illustrating an example of a DOT coefficient table arranged in a zig-zag patten; 30 Fig. 12 is a schematic representation illustrating observed edge characteristics of a detected region of change for an abandoned object event, according to one embodiment of the present disclosure; is aschmatc fow dagrm ilusratng ametod f tavesg8boundary -6 Fig. 13 is a schematic block diagram of a camera upon which the methods of Figs 2 to 12 may be practised; Figs 14A and 14B form a schematic block diagram of a general purpose computer system upon which the arrangements described can be practised; s Fig. 15 is a schematic representation illustrating an example of a detected region of change within a frame, and further shows the boundary blocks highlighted as found by the method of Fig. 8; Fig. 16 is a schematic flow diagram illustrating a method for processing a video frame, in accordance with an embodiment of the present disclosure; Fig. 17 is a schematic block diagram illustrating a method of refining the foreground / background boundary, in accordance with one embodiment of the present disclosure; and Fig. 18 is a schematic block diagram illustrating a component of Fig. 17, in which the properties of the coefficients of the original block data are used, 15 DETAILED DESCRIPTION Where reference is made in any one or more of the accompanying drawings to steps and/or features that have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears. 20 A video image capture device, or video camera, captures information as a sequence of images or frames. The amount of information captured in each frame is determined by the resolution of the camera. The amount of information processed, stored and transmitted in each frame is then detennined by the quality level, which is commonly a function of the image representation and compression. Information with different quality levels may be 25 used for the processing and transmission respectively. The total amount of information is determined by the number of frames collected over time, As indicated above, the information, otherwise known as video content, captured over time in a video frame sequence typically exceeds the amount of information that a human viewer can process. Consequently, automatic analysis of video content is often required, 30 Some video cameras have limited processing capabilities for analysing video content. Accordingly, it is common for a video camera to capture a full resolution image and then transmit the video content of that image to a client, or external processing device, for 1 6 )O.. oc 'DOC:spc i -7 processing. It is often desirable to reduce the quantity of video information to be transmitted from a camera to a remote client, in order to reduce storage and transmission costs. Accordingly, video cameras often support the transmission of video content at reduced quality levels and lower resolutions, Such reduced quality levels can include, for s example, a JPEG-compressed representation of video frames with chroma subsampling and coarse quantisation matrices. Such reduced resolutions can include, for example, half or quarter of the resolution of the camera. Situations arise when a client only has access to a lower quality level or resolution image that has been transmitted from a camera, yet the client is required to perform image 10 processing tasks that require image data of a different quality level or resolution than has been made available. Such image processing tasks may include, for example, classifying a detected region of change in a video frame as an abandoned object event or an object removal event, or refining one or more edges of an object to produce an accurate outline of the object, such as may be required for accurate background replacement. 15 According to one embodiment of the present disclosure, there is provided a computer-implementable method for processing a video frame defined by a plurality of visual elements. The visual elements may be pixels, Discrete Cosine Transform (DCT) blocks, or wavelet transforms, for example, Each of the visual elements is associated with a plurality, n, of coefficients. The method detects at least one object in a scene depicted in 20 the video frame. Each detected object is associated with a region of the video frame, wherein the region includes at least one visual element. A region can be any shape, including free-form shapes, and is not restricted to being a regular polygon, For each of the regions associated with the detected objects, the method determines a set of boundary visual elements and identifies, for each boundary visual element in the set of boundary 25 visual elements, a subset of at dominant coefficients. As indicated above, each visual element is associated with a plurality, n, of coefficients. The subset of dominant coefficients is selected from the plurality, n, coefficients associated with each boundary visual element, such that the subset of dominant coefficients includes a number, m, of dominant coefficients, wherein 0 < m <n. Thus, the set of dominant coefficients is a strict 30 subset of the plurality, n, of coefficients, as the set of dominant coefficients includes at least one coefficient, but does not include all of the plurality, n, of coefficients. The method then transmits, to a remotely located client, the subset of dominant coefficients, 100686 DOc, 88 0S 3 3_spcci -8 The subset of dominant coefficients constitute image processing information associated with the video frame. An embodiment of the present disclosure enables further processing of object information to be performed off-camera, without requiring that all information relating to a s full resolution video frame be transmitted to an off-camera, remotely located client. One example of an image processing task is distinguishing abandoned object events from object removal events. It is advantageous for a surveillance or monitoring system to be able to draw an operator's attention to such events, and to be able to give different alerts based on the type of event that has occurred. In many video surveillance settings, these two events 10 appear indistinguishable from each other from a camera's perspective. Another example of an image processing task is refining the detail of the boundary of the object regions to a level finer than the blocks at which the boundary is defined. This is advantageous for visualisation, and tasks such as background subtraction where the visual appearance of the segmentation is important. is In one embodiment, the subset of dominant coefficients is transmitted in conjunction with a full resolution image, wherein the full resolution representation of the video frame includes coefficients at a quality level that is lower than a quality level of the dominant coefficients. In another embodiment, the subset of dominant coefficients is transmitted in conjunction with a reduced resolution image. In another embodiment, the reduced set of 20 image processing information is transmitted in conjunction with a reduced quality level image. In yet another embodiment, the reduced set of image processing information is transmitted in conjunction with a different quality level or representation of the image. According to another embodiment of the present disclosure, there is provided a computer readable storage medium having recorded thereon a computer program for 25 processing a video frame defined by a plurality of visual elements, wherein each of the visual elements is associated with a plurality, n, of coefficients. The computer program product includes code for detecting at least one object in the video frame, each detected object being associated with a region of at least one visual element. The computer program product also includes code for processing each region to: determine a set of boundary .3 visual elements; and identify, for each boundary visual element in the set of boundary visual elements for the region, a subset of dominant coefficients selected from the plurality of coefficients associated with the boundary visual element, wherein the subset includes a 1906806. 1rx 880933,ppeci -9 number, m, of dominant coefficients, such that 0 <m <n. The computer program product further includes code for identifying the dominant coefficients as a set of image processing information associated with the video frame. According to another embodiment of the present disclosure, there is provided a s method for obtaining a refined boundary of a region of an image captured by an image capture device, wherein the image is represented by a plurality of blocks, and each of the blocks is associated with a plurality, n, of coefficients. At a first processing module on the image capture device, the method detects at least one object in the video frame, wherein each detected object is associated with a region of at least one block. The first processing 10 module also determines a set of boundary blocks for each of the regions associated with the detected objects and identifies a subset of dominant coefficients for each of the boundary blocks. The subset includes a number, m, of dominant coefficients, wherein 0 < m < n. The method then transmits the dominant coefficients. A second processing module on an external processing device receives the transmitted subset of dominant coefficients and 15 processes the subset of dominant coefficients to obtain a refined representation of the boundary of each of the regions associated with the detected objects. According to a further embodiment of the present disclosure, there is provided a method for classifying a region of an image captured by an image capture device as one of an abandoned object event and an object removal event. The image is represented by a 2o plurality of blocks, wherein each of the blocks is associated with a plurality, n, of coefficients. A first processing module on the image capture device detects at least one object in the video frame, wherein each detected object is associated with a region of at least one block. The first processing module then determines a set of boundary blocks for each of the regions associated with the detected objects and identifies a subset of dominant 25 coefficients for each of the boundary blocks. The subset includes a number, m, of dominant coefficients, wherein 0 < m <n. The method then transmits the dominant coefficients. A second processing module on an external processing device receives the subset of dominant coefficients and processes the subset of dominant coefficients to classify each of the regions associated with the detected objects as a one of an abandoned 3o object event and an object removal event I906H66. i.I)OC 8 S0933. spec i -10 Boundary Information Transmission Fig. 16 is a flow diagram 1601 that illustrates a method for identifying and transmitting boundary information relating to objects detected in a video frame. The video frame is represented by a plurality of visual elements in the form of blocks. The process s begins at a Start step 1600, and control passes to step 1610, which performs block-based obj ect-detection. The result of the object detection step 1610 is a collection of one or more regions representing objects in the scene. Each region includes at least one block. Step 1620 identifies a set of boundary blocks for each of those regions identified in step 1610. In one embodiment, identification of the set of boundary blocks is performed by scanning 10 each block in a region and classifying the blocks as either a boundary block or a non-boundary block, as is described with reference to Fig. 8. In another embodiment, identification of the set of boundary blocks is performed as described with reference to Fig. 6. The dominant AC coefficients of each of the identified boundary blocks are in turn identified in step 1630. In one embodiment, the coefficients used are as shown in Fig. 11. is In another embodiment, the coefficients identified comprise AC1O, ACO1, ACI 1, AC20, AC02, In yet another embodiment, the coefficients identified are dependent on a user setting. Once a set of coefficients of boundary blocks has been identified, those coefficients are transmitted in step 1640, in the form of a set of reduced image processing information 20 associated with the video frame, to an external processing device. In one embodiment the external processing device is a computer server. In another embodiment, the external processing device is a desktop Personal Computer (PC). In yet another embodiment, the external processing device is a custom hardware item. In yet another embodiment, the external processing device is another camera. In a further embodiment, the external 2s processing device is a general purpose computer system as described in Fig. 14A and Fig. 14B. In step 1650, the external processing device performs required processing of the transmitted image processing information in the form of the subset of dominant coefficients, and the process ends at step 1660. In one embodiment, the processing performed in step 1650 relates to the classification of abandoned object events and object 30 removal events. In another embodiment, the processing performed in step 1650 is an edge refinement stage. 1 IXW 6. 1.POC,$8SA~c 11 Description of Embodiments A video is a sequence of images orfranes, Thus, each frame is an image in an image sequence. Each frame of the video has anx axis and ay axis. A scene is the information contained in a flame and may include, for example, foreground objects, background 5 objects, or a combination thereof. A scene model is stored information relating to a background. A scene model generally relates to background information derived from an image sequence. A video may be encoded and compressed. Such encoding and compression may be performed intra-frarne, such as motion-JPEG (M-JPEG), or inter frame, such as specified in the H.264 standard. 10 An image is made up of visual elements. The visual elements may be, for example, pixels, or 8x8 DCT (Discrete Cosine Transform) blocks as used in JPEG images in a motion-JPEG stream. For the detection of real-world objects visible in a video, a foreground separation method is applied to individual frames of the video, resulting in detections. Other methods Is of detecting real-world objects visible in a video are also known and may equally be practised. Such methods include, for example, image segmentation. In one arrangement, foreground separation is perfonned by frame differencing. Frame differencing subtracts a current frame from a previous frame. In another arrangement, foreground separation is done by background modelling. That is, a scene zo model is created by aggregating the visual characteristics of pixels or blocks in the scene over multiple frames spanning a time period. Visual characteristics that have contributed consistently to the model are considered to form the background. Any area where the background model is different from the current frame is then considered to be foreground. Video cameras are often utilised for surveillance purposes, The abandonment of 25 objects and removal of objects are of particular interest in such applications. A common example of an abandoned object in a video surveillance setting is a dropped bag or suspicious package that is placed in a scene that is being observed over a sequence of video frames. Fig. 1 is a schematic block diagram of an electronic system 101 on which one or 30 more described arrangements for detecting abandoned object and object removal events can be practised. Sensors 100 are used to obtain the images of the image sequence. The sensors may represent a stand alone sensor device (i.e., detector or a security camera) or be Seoc~.)OC 880433 speci -12 part of an imaging device, such as camera, mobile phone, etc. The remaining electronic elements 110 to 168 may also be part of the imaging device comprising camera sensors 100, as indicated by dotted line 10. The electronic elements 110 to 168 may also be part of a computer system that is located either locally or remotely with respect to the s sensors 100. In the case indicated by dotted line 20, electronic elements form a part of personal computer 180. The transmission of the images from the camera sensors 100 to the processing electronics 120 to 168 is facilitated by an input/output interface 110, which could be a serial bus compliant with Universal Serial Bus (USB) standards and having corresponding io USE connectors. Alternatively, the image sequence may be retrieved from camera sensors 100 via Local Area Network 190 or Wide Area Network 195, The image sequence may also be downloaded from a local storage device (e.g., 170), that can include SIM card, SD card, USB memory card, etc. The images are obtained by input/output interface 110 and sent to the memory 150 or 15 another of the processing elements 120 to 168 via a system bus 130. The processor 120 is arranged to retrieve information relating to one or more video frames from sensors 100 or from memory 150. The processor 120 is also arranged to fetch, decode and execute all steps of the disclosed method. The processor 120 then records the results from the respective operations to memory 150, again using system bus 130. Apart from memory 20 150, the output could also be stored more permanently on a storage device 170, via an input/output interface 160. The same output may also be sent, via network interface 164, either to a remote server which may be part of the network 190 or 195, or to personal computer 180, using input/output interface 110. The output may also be displayed for human viewing, using AV interface 168, on a monitor 185. Alternatively, the output may 2s be processed further. One example of further processing may include using the output data, written back to memory 150, memory 170 or computer 180, as the input to a background modelling system. As was discussed above and indicated in Fig. 1, the above method may be embodied in various forms. In the particular form, indicated by rectangle 10 in Fig, 1, the method is 30 implemented in an imaging device, such as a camera, a network camera, or a mobile phone with a camera. In this case, all the processing electronics 110 to 168 will be part of the imaging device, as indicated by rectangle 10. As already mentioned in the above 1)0 0 O380933 spcci - 13 description, such an imaging device for capturing a sequence of images and processing information from one or more of the captured images will comprise: a sensor 100, memory 150, a processor 120, an input/output interface 110 and a system bus 130. The sensor 100 is arranged for capturing imaging data relating to visual elements of each image in the s sequence of images. The memory 150 is used for storing each image in the sequence of images captured by the sensor and background scene information. The processor 120 is arranged for receiving, from the sensor 100 or from the memory 150, stored background scene information. The processor 120 also computes predicted and observed edge characteristics for each visual element that defines a boundary of a region of change in a 1w video frame that is being analysed. Further, the processor 100 is arranged to determine a score for a set of visual elements that define the boundary of the region of change. Depending on the application and implementation, not every one of the visual elements that define the boundary of the region of change is processed by the processor 100. For some applications, for example, it 15 may be sufficient to determine a score for every second or third, or some other selection, of those visual elements that define the boundary of the region of change. The processor is further arranged to determine an overall score for the region of change and thus determine whether the region of change relates to an abandoned object event or an object removal event. The input/output interface 110 facilitates the transmitting of the imaging data from 20 the sensor 100 to the memory 150 and to the processor 120, while the system bus 130 transmits data between the input/output interface 110 and the processor 120. Fig. 2 shows four sequential video frames 201 to 204 in a scenario that illustrates detection of an abandoned object, The sequential video frames 201 to 204 are not necessarily consecutive video frames. In one embodiment, for example, video frames are 25 sampled at a predetermined periodic rate to minimise computational cost. Accordingly, the video frames 201 to 204 may represent every second, fifth, or tenth frame in a video sequence, for example. Alternatively, the video frames 201 to 204 may represent consecutive video frames. Further, the video frames 201 to 204 may be derived from a single video camera or multiple video cameras, 30 In a first frame 201, a person 210 enters a scene 219 carrying a bag 211. In a second, later frame 202, a person 220 lowers a bag 221 towards the floor (not shown). In a third, later frame 203, a person 230 walks away from a bag 231, which has been placed on the I X)686 x-), 580833 spcci - 14 floor (not shown). In a fourth, later frame 204, a person 240 exits the scene and a bag 241 is on the floor (not shown). The appearance of the bag 231 when it first appears as a separate object, in the frame 203, does not substantially change in its appearance as the bag 241 in the later frame 204. s A second type of stationary change to a video scene is the removal of an object that was previously considered to form part of the background. In a video surveillance setting, removal of an object that was previously considered to form part of the background could relate to an item of value that is being monitored, such as a painting on a wall. In this case, the change to the visual appearance of the scene is not the object itself, but the newly 10 observed background behind the (removed) object. Fig. 3 shows four sequential video frames 301 to 304 from an example that illustrates detection of object removal, As described above with reference to Fig. 2, the sequential video frames 301 to 304 are not necessarily consecutive video frames. In one embodiment, for example, video frames are sampled at a predetermined periodic rate to minimise is computational cost. Accordingly, the video frames 301 to 304 may represent every second, fifth, or tenth frame in a video sequence, for example. Alternatively, the video frames 301 to 304 may represent consecutive video frames. Further, the video frames 301 to 304 may be derived from a single video camera or multiple video cameras, In a first frame 301, three bags 311, 312, 313 are shown as part of a scene 319. The 20 three bags 311, 312, and 312 have been part of the scene 319 long enough to be considered to be background, as may have been determined from analysis of one or more earlier video frames. A person 310 is shown to have entered the scene 319. In a second, later frame 302, a person 320 takes hold of a bag 321, and in a third, later frame 303, a person 330 walks away from bags 332, 333, while holding bag 331. In frame 303, the appearance of 25 the background differs in a region 335 that the bag 331 previously occupied. However, the new appearance of region 345 does not vary in a fourth, later frame 304. These two types of stationary changes, relating respectively to an abandoned object event and an object removal event, have similar properties. In both cases, the visual elements that constitute the region of change (i.e., the bag 241 in Fig. 2, and the "empty" 30 region 341 in Fig. 3) are different in appearance from the background model, but otherwise do not change in appearance in subsequent frames during the period of analysis. Hence, I1q068 6 . IJ)c K83peci -15 these two types of stationary changes can both be classified as regions of change and be potential events that should trigger an alert. Fig. 4 illustrates a region of change for both abandoned object events and object removal events. Fig. 4 shows a bag 401 and an "empty" region 411. The "empty' region 5 411 corresponds to the change in background in frame 303 when bag 331 is picked up by the person 330. The bag 401 produces a first region of change 402 and the "empty" region produces a second region of change 412. The first and second regions of change 402 and 412 are difficult to distinguish automatically from one another, based only on a region 499, Differentiating Between Abandoned and Removed Object Events 1o Fig. 5 is a flow diagram of a process 501 that illustrates fimetionality associated with a data processing architecture according to an embodiment of the present disclosure, such that abandoned object events and object removal events may be differentiated from each other. The process 501 calculates a correlation score between an appearance of each of the boundary blocks of a detected region of change, and an observed edge calculated from the 15 boundary blocks. In one embodiment, the blocks used are Discrete Cosine Transform (DCT) blocks of the video images. However, blocks that are the results of other transformation methods can equally be used without departing from the spirit and scope of the present disclosure, The process 501 starts at a step 500 with an input that includes a one or more 20 boundary blocks of a detected region of change. Fig. 9 shows an example of a detected region of change 901 with boundary blocks highlighted, and an exemplary input to the process 501 is shown as input 902. One method for determining an object boundary is disclosed in United States Patent Publication No. 2008/0152236 (Vendrig et al.). Other methods for determining a visual object boundary may equally be utilised, such as pixel 25 based object detection and user definition. Returning to Fig. 5, control proceeds from step 500 to decision step 510, which determines if there are still more boundary blocks to be processed. If there are more boundary blocks to be processed, Yes, control passes to step 520, which determines a predicted edge orientation of the boundary block to be processed. This determination step 30 is performed, for example, based on the location of nearby boundary blocks (as is described in more detail with reference to Fig. 10), or through user input. Control passes from step 520 to step 530, which calculates an observed edge orientation and observed edge strength l9O6'866 IJXXC ~8833speci - 16 for the boundary block, based on equations using the DCT coefficients of the block. Control then passes to step 540, which utilises the predicted and observed edge orientations, along with the observed edge strength of the boundary block, to calculate an individual block score, or abandoned/vanished (A/V) score, to determine how likely it is 5 that this particular boundary block contains a physical object boundary. Step 550 adds the score for the boundary block to a global score for the whole detected region of change. Returning to step 510, if there is not another boundary block left to be processed, No, the process 501 is directed from the step 510 by a NO arrow to step 560, which normalises the global A/V score with respect to the number of boundary blocks processed. Control 10 passes from step 560 to step 570, which determines, based on the normalised global A/V score, whether the detected region of change is an abandoned object or a removed object. This determination can be, for example, a binary abandoned/removed answer based on a threshold, or a probabilistic answer concerning the likelihood of the detected region of change to be either abandoned or removed. Finally, the process 501 is directed to an END is step 599, which completes the processing with respect to the input. Boundary Traversal for Regions of Change The method described above with reference to Fig. 5 requires the boundary blocks of a detected region of change as an input. Boundary blocks can be found in a number of ways known in the art, or alternatively the boundary blocks can be specified through user 20 input. Fig. 6 is a flow diagram of a process 601 for determining a set of boundary blocks of a detected region of change. The process 601 starts at step 600 with an input comprising blocks of a detected region of change, as described earlier with reference to the region of change 499 of Fig. 4. Control passes to step 610, which sets a search direction to right and 25 control passes to step 620, which traverses the top row of the blocks comprising the detected region of change to identify a "left-most" boundary block of the top row of blocks that is located furthest to the left. Step 630 sets the identified left-most block of the top row of bounding blocks as a starting block, the block from which traversal of the bounding blocks will begin. Flow then passes to step 640, which identifies a next boundary block. 30 Identification of the next boundary block is discussed in greater detail with reference to Fig. 7. ]QOO~6] J"8833, speci - 17 Control passes from step 640 to decision step 650, which on the first time the step is processed sets a starting search direction to be the direction to a next boundary block immediately following the starting block. The variables starting block and starting search direction are used as termination conditions. Control passes to decision step 660, which s determines whether to terminate traversal of the bounding blocks, based on whether a current boundary block is the starting boundary block and a current search direction is the same as the starting search direction. If the current boundary block is not the starting boundary block and/or the current search direction is not the same as the starting search direction, No, flow is directed to step 670, which adds the current boundary block to a list to of boundary blocks to be processed, and step 680 sets the next boundary block found in step 640 to the current boundary block. Flow then passes to step 640 to continue the traversal of the boundary blocks. Returning to decision step 660, if the current boundary block is the starting boundary block and the current search direction is the same as the starting search direction, and thus traversal of the boundary blocks has arrived back at the 15 start then the process 601 passes to an END step 699, which completes the processing with respect to the input, and a list of the boundary blocks of a region of change is outputted, as illustrated by input 902 of Fig. 9. It will be appreciated by a person skilled in the art that, while the process 601 of Fig. 6 utilises a left-most block on a top row of the bounding blocks as a starting block and 20 an initial search direction to the right, other starting blocks and initial search directions may equally be practised without departing from the spirit and scope of the present disclosure. For example, a starting block of the right-most block on a top or bottom row of boundary blocks and an initial search direction to the left may equally be utilised. In one embodiment, processing to differentiate between abandoned object and object 25 removal events is performed while the boundary traversal is being conducted. Fig. 7 is a schematic flow diagram of a method 701 of determining the next boundary block of a region of change according to step 640 of Fig. 6. The process 701 starts at a Start step 700 with inputs comprising of the blocks of a detected region of change, as well as the variables current boundary block and search direction. 30 The process for traversing the boundary blocks searches for a next boundary block by looking at boundary blocks that are adjacent to a current boundary block. In this embodiment, the boundary blocks adjacent to the current boundary block are examined in I1*0t. IX)(: 3 80&l3sp'i -Is the following order: (i) 90 degrees anti-clockwise to the search direction 710; (ii) along the search direction 720; (iii) 90 degrees clockwise to the search direction 730; and (iv) opposite to the search direction 740. If any of these searches discovers a boundary block (described in detail with reference to Fig. 8), flow is directed via a YES arrow to step 750 s where the newly discovered boundary block is set to be the next boundary block, and the search direction used to traverse to this block is set as the new search direction in step 760. Fig. 7 begins at a Start step 700 and proceeds to a first decision step 710, which determines whether there is a boundary block adjacent to the current boundary block in a direction 90 degrees anti-clockwise to the search direction. If Yes, control passes to step 10 750. However, if there is not a boundary block adjacent to the current boundary block in a direction 90 degrees anti-clockwise to the search direction, No, control passes from step 710 to a second decision step 720. Step 720 determines whether there is a boundary block adjacent to the current boundary block along the search direction, If Yes, control passes to step 750. However, if there is not a boundary block adjacent to the current boundary block is along the search direction, No, control passes from step 720 to a third decision step 730. Step 730 determines whether there is a boundary block in a direction 90 degrees clockwise to the search direction. If Yes, control passes to step 750. However, if there is not a boundary block adjacent to the current boundary block in a direction 90 degrees clockwise to the search direction, No, control passes from step 730 to step 740. Step 740 determines 20 whether there is a boundary block adjacent to the current boundary block in the opposite direction to the search direction. If Yes, control passes to step 750. However, if there is not a boundary block adjacent to the current boundary block in the direction opposite the boundary block, No, control passes from step 740 to an End step 799 and the process 701 terminates, 25 Step 750 sets the newly discovered boundary block to be the next boundary block, and step 760 sets the search direction used to traverse to this block as the new search direction, The process 701 then proceeds to the END step 799, which completes the processing with respect to the input. Fig. 8 is a flow diagram that illustrates an embodiment of a process 801 to determine 30 whether a block under consideration is a boundary block of a detected region of change. The process 801 starts at a Start step 800 with an input comprised of a potential boundary block, as well as 8 surrounding blocks that surround the potential boundary block. All nine 1)0620 I.o C 1033s - 19 blocks are encoded with information regarding whether the respective blocks are part of the detected region of change. In a following step 810, the process checks whether the potential boundary block lies on the boundary of the image (i.e., on an edge of the image frame), by asking whether the s block is not on a boundary of the image. If true, flow is directed by a NO arrow to step 860, which classifies the block as a non-boundary block. Returning to step 810, if the block does not lie on the image boundary, flow is directed via a YES arrow to step 820. In step 820, the process checks whether the block is part of the region of change and thus does not form part of the background. If the block is not part of the region of change, 10 No, flow is directed by a NO arrow to step 860, which classifies the block as a non-boundary block. Returning to step 820, if the block is part of the region of change, Yes, flow is directed from step 820 via a YES arrow to step 830. In step 830, the process checks each of the 8 surrounding blocks in tum and step 840 determines whether any of the adjacent blocks is a background block (i.e., a block that is is not part of the region of change). If a background block is found, Yes, step 840 directs the flow to step 850 via a YES arrow. Step 850 designates the current block as a boundary block. The process 801 then terminates at an END step 899. Returning to step 840, if none of the surrounding adjacent blocks is detected as forming part of the background, No, then processing is directed to step 860 via a NO 20 arrow. Step 860 designates the potential boundary block under consideration as a non-boundary block, The process 801 is directed from step 860 to an END step 899, which completes the processing with respect to the input. Determination of Predicted Edge Characteristics After process 501 receives a boundary block at step 520, a predicted edge 2s characteristic for the boundary block needs to be determined. Predicted edge characteristics that may be used in one or more embodiments of the present disclosure are, for example, the orientation of the edge, or the strength of the edge. Fig. 10 is a schematic representation of an exploded view 1001 of an example of a set of predicted edge orientations for boundary blocks of a region of change. The exploded 30 view 1001 comprises a set of boundary blocks 1010 of a detected region of change. To determine the edge orientation of a particular boundary block, adjacent boundary blocks to either side of the particular boundary block are checked and the geometrical configurations J 40O6966. I JXC MsV.0S3 5pWI - 20 of the three boundary blocks, being the particular boundary block and the adjacent boundary block on each side are considered. For example, if a boundary block 1022 that is presently under consideration is part of a vertical arrangement 1020 in which the adjacent blocks are above and below the s boundary block under consideration 1022, a predicted edge orientation associated with the boundary block under consideration 1022 is classified as a vertical line 1021. As a second example, if a boundary block under consideration 1052 is part of a corner arrangement 1050 in which the adjacent blocks are left and below the block, then a predicted edge orientation associated with the boundary block under consideration 1052 is 10 classified as a sloped line 1051 with a 45 degree angle. Further examples of this embodiment are configurations 1030 and 1040, which produce predicted edge orientations 1031 and 1041 respectively. It is possible for two adjacent blocks of a boundary block under consideration to be the same. Such a scenario may arise, for example, at a corner of a region of change that is comprises a single file of blocks either in a horizontal or vertical configuration. In such a situation, a predicted edge orientation associated with the boundary block under consideration will also be horizontal or vertical, respectively. This embodiment for determining a predicted edge orientation of a boundary block can be extended to examining nearby boundary blocks beyond just the two adjacent blocks. 20 Determination of Observed Edge Characteristics After a predicted edge characteristic is determined at step 520 of process 501 of Fig. 5, flow passes to step 530, which determines an observed edge characteristic of the boundary block. One method of determining observed edge characteristics, such as edge orientation 25 and edge strength, is through calculations using the DCT coefficients of a boundary block under consideration. Edges in a block can be considered to be those pixels that have the greatest change in intensity in the Y channel. One embodiment uses only 4 DCT coefficients to calculate the intensity in the Y channel through the equations: 30 f(x)= A 01 cos (X+0.5)J +AC2 cos (x+0.5) ... 1906866 L.J)Oc 880833_speci - 21 f(y)= ACco '(y+0.5) + ACco (y + 0.5) ... (2) (0 s x s 7), (0 y 7) s in the horizontal and vertical directions respectively, where x andy are the pixel numbers from the origin. AC represents the DCT coefficients, as shown in Fig. 11 with respect to a DCT coefficient table 1150. Arranging the DCT coefficient table 1150 in a zigzag pattern, as shown, then the coefficients used are ACO, 1101, ACO2 1102, ACIO 1110 and AC 2 0 1120. The edge strength in the horizontal and vertical directions for the 8 distances can then 1o be calculated by taking the derivative of Equation (1) and Equation (2). f'(x)=-ACo i (x+0.5 C sin (x+. .(3) ) oz(-g-l(y .5)) ... (3 f'(y) =-A CO si y0.5) -AC20 ( in S, ( y+0. 5)) ... (4) 15 (0 5 x 7 ),(0 s y 5 7) Of the 8 values obtained, the maximum value in each direction is used both to calculate the observed edge strength as well as the observed edge orientation in this block. 20 The maximum edge strength is given by observededgestrength= f'(x)L 2 +f'() 2 ... (5) and the observed edge orientation (in degrees) by 25 observed edgeorientation = tan-' fr )m (6) .IY)C 8H083 -22 wheref(y) andf(x) are the maximum values in each direction as calculated using Equation (3) and Equation (4). Correlation of Predicted and Observed Edge Characteristics After the observed edge characteristics are calculated at step 530 of process 501 of s Fig. 5, flow passes to step 540, which calculates an abandoned/removed (A/V) score for the boundary block under consideration. There are many different schemes that can be used to calculate this score, and one embodiment is discussed below. If given a single edge characteristic of edge orientation, one possible way of evaluating the A/V score is to look at the correspondence between the predicted and io observed edge orientations as defined in Fig. 10 and Equation (6): A/Vscore = 100 -- predictededge orientation - observed_edge orientation .(7) is where the predicted edgeorientation and observed-edgeorientation are given in degrees. This produces a score that is between 10 and 100, with the larger numbers representing a higher likelihood of the predicted and observed orientations corresponding with each other. Conversely, lower numbers represent a lower likelihood of the predicted and observed orientations corresponding with each other. 20 One further refmement to this scoring scheme is to factor in the edge characteristic: edge strength, An edge strength threshold can be set such that any boundary block with observed edge strength, as calculated using Equation (5), below the edge strength threshold receives an A/V score of zero. This prevents very weak edges that may be random noise from affecting the final prediction of the algorithm. The threshold can be a user defined 25 value, or alternatively can be set as the average evaluated edge strength of the entire scene at initialisation. Fig. 12 is a schematic representation 1201 that shows observed edges of the object 241 from Fig, 2. Fig. 12 comprises a set of boundary blocks 1210 of a detected region of change. For boundary block sets 1220, 1230, 1240 and 1250, which correspond to similar 30 arrangements 1020 to 1050 in Fig. 10, the observed edge orientations are shown as 1221, 1231, 1241 and 1251, respectively. I I tSgt)C 8833 poi~ 23 After the A/V score for the current boundary block is determined in step 540 of process 501 of Fig. 5, flow is directed to step 550 where the score is added to a global total for the detected region of change. When all the boundary blocks for a particular detected region of change have been 5 processed, the flow of process 501 is directed to step 560 where the global A/V score for the region is nornalised by the number of boundary blocks that are used during the processing. The nonnalisation process produces a score between 10 and 100 for the region that can be used to determine whether the region of change is an abandoned object or a removed object. 10 Other methods can also be used for calculating a global A/V score. For example, a second method performs the following steps: A) Initialise an edge strength running total to 0. B) Initialise an edge angle-strength running total to 0. C) Initialise a maximum edge strength to 0. is D) For each boundary edge point: 1. Calculate the observed edge strength, as per Equation (5). 2. Add the observed edge strength to an edge strength running total. 3, Calculate the angle-difference, the absolute value of the predicted edge orientation minus the observed edge orientation. 20 4. Add the product of the angle-difference and the observed edge strength to an edge angle-strength running total. 5. If the edge strength is greater than the maximum edge strength, set the maximum edge strength to be the edge strength. E) Calculate a normalised edge strength running total as the edge strength running 25 total divided by the maximum edge strength. F) Calculate the global A/V score as the edge angle-strength running total divided by the normalised edge strength running total. This algorithm will produce a number between 0 and 90 (provided the angle differences are calculated in degrees), with numbers towards 90 indicating a greater 30 likelihood of the blob representing a vanished object, and numbers towards 0 indicating a greater likelihood of the blob representing an abandoned object. In this way, boundary 1906. L DOC $S3pu -24 edge points with lower observed edge strengths contribute less to the global AN score. Note that because this method does not use a threshold on edge strength, the maximum edge strength should also be considered as an output of this algorithm, so subsequent decisions on whether the object is abandoned or vanished can override the decision to s "vanished", if the maximum edge strength is considered to be too weak to be meaningful. This may occur in situations where there is a very faint outline left by a vanished object (for example, a discoloration behind a removed painting). The final step 570 of process 501 predicts whether a detected event is an abandoned object event or an object removal event. One embodiment utilises a user-defined threshold 10 for the global AN score, A global A/V score above the threshold indicates an abandoned object, and a global AN score below the threshold indicates a removed object. In another embodiment, the threshold is based on an average evaluated edge strength of the entire scene at initialisation, as may occur when a system embodying the method is turned on or reset. 15 In another embodiment, a probabilistic approach is used, in which a likelihood of a detected region of change being an abandoned object or a removed object is calculated based on a value of the global score for the detected region of change and a threshold. Boundary Edge Refinement The abandoned object region 499 resulting from the abandoned object event 402 in 20 Fig. 4 is a candidate for edge refinement. The edges are not limited to the outer boundary in this case. Fig. 15 is a schematic representation 1500 illustrating a detected region of change 1520 within a video frame 1510. Boundary blocks 1530 of the region 1520 of the frame 1510 also have an internal component 1540. Fig. 17 is a flow chart showing one embodiment of a process 1701 used to refine the 2s boundary detail using the transmitted subset of coefficients for each boundary block. The process begins at a Start step 1700 and proceeds to step 1710, which performs a check for blocks remaining to process. As long as there are remaining blocks to process, the remainder of the process is performed in a loop on boundary blocks of the regions of the image. If there are no blocks remaining to be processed, No, control passes to an End step 30 1790 and the process 1701 terminates. If there is another block remaining to be processed, control passes from step 1710 to step 1720 19'069b6 I SoX 8R083i peci - 5 In step 1720, the transmitted coefficients of the current boundary block are examined to obtain the observed strength, position, and orientation of the edge within the block. This process is described in more detail with reference to Fig. 18. Once this information has been obtained, a check is made in step 1730 on the observed edge strength. If the observed s edge strength is below a threshold, then no edge determination can be made for that block, so control passes to step 1760, which marks the entire block with a flag to indicate the lack of edge information for that block. If the edge is to be later rendered, it can be done so using information from pixels surrounding this block, in step 1770. Control returns from step 1770 to step 1710 to determine whether further boundary blocks are to be processed. 10 Retumning to step 1730, if the observed edge strength is deemed 1730 to be above a threshold, and thus sufficient, and the position and orientation are known from step 1720, then the polarity of the edge needs to be determined at step 1740. In one embodiment of the invention, the edge is defined in step 1740 by taking account of those pixels adjacent to the current block which fall on either side of the edge. A majority count is sufficient is define which side of the found edge is foreground and which side is background. The edge through that block is now defined in step 1750 with a determined position and a determined orientation, and that information may be encoded, or the block may be rendered. Control passes from step 1750 and returns to step 1710. The process 1701 ends 1790 when the check 1710 for unprocessed blocks reports 20 that there are no boundary blocks remaining to be processed. Fig. 18 is a flow diagram 1801 that illustrates a process used to calculate the edge strength, location, and orientation, as required in step 1720 of the embodiment as described in Fig. 17. The process 1801 begins at a Start step 1800 and proceed to step 1810, which finds a location and magnitude of the maximum first derivative of the vertical components 25 of the block, In one embodiment of the present disclosure, the magnitude is found by taking the maximum of the eight values obtained from Equation (3). The location is determined by the position at which the magnitude is calculated. The next step of the process, step 1820, perform a similar calculation in the horizontal direction. With the horizontal and vertical magnitudes known, control passes to step 1830 to 30 determine the observed edge strength as the square root of the sum of the squares of the horizontal and vertical edge magnitudes. The observed edge location is known at step 1840, as the edge must pass through the location of the maximum derivatives. The -26 observed edge orientation is calculated in step 1850 from the known vertical and horizontal edge magnitudes. It is defined in Equation (6). Control passes to step 1860 and the method 1801 terminates. In another embodiment, the boundary refinement process is performed in the spatial s domain as shown in Fig. 15. Those blocks 1530 of the frame 1510 which were identified as edge blocks prior to transmission are decoded to reconstruct an approximation to the original pixels. This is done for all transmitted boundary blocks, e.g., including area 1540. A thresholding process is then applied to extend the boundaries of the detected region in from the known boundary 1530, and out from the region centre 1520. In one embodiment, 20 the thresholding process is applied by making a binary cut at an intermediate luminance level, for example 128, and a choice is made whether to invert the final result depending on whether a nearest block of object interior 1520 is darker or lighter than the nearest block of object exterior 1510. Camera Implementation is One implementation of a system in accordance with the present disclosure is embodied in a camera. Fig. 13 shows a schematic block diagram of a camera 1300 upon which embodiments for processing a video frame may be practised. In one implementation, the steps of the methods of Figs 2 to 12 and 16 to 18 are implemented as software executable within the camera 1300. The steps of the methods of Figs 2 to 12 and 20 16 to 18 may be performed on a single processor or on multiple processors, either within the camera or external to the camera 1300. The camera 1300 is a pan-tilt-zoom camera (PTZ) fonned by a camera module 1301, a pan and tilt module 1303, and a lens system 1314. The camera module 1301 typically includes at least one processor unit 1305, and a memory unit 1306, a photo-sensitive sensor 25 array 1315, an input/output (I/O) interfaces 1307 that couples to the sensor array 1315, an input/output (I/O) interfaces 1308 that couples to a communications network 1320, and an interface 1313 for the pan and tilt module 1303 and the lens system 1314. The components 1305 to 1313 of the camera module 1301 typically communicate via an interconnected bus 1304 and in a manner which results in a conventional mode of operation known to those in so the relevant art. The pan and tilt module 1303 includes servo motors which, in response to signals from the camera module 1301, move the camera module 1301 about the vertical and I~)Ot$i6 I80834 Spcc -27 horizontal axes. The lens system 1314 also includes a servo motor which, in response to signals from the camera module 1301, is adapted to change the focal length of the lens system 1314. Computer Implementation s Figs. 14A and 14B collectively form a schematic block diagram of a general purpose computer system 1400, upon which the various arrangements described can be practised. In one implementation, the general purpose computer system 1400 is coupled to a camera to form a video camera on which the various arrangements described are practised. In another implementation, one instance of the general purpose computer system 1400 is an I> external computing device that receives image processing information, such as a subset of dominant coefficients, from a camera and performs image processing based on the coefficients included in the image processing information. As seen in Fig. 14A, the computer system 1400 is formed by a computer module 1401, input devices such as a keyboard 1402, a mouse pointer device 1403, a is scanner 1426, a camera 1427, and a microphone 1480, and output devices including a printer 1415, a display device 1414 and loudspeakers 1417. An external Modulator Demodulator (Modem) transceiver device 1416 may be used by the computer module 1401 for communicating to and from a communications network 1420 via a connection 1421. The network 1420 may be a wide-area network (WAN), such as the Internet or a private 2o WAN. Where the connection 1421 is a telephone line, the modem 1416 may be a traditional "dial-up" modem. Alternatively, where the connection 1421 is a high capacity (e.g., cable) connection, the modem 1416 may be a broadband modem, A wireless modem may also be used for wireless connection to the network 1420. The computer module 1401 typically includes at least one processor unit 1405, and a 25 memory unit 1406 for example formed from semiconductor random access memory (RAM) and semiconductor read only memory (ROM). The module 1401 also includes an number of input/output (I1O) interfaces including an audio-video interface 1407 that couples to the video display 1414, loudspeakers 1417 and microphone 1480, an 1/O interface 1413 for the keyboard 1402, mouse 1403, scanner 1426, camera 1427 and 30 optionally ajoystick (not illustrated), and an interface 1408 for the external modem 1416 and printer 1415. In some implementations, the modem 1416 may be incorporated within the computer module 1401, for example within the interface 1408. The computer module I1Q06$&6 i DoC 880833_spci -28 1401 also has a local network interface 1411 which, via a connection 1423, permits coupling of the computer system 1400 to a local computer network 1422, known as a Local Area Network (LAN). As also illustrated, the local network 1422 may also couple to the wide network 1420 via a connection 1424, which would typically include a so-called s "firewall" device or device of similar functionality The interface 1411 may be fonned by an Ethernets' circuit card, a BluetoothTm wireless arrangement or an IEEE 802.11 wireless arrangement. The interfaces 1408 and 1413 may afford either or both of serial and parallel connectivity, the former typically being implemented according to the Universal Serial Bus to (USB) standards and having corresponding USB connectors (not illustrated). Storage devices 1409 are provided and typically include a hard disk drive (HDD) 1410. Other storage devices such as a floppy disk drive and a magnetic tape drive (not illustrated) may also be used. An optical disk drive 1412 is typically provided to act as a non-volatile source of data, Portable memory devices, such optical disks (e.g., CD-ROM, DVD), USB is RAM, and floppy disks for example may then be used as appropriate sources of data to the system 1400. The components 1405 to 1413 of the computer module 1401 typically communicate via an interconnected bus 1404 and in a manner which results in a conventional mode of operation of the computer system 1400 known to those in the relevant art. Examples of 20 computers on which the described arrangements can be practised include IBM-PCs and compatibles, Sun Sparostations, Apple Maci" or alike computer systems evolved therefrom. The method of processing a video frame may be implemented using the computer system 1400 wherein the processes of Figs 2 to 12 and 16 to 18 may be implemented as 25 one or more software application programs 1433 executable within the computer system 1400. In particular, the steps of the method of processing a video frame are effected by instructions 1431 in the software 1433 that are carried out within the computer system 1400. The software instructions 1431 may be formed as one or more code modules, each for performing one or more particular tasks. The software may also be divided into two 30 separate parts, in which a first part and the corresponding code modules performs the detecting, determining and identifying methods and a second part and the corresponding code modules manage a user interface between the first part and the user. )06H66 I .Do c: J -29 The software 1433 is generally loaded into the computer system 1400 from a computer readable medium, and is then typically stored in the HDD 1410, as illustrated in Fig. 14A, or the memory 1406, after which the software 1433 can be executed by the computer system 1400. In some instances, the application programs 1433 may be supplied s to the user encoded on one or more CD-ROM 1425 and read via the corresponding drive 1412 prior to storage in the memory 1410 or 1406. Alternatively the software 1433 may be read by the computer system 1400 from the networks 1420 or 1422 or loaded into the computer system 1400 from other computer readable media. Computer readable storage media refers to any storage medium that participates in providing instructions 10 and/or data to the computer system 1400 for execution and/or processing. Examples of such storage media include floppy disks, magnetic tape, CD-ROM, a hard disk drive, a ROM or integrated circuit, USB memory, a magneto-optical disk, or a computer readable card such as a PCMCIA card and the like, whether or not such devices are internal or external of the computer module 1401. Examples of computer readable transmission is media that may also participate in the provision of software, application programs, instructions and/or data to the computer module 1401 include radio or infra-red transmission channels as well as a network connection to another computer or networked device, and the Internet or Jntranets including e-mail transmissions and infonnation recorded on Websites and the like, 20 The second part of the application programs 1433 and the corresponding code modules mentioned above may be executed to implement one or more graphical user interfaces (GUs) to be rendered or otherwise represented upon the display 1414. Through manipulation of typically the keyboard 1402 and the mouse 1403, a user of the computer system 1400 and the application may manipulate the interface in a functionally adaptable 25 manner to provide controlling commands and/or input to the applications associated with the GUI(s). Other forms of functionally adaptable user interfaces may also be implemented, such as an audio interface utilizing speech prompts output via the loudspeakers 1417 and user voice commands input via the microphone 1480. Fig. 14B is a detailed schematic block diagram of the processor 1405 and a 30 "memory" 1434. The memory 1434 represents a logical aggregation of all the memory devices (including the HDD 1410 and semiconductor memory 1406) that can be accessed by the computer module 1401 in Fig. 14A.

- 30 When the computer module 1401 is initially powered up, a power-on self-test (POST) program 1450 executes. The POST program 1450 is typically stored in a ROM 1449 of the semiconductor memory 1406. A program permanently stored in a hardware device such as the ROM 1449 is sometimes referred to as finuware. The POST s program 1450 examines hardware within the computer module 1401 to ensure proper functioning, and typically checks the processor 1405, the memory (1409, 1406), and a basic input-output systems software (BIOS) module 1451, also typically stored in the ROM 1449, for correct operation. Once the POST program 1450 has run successfully, the BIOS 1451 activates the hard disk drive 1410. Activation of the hard disk drive 1410 10 causes a bootstrap loader program 1452 that is resident on the hard disk drive 1410 to execute via the processor 1405. This loads an operating system 1453 into the RAM memory 1406 upon which the operating system 1453 commences operation. The operating system 1453 is a system level application, executable by the processor 1405, to fulfil various high level functions, including processor management, memory management, is device management storage management, software application interface, and generic user interface. The operating system 1453 manages the memory (1409, 1406) in order to ensure that each process or application running on the computer module 1401 has sufficient memory in which to execute without colliding with memory allocated to another process. 2 Furthermore, the different types of memory available in the system 1400 must be used properly so that each process can run effectively. Accordingly, the aggregated memory 1434 is not intended to illustrate how particular segments of memory are allocated (unless otherwise stated), but rather to provide a general view of the memory accessible by the computer system 1400 and how such is used. 25 The processor 1405 includes a number of functional modules including a control unit 1439, an arithmetic logic unit (ALU) 1440, and a local or intemal memory 1448, sometimes called a cache memory. The cache memory 1448 typically includes a number of storage registers 1444 - 1446 in a register section. One or more internal buses 1441 functionally interconnect these functional modules, The processor 1405 typically also has 30 one or more interfaces 1442 for communicating with external devices via the system bus 1404, using a connection 1418. 1 "o6goob -IMB032pv -31 The application program 1433 includes a sequence of instructions 1431 that may include conditional branch and loop instructions. The program 1433 may also include data 1432 which is used in execution of the program 1433. The instructions 1431 and the data 1432 are stored in memory locations 1428-1430 and 1435-1437 respectively, s Depending upon the relative size of the instructions 1431 and the memory locations 1428 1430, a particular instruction may be stored in a single memory location as depicted by the instruction shown in the memory location 1430. Alternately, an instruction may be segmented into a number of parts each of which is stored in a separate memory location, as depicted by the instruction segments shown in the memory locations 1428-1429, 10 In general, the processor 1405 is given a set of instructions which are executed therein. The processor 1405 then waits for a subsequent input, to which it reacts to by executing another set of instructions. Each input may be provided from one or more of a number of sources, including data generated by one or more of the input devices 1402, 1403, data received from an external source across one of the is networks 1420, 1422, data retrieved from one of the storage devices 1406, 1409 or data retrieved from a storage medium 1425 inserted into the corresponding reader 1412. The execution of a set of the instructions may in some cases result in output of data. Execution may also involve storing data or variables to the memory 1434. The disclosed classification arrangements use input variables 1454, that are stored in 20 the memory 1434 in corresponding memory locations 1455-1458, The classification arrangements produce output variables 1461, that are stored in the memory 1434 in corresponding memory locations 1462-1465. Intermediate variables may be stored in memory locations 1459, 1460, 1466 and 1467. The register section 1444-1446, the arithmetic logic unit (ALU) 1440, and the control 25 unit 1439 of the processor 1405 work together to perform sequences of micro-operations needed to perform "fetch, decode, and execute" cycles for every instruction in the instruction set making up the program 1433. Eanh fetch, decode, and execute cycle comprises: (a) a fetch operation, which fetches or reads an instruction 1431 from a memory 30 location 1428; (b) a decode operation in which the control unit 1439 determines which instruction has been fetched; and 'oo66.txo 380833 ,ppeci -32 (c) an execute operation in which the control unit 1439 and/or the ALU 1440 execute the instruction. Thereafter, a further fetch, decode, and execute cycle for the next instruction may be executed. Similarly, a store cycle may be performed by which the control unit 1439 stores s or writes a value to a memory location 1432. Each step or sub-process in the processes of Figs 2 to 12 and 16 to 18 is associated with one or more segments of the program 1433, and is performed by the register section 1444-1447, the ALU 1440, and the control unit 1439 in the processor 1405 working together to perform the fetch, decode, and execute cycles for every instruction in the 1o instruction set for the noted segments of the program 1433. The method of processing a video frame may alternatively be implemented in dedicated hardware such as one or more integrated circuits performing the functions or sub functions of detecting objects, determining boundary visual elements, and determining a subset of dominant coefficients for a video image. Such dedicated hardware may include is graphic processors, digital signal processors, or one or more microprocessors and associated memories. INDUSTRIAL APPLICABILITY The arrangements described are applicable to the computer and data processing industries and particularly for the image processing and surveillance industries. 20 The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive. In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including", and not "consisting only 25 of'. Variations of the word "comprising", such as "comprise" and "comprises" have correspondingly varied meanings. I 066 .)C S033s

Claims (18)

1. A computer-implementable method for processing a video frame defined by a plurality of visual elements, each of said visual elements being associated with a plurality, n, of coefficients, said method comprising the steps of s detecting at least one object in said video frame, each said detected object being associated with a region of at least one visual element; for each said region: determining a set of boundary visual elements; and identifying, for each boundary visual element in said set of boundary visual to elements for said region, a subset of dominant coefficients selected from said plurality of coefficients associated with said boundary visual element, wherein said subset includes a number, m, of dominant coefficients, wherein 0 <m < n; and transmitting to an external processing device said subset of dominant coefficients, is

2. The method according to claim 1, wherein said visual elements are Discrete Cosine Transform (DCT) blocks and said coefficients are DCT coefficients.

3. The method according to claim 1, comprising the further step of: processing, by said external processing device, said subset of dominant coefficients 20 to determine a boundary of each said region,

4. The method according to claim 3, wherein said determined boundary has a level of detail greater than a level of detail provided by determining said set of boundary visual elements. 25

5. The method according to claim 1, comprising the further step of: processing, by said external processing device, said subset of dominant coefficients to aid in the classification of each said region as one of an abandoned object event and an object removal event. 30 190A46 LJXX 88)833 ,pcc -34

6. The method according to any one of claims I to 5, wherein said subset of dominant coefficients is transmitted to said external processing device in conjunction with a reduced representation of said video frame. s

7. The method according to any one of claims I to 5, wherein said subset of dominant coefficients is transmitted to said external processing device in conjunction with a full resolution representation of said video frame, wherein said full resolution representation of said video frame includes coefficients at a quality level that is lower than a quality level of said dominant coefficients. 10

8. The method according to any one of claims I to 7, wherein said subset of dominant coefficients is transmitted in conjunction with said set of boundary visual elements for each detected object. is

9. The method according to any one of claims I to 8, wherein said external processing device is selected from the group ofprocessing devices consisting of: a computer server; a Personal Computer (PC), a custom hardware item; a general purpose computer system; and a camera. 20

10. A computer readable storage medium having recorded thereon a computer program for processing a video frame defined by a plurality of visual elements, each of said visual elements being associated with a plurality, n, of coefficients, said computer program product comprising: code for detecting at least one object in said video frame, each said detected object 25 being associated with a region of at least one visual element; code for processing each said region to: determine a set of boundary visual elements; and identify, for each boundary visual element in said set of boundary visual elements for said region, a subset of dominant coefficients selected from said 30 plurality of coefficients associated with said boundary visual element, wherein said subset includes a number, m, of dominant coefficients, wherein 0 < m < n; and J946 M IX) 880833 speci -35 code for identifying said dominant coefficients as a set of image processing information associated with said video frame.

11. A method for obtaining a refined boundary of a region of an image captured by an s image capture device, said image being represented by a plurality of blocks, each of said blocks being associated with a plurality, n, of coefficients, said method comprising the steps of: (a) at a first processing module on said image capture device: detecting at least one object in said video frame, each said detected object being associated with a region of at least one block; determining a set of boundary blocks for each of said regions associated with said detected objects; and identifying a subset of dominant coefficients for each of said boundary blocks, wherein said subset includes a number, m, of dominant coefficients, is wherein 0 <m <n; (b) transmitting said subset of dominant coefficients; and (c) at a second processing module on an external processing device. receiving said transmitted subset of dominant coefficients; and Processing said subset of dominant coefficients to obtain a refined 20 representation of the boundary of each of said regions associated with said detected objects.

12. The method according to claim 11, wherein said subset of dominant coefficients is transmitted in conjunction with a reduced representation of said image. 25

13. A method for classifying a region of an image captured by an image capture device as one of an abandoned object event and an object removal event wherein said image is represented by a plurality of blocks, each of said blocks being associated with a plurality, n, of coefficients, said method comprising the steps of: 30 (a) at a first processing module on said image capture device: detecting at least one object in said video frame, each said detected object being associated with a region of at least one block; -36 determining a set of boundary blocks for each of said regions associated with said detected objects; and identifying a subset of dominant coefficient for each of said boundary blocks, wherein said subset includes a number, m, of dominant coefficients, 5 wherein 0 <m <n; (b) transmitting said subset of dominant coefficients; and (c) at a second processing module on an external processing device: receiving said subset of transmitted dominant coefficients; and processing said subset of dominant coefficients to classify each of said to regions associated with said detected objects as a one of an abandoned object event and an object removal event.

14. The method according to claim 13, wherein said subset of dominant coefficients is transmitted in conjunction with a reduced representation of said image. 15

15. A computer-implementable method for processing a video frame defined by a plurality of visual elements, each of said visual elements being associated with at least one coefficient, said method being substantially as described herein with reference to the accompanying drawings. 20

16. A computer readable storage medium having recorded thereon a computer program for processing a video frame defined by a plurality of visual elements, each of said visual elements being associated with at least one coefficient, said computer program product being substantially as described herein with reference to the accompanying drawings. 25

17. A method for obtaining a refined boundary of a region of an image captured by an image capture device, said image being represented by a plurality of blocks, each of said blocks being associated with a plurality of coefficients, said method being substantially as described herein with reference to the accompanying drawings. 30

18. A method for classifying a region of an image captured by an image capture device as one of an abandoned object event and an object removal event, wherein said image is I')W833 11)O(' -37 represented by a plurality of blocks, each of said blocks being associated with a plurality of coefficients, said method being substantially as described herein with reference to the accompanying drawings. DATED this Thirtieth Day of December, 2008 Canon Kabushiki Kaisha Patent Attorneys for the Applicant SPRUSON & FERGUSON I moc 880S33 specd