AU2005200021B2 - Pre-event buffer management for operator triggered video recording - Google Patents

Description

-1- PRE-EVENT BUFFER MANAGEMENT FOR OPERATOR TRIGGERED VIDEO

RECORDING

U d) SField of the Invention C The present invention relates generally to the field of network cameras and, in particular, to a method, apparatus and system for storing video image data captured by one or more video cameras. The present invention also relates to a computer program Sproduct including a computer readable medium having recorded thereon a computer Sprogram for storing video image data captured by one or more video cameras.

Background The popularity of the Internet and the availability of general-purpose network cameras have resulted in networked surveillance and security systems. Such systems operate over an existing local area network (LAN) or wide area network (WAN) like the Internet. The network cameras may be implemented as special purpose computers, for example. These computers may be implemented as nodes in a computer network and may be remotely controlled and operated by a user, via a desktop computer, for example.

Some existing networked surveillance and security systems use computer software applications to record request and store) video image data captured by network cameras. The video image data may be stored in some form of non-volatile storage device such as a hard disk drive. Some of these software applications may enable prerecorded video image data to be viewed. This pre-recorded video image data may be streamed from a different computer on the network. The software applications may also enable live video image data to be retrieved directly from the network cameras and viewed on a general-purpose computer or the like.

Some software applications used in existing surveillance and security systems are configured to constantly retrieve video data from all camera servers connected to a network, but to only record request and store) the retrieved data to non-volatile 693277.doc -2- O storage means when a certain event is activated. Such an event is typically generated by the activation of an electronic sensor device a heat sensor, motion sensor or pressure sensor). The event may also be activated by means of an operator instructing the N application to begin recording video image data for a period of time.

Some software applications used in existing surveillance and security systems are Oconfigured to buffer a portion of retrieved video image data from all of the cameras rconnected to network for a predetermined period of time. The portion of retrieved video image data may be buffered in a separate area of memory. This separate area of memory is typically a memory buffer or a temporary section of a non-volatile storage means. The buffered data may then be written to a permanent section of a non-volatile storage means upon the occurrence of an event. Buffering data in this manner allows video image data, which occurred immediately prior to an event, to be recorded. Such data can often contain information that is useful to the operator of the surveillance or security system.

However, such systems require a large amount of bandwidth since the system is constantly buffering video image data from all of the cameras connected to the network.

One known digital video management system provides live digital video image data captured by a number of network cameras to a number of client terminals ('clients'), via a computer communications network, in real time. This known system comprises a networked video recorder that constantly buffers video image data from all enabled cameras connected to the network. In response to a command received over the network to record from one camera, the video recorder writes the buffered data received from that camera to disk and continues to record data subsequently received from that camera until instructed to stop. However, when scaling this known system to record from a large number of networked cameras, the bandwidth required to buffer video image data for each camera grows until the network is saturated. As a result, the quality of the video 693277.doc -3- 00 image data streamed from the network cameras is degraded as the frame rate for each i camera begins to decrease.

SAnother known digital video management system allows a television subscriber to begin recording a television program after the program has already started. In this system, a server is positioned between the source of transmission of the television program and the subscriber. The server stores the transmitted programs in a circular c, buffer large enough to store video image data for a predetermined period of time. The Ssubscriber is then able to connect to this server and retrieve video image data that is still present within the buffer. However, such a system still assumes a high bandwidth connection between the buffer server and the source of the video image data.

Thus, a need clearly exists for an improved method, system and apparatus for storing video image data captured by one or more video cameras, which minimises the bandwidth used to store the captured data.

Summary It is an object of the present invention to substantially overcome, or at least ameliorate, one or more disadvantages of existing arrangements.

According to one aspect of the present invention there is provided a system for storing video image data, said system comprising: one or more video cameras for capturing video image data; display means for displaying the video image data captured by at least a sub-set of said video cameras, the video image data being displayed according to one or more layouts, each of said layouts referencing at least one of the sub-sets of said video cameras; and buffer means for temporarily storing therein the video image data captured by the sub-set of video cameras referenced by the layout currently being used by the display means to display the video image data, wherein at least a portion of the video image data 693277.doc -4- 0 stored in the buffer means is further stored in a non-volatile storage means depending on a Nrecording configuration of the current sub-set of video cameras.

SAccording to another aspect of the present invention there is provided a method for storing video image data, said method comprising the steps of: capturing the video image data using one or more video cameras; displaying the video image data captured by at least a sub-set of said video cameras, Sthe video image data being displayed according to one or more layouts, each of said 0 layouts referencing at least one of the sub-sets of said video cameras; and storing temporarily in a buffer the video image data captured by the sub-set of video cameras referenced by the layout currently being used to display the video image data, wherein at least a portion of the video image data stored in the buffer is further stored in a non-volatile memory depending on a recording configuration of the current sub-set of video cameras.

According to still another aspect of the present invention there is provided an apparatus for storing video image data, said apparatus comprising: one or more video cameras for capturing video image data; display means for displaying the video image data captured by at least a sub-set of said video cameras, the video image data being displayed according to one or more layouts, each of said layouts referencing at least one of the sub-sets of said video cameras; and buffer means for temporarily storing therein the video image data captured by the sub-set of video cameras referenced by the layout currently being used by the display means to display the video image data, wherein at least a portion of the video image data stored in the buffer means is further stored in a non-volatile storage means depending on a recording configuration of the current sub-set of video cameras.

693277.doc 0 According to still another aspect of the present invention there is provided a N computer program for storing video image data, said program comprising: Scode for capturing the video image data using one or more video cameras; IND code for displaying the video image data captured by at least a sub-set of said video cameras, the video image data being displayed according to one or more layouts, each of said layouts referencing at least one of the sub-sets of said video cameras; and code for storing temporarily in a buffer the video image data captured by the sub-set Sof video cameras referenced by the layout currently being used to display the video image data, wherein at least a portion of the video image data stored in the buffer is further S 10 stored in a non-volatile memory depending on a recording configuration of the current sub-set of video cameras.

Other aspects of the invention are also disclosed.

Brief Description of the Drawings One or more embodiments of the present invention will now be described with reference to the drawings and appendices, in which: Fig. 1 is a schematic diagram of a video surveillance system upon which arrangements described may be practiced; Fig. 2 is a representation showing the software modules of a storage server software application executing on the storage server of Fig. 1; Fig. 3 is a flow diagram showing a viewer interface process; Fig. 4 is a flow diagram showing a login process; Fig. 5 is a flow diagram showing an add cameras process; Fig. 6 is a flow diagram showing a remove cameras process; Fig. 7 is a flow diagram showing a trigger recording process; Fig. 8 is a flow diagram showing a logout process; Fig. 9 shows a sample schedule; Fig. 10 is a flow diagram showing a set camera schedule process; Fig. 11 is a flow diagram showing a schedule item timer process; Fig. 12 is a flow diagram showing a recording timer expiry process; Fig. 13 is a flow diagram showing a set video image data acquisition process; 693277.doc Fig. 14 is a flow diagram showing a begin video image data acquisition process; Fig. 15 is a flow diagram showing a stop video image data acquisition process; Fig. 16 is a flow diagram showing a video data acquisition process; (Fig. 17 is a flow diagram showing an event notification process; Fig. 18 shows a sample schedule item with frame rates specified; SFig. 19 shows a user interface for use by a viewer application process; N Fig. 20 is a flow diagram showing the viewer application process; SFig. 21 is a flow diagram showing a viewer event handler process; Fig. 22 is a flow diagram showing a user input handler process; Fig. 23 is a flow diagram showing an activate layout sequence process; Fig. 24 is a flow diagram showing an activate layout process; Fig. 25 is a flow diagram showing a resume layout sequence process; Fig. 26 is a flow diagram showing a remove window from layout process; Fig. 27 is a flow diagram showing an add window to layout process; Fig. 28 is a flow diagram showing a trigger recording process; Fig. 29 is a flow diagram showing a pause layout sequence process; Fig. 30 is a flow diagram showing a pre-event buffering process; Fig. 31 is a schematic block diagram of a general-purpose computer upon which the viewer of Fig. 1 may be practiced; and Fig. 32 is a schematic block diagram of a general-purpose computer upon which the storage server of Fig. 1 may be practiced.

Detailed Description including Best Mode Where reference is made in any one or more of the accompanying drawings to steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears.

693277.doc It is to be noted that the discussions contained in the "Background" section relating to prior art arrangements relate to discussions of documents or devices which form public

U

knowledge through their respective publication and/or use. Such should not be N interpreted as a representation by the present inventor(s) or patent applicant that such documents or devices in any way form part of the common general knowledge in the relevant art.

For ease of explanation the following description has been divided into Sections Sto 1.3.5.

VIDEO SURVEILLANCE SYSTEM OVERVIEW Fig. 1 shows a video surveillance system 100. The system 100 comprises video cameras 112, 113, 114 and 115 connected to a computer network 3120, such as the Internet or an Intranet. Each of the cameras 112, 113, 114 and 115 are connected to the network 3120 via an associated camera server 109, 110 and 111, as seen in Fig. 1. In some implementations the network 3120 may be a local area network (LAN). In one implementation, one or more of the cameras 112-115 may be configured within an associated camera server 109-111, such that the camera and camera server form a single unit.

Some examples of proprietary cameras 112-115 are the Canon T M VC-C4 video camera. Some examples of proprietary camera servers 109-111 are the Canon

T

M VB150 and the Canon T m VB-C 10, where the VB-C10 is an example of a model where the camera and camera server are a single unit.

Each of the cameras 112, 113, 114 and 115 and the associated camera servers 109- 111, may be used to capture video image data representing one or more images. The video image data may be output by the camera servers 109-111 as a stream of video image data representing one or more video frames. Prior to outputting the video image 693277.doc -8data, the camera servers 109-111 may compress the video image data using any suitable Scompression method the JPEG compression method).

The camera servers 109-111 optionally comprise sensor inputs to which electronic K sensor devices motion sensors, heat and/or pressure sensors) may be connected. If a connected sensor is activated, then a camera server 109-111 may be configured to recognise that activated sensor and to generate an event notification in response to such San activation.

SThe system 100 also comprises storage servers 3200A, 3200B and 3200C. The storage servers 3200A, 3200B and 3200C may be used for monitoring the output of the video image data (or sample data) from any one of the camera servers 109-111 and for recording requesting and storing) the video image data. The storage servers 3200A, 3200B and 3200C may also be used for accessing the video image data, for event handling and for controlling the system 100. The storage servers 3200A to 3200C will hereinafter be generically referred to as 'the storage server 3200' excepting where explicitly distinguished.

The video image data captured by one or more of the cameras 112-115 and associated camera servers 109-111 may be uploaded, via the computer network 3120, from any one of the camera servers 109-111 to the storage server 3200. The video image data may be processed by the storage server 3200 and/or stored on a hard disk drive 3210 of the storage server (see Fig. 32). The stored video image data may be viewed by a user using a display 3214 (see Fig. 32) of the storage server 3200. The storage server 3200 may also be used to query the status of the electronic sensors connected to the camera servers 109-111.

Alternatively, the video image data may be uploaded, via the computer network 3120, from the storage server 3200 to one or more viewers 3100A, 3100B and 3100C, as seen in Fig. 1. The viewers 3100A to 3100C will hereinafter be generically referred to as 693277.doc -9the viewer 3100 excepting where explicitly distinguished. The viewer 3100 may be used Sfor processing and displaying the video image data, using a display device 3114 (see Fig.

U

S31) configured with the viewer 3100.

r The viewer 3100 may also be configured to receive the video image data directly from one or more camera servers 112-115 and to present the video image data to a user using the display device 3114. The viewer 3100 may also be used to query the status of Selectronic sensors connected to the camera servers 109-111.

SThe communications network 3120 may be a Transmission Control Protocol/Internet Protocol (TCP/IP) communications network. Communication between the camera servers 109-111, the storage server 3200 and the viewer 3100 may be performed over the communication network 3120 using the Hypertext Transfer Protocol

(HTTP).

As seen in Fig. 31, the viewer 3100 may be formed by a computer module 3101, input devices such as a keyboard 3102 and mouse 3103, output devices including a printer 3115, a display device 3114 and loudspeakers 3117. A network interface 3108 configured within the computer module 3101 may be used for communicating to and from the computer network 3120, for example connectable via a network link 3121 (such as a coaxial cable, a twisted pair cable, a fibre optic cable, a wireless connection using 802.1 lb or BluetoothTM or other connection type). A Modulator-Demodulator (Modem) transceiver device(not shown) incorporated within the network interface 3108 or otherwise, may also be used to obtain access to the computer network 3120, via a telephone line for example.

The computer module 3101 typically includes at least one processor unit 3105, and a memory unit 3106, for example, formed from semiconductor random access memory (RAM) and read only memory (ROM). The module 3101 also includes a number of input/output interfaces including an audio-video interface 3107 that couples to the 693277.doc video display 3114 and loudspeakers 3117, an I/O interface 3113 for the keyboard 3102 mouse 3103, printer 3115 and optionally a joystick (not illustrated) or trackball (not

U

illustrated). Optionally the module 3101 may include a touch-screen (not shown) formed (by an overlaid touch-sensitive surface on the video display 3114, allowing user input by touching or moving a finger along the video display 3114. A storage device 3109 is provided and may include a hard disk drive 3110 and a floppy disk drive 3111. A Smagnetic tape drive (not illustrated) may also be used. A CD-ROM drive 3112 is I/3 Stypically provided as a non-volatile source of data. The components 3105 to 3113 of the computer module 3101, typically communicate via an interconnected bus 3104 and in a manner, which results in a conventional mode of operation of a computer system as known to those in the relevant art. Examples of computers on which the described arrangements can be practiced include IBM-PC's and compatibles, Sun Sparcstations or alike computer systems evolved therefrom.

The storage server 3200 is also shown in detail in Fig. 32. The storage server 3200 may be formed by a computer module 3201, input devices such as a keyboard 3202 and mouse 3203, output devices including a printer3215, a display device3214 and loudspeakers 3217. A network interface 3208 is also configured within the computer module 3201 and can be used for communicating to and from the computer network 3120, for example connectable via network link 3221 (such as a coaxial cable, a twisted pair cable, a fibre optic cable, a wireless connection using 802.1 lb or Bluetooth T M or other connection type). A Modulator-Demodulator (Modem) transceiver device (not shown) incorporated within the network interface 3208 or otherwise, may also be used to obtain access to the computer network 3220, via a telephone line for example.

Similar to the computer module 3101, the computer module 3201 typically includes at least one processor unit 3205, and a memory unit 3206, for example formed from semiconductor random access memory (RAM) and read only memory (ROM). The 693277.doc -11module 3201 also includes an number of input/output interfaces including an audio- Svideo interface 3207 that couples to the video display 3214 and loudspeakers 3217, an 11O

U

interface 3213 for the keyboard 3202, printer 3215 and mouse 3203 and optionally a rjoystick (not illustrated) or trackball (not illustrated). Optionally the module 3201 may include a touch-screen (not shown) formed by an overlaid touch-sensitive surface on the video display 3214, allowing user input by touching or moving a finger along the video rdisplay 3214 A storage device 3209 is provided and typically includes a hard disk O drive 3210 and a floppy disk drive 3211. A magnetic tape drive (not illustrated) may also be used. Peripheral storage devices (not shown) connected to the computer module 3201 can be used. In addition, network accessible storage devices or collections of such devices (not shown), including Network Attached Storage (NAS) and Storage Area Networks (SAN), may be connected to the network 3120 and may be accessed through the network interface 3208. A CD-ROM drive 3212 is typically provided as a non-volatile source of data. The components3205 to3213 of the computer module3201, typically communicate via an interconnected bus 3204 and in a manner, which results in a conventional mode of operation of such a computer system as known to those in the relevant art.

The camera servers 109-111 have a similar configuration to the computer modules 3101 and 3201. The camera servers 109-111 include a memory memory 3206) and a processor a processor 3205). However, the hardware configuration of the camera servers 109-111 will not be explained in further detail herein. As described above, the camera servers 109-111 may have one or more electronic sensor devices motion sensor, heat connected to a particular one of the camera servers As described above, the storage server 3200 may be used for monitoring and handling events from electronic sensors sensors connected to the camera servers 109-111). One of these events may include motion detection using a motion detector (not 693277.doc -12shown) connected to one or more of the camera servers 109-111 directly. Further events c include heat/smoke detection using a heat/smoke detector, a door opening/closing using a

O

Slimit switch, for example.

C 1.1 STORAGE SERVER As described above, the storage servers 3200 may be used for monitoring the output of the video image data (or sample data) from any one of the cameras 112-115 and for Srecording requesting and storing) the video image data. The storage server 3200 Smay be controlled by a storage server software application 200 (see Fig. 2) together with an operating system. The storage server software application 200 comprises a file system module 201 and a network module 203. The storage server software application 200 also comprises a viewer interface module 205, a configuration management module 207, a video file management module 209, a camera server communications module 211, a HTTP server module 215 and a storage server control module 213.

1.1.1 Viewer interface module The viewer interface module 205 is responsible for processing messages (or requests) received by the storage server 3200 from the viewer 3100. The viewer 3100 may send messages in accordance with the HTTP protocol, using the HTTP POST method. The HTTP POST method defines the manner in which a HTTP client transfers data to a server.

The storage server software application 200 comprises an HTTP server module 215 that receives messages and sends the messages to the viewer interface module 205. The viewer interface module 205 executes a viewer interface process 300, which will be described in detail below with reference to Fig. 3. The process 300 waits for the arrival of a new message and calls one of a plurality of processes 400, 500, 600, 700 and/or 800, depending on the message, in order to process each incoming message. The processes called by the process 300 will be described in detail below with reference to Figs. 4 to 8.

693277.doc 13 The viewer interface module 205 operates using a session-based protocol, where each Sclient each of the viewers 3100A, 3100B and 3100C) registers with the viewer

U

Sinterface module 205 and receives a client identifier The client ID may be supplied r as part of subsequent messages. For each client, the viewer interface module 205 maintains the following information within memory 3206 of the storage server 3200: a client view list: a list of cameras 112-115 from which the client the viewer 3200A) is interested in viewing video image data; and (ii) a client record list: a list of cameras 112-115 that are currently being recorded (Nii from as a result of a "trigger recording" message from the associated client.

The client view list and the client record list are initially empty.

The viewer interface module 205 also maintains the following counters, and (ii), for each camera 112-115. The counters may be configured within memory 3106 or the hard disk drive 3210 of the storage server 3200.

client view count the number of clients that are interested in viewing video image data from the particular camera 112-115; and (ii) client record count the number of clients that have triggered recording video image data from the particular camera 112-115.

The client view count counter and the client record count counter are initially set to zero The content of each posted message (or request) may be configured as an XML document. The root element of such an XML document for a message is labelled "nvrcommand". The further contents of the XML document differ depending on the nature of the message. Typically, the root element comprises a sub-element that is named after the message type.

693277.doc -14- After processing the message, the processor 3205 of the storage server 3200 returns

O

Ca response in the form of an HTTP reply. Such a response may be stored within an XML document with a root element of type "nvr-response".

(The viewer interface process 300 will now be described in detail with reference to Fig. 3. The process 300 may be implemented as software resident on the hard disk drive 3210 of the storage server 3200 and being controlled in its execution by the processor 3205.

SThe process 300 begins at step 301, where the processor 3205 waits for a HTTP message. HTTP messages are received via the HTTP server module 215. The process 300 continues at the next step 302, where the processor 3205 examines the message that has been received to determine the type of the received message. The type of message may be determined by the processor 3205 by searching for an XML element corresponding to one of the known message types and examining the contents of the XML document passed as part of the message.

If the received message is a "login" message, then the process 300 proceeds to step 303. At step 303, the process 300 calls a "login" process 400, as shown in Fig. 4, to process the message. The process 300 then proceeds to step 309. The process 400 determines an initial layout that may be used by the client viewer 3100A) that sent the login message. The initial layout describes the manner in which video image data uploaded from the camera servers 109-111 are displayed on the display device 3114 of the viewer 3100. The process 400 also adds the cameras 112-115 associated with the layout to a client view list configured in memory 3206 for the client the viewer 3200A) that sent the login message. The process 400 also increments the client view counts for each of the cameras 112-115 that were added to the client view list. The process 400 will be described in detail below with reference to Fig. 4.

693277.doc If the received message is an "add cameras" message, then the process 300 proceeds

O

to step 304. At step 304, the process 300 calls an "add cameras" process 500, as shown in Fig. 5, to process the message. The process 300 then proceeds to step 309. As will be N described in detail below, the add cameras message may reference one or more cameras 112-115. The process 500 adds the cameras 112-115 referenced in the add cameras message to the client view list for the client that sent the add cameras message. The Sprocess 500 will be described in detail below with reference to Fig. SIf the received message is a "remove cameras" message, then the process 300 proceeds to step 305. At step 305, the process 300 calls a "remove cameras" process 600, as shown in Fig. 6, to process the remove cameras message. The process 300 then proceeds to step 309. As will be described in detail below, the remove cameras message may be sent to the storage server 3200 to stop pre-event buffering for one or more of the cameras 112-115. The process 600 instructs the storage server 3200 to remove one or more of the cameras 112-115 referenced in the remove cameras message from the client view list for the client that sent the remove cameras message. The process 600 will be described in detail below with reference to Fig. 6.

If the received message is a "trigger recording" message, then the process 300 proceeds to step 306. At step 306, the process 300 calls a "trigger recording" process 700, as shown in Fig. 7, to process the message. The process 300 then proceeds to step 309. As will be described in detail below, the trigger recording message may be sent to the storage server 3200 to request recording of video image data for a specified period of time. The process 700 disables any recording timer associated with one or more cameras 112-115 referenced by the trigger recording message and may start a new timer for the referenced camera 112-115 to expire after a predetermined time. The process 700 will be described in detail below with reference to Fig. 7.

693277.doc -16- If the received process is a "logout" message, then the process 300 proceeds to step 308. At step 308, the process 300 calls a "logout" process 800, as shown in Fig. 8, to process the message. The process 800 then proceeds to step 309. As will be described in Sdetail below, the logout message is sent to the storage server 3200 to indicate that a client is terminating a connection with the storage server 3200. The process 800 may remove all of the cameras 112-115 from the client record list for the client that sent the logout Smessage. The process 800 will be described in detail below with reference to Fig. 8.

SAt step 309, the process 300 enters a loop to process every camera 112-115 present in the surveillance system 100. At the next step 310, the process 300 calls a set video data acquisition process 1300 (see Fig. 13) for one of the cameras 112-115. The camera 112-115 currently being processed at step 310 will be referred to as "the current camera" for the purposes of the process 300. The process 1300 determines whether video image data should be acquired from the camera server 109-111 associated with the current camera 112-115. The process 1300 also ensures that any changes in the status of the current camera 112-115 are reflected during the acquisition of the video image data from the current camera 112-115. The process 1300, as executed at step 310, will be described in detail below with reference to Fig. 13.

When every camera 112-115 in the surveillance system 100 has been processed, at step 309, the process 300 returns to step 301 to await a further message.

1.2.1.1 Login message The login message comprises a login request and a login response. The login request is sent by the client the viewer 3100A) to the storage server 3200 the storage server 3200B) to request a client ID from the storage server 3200 as well as a list of all cameras 112-115, layouts and layout sequences present in the system 100. Layouts and layout sequences will be described in detail below.

The form of the XML document describing a login request is as follows: 693277.doc 17 <nvr- command> <login!> <nvr -command> The form of the XML document describing the login response is as follows: <nvr-response status="{status:okj*)"> <login> <clientid>{client-id}</clientid> <zone id=1"{zone-idP"> <name> {name }</name> <camera id=" {camera-id}"> <name> {namej}</name> </camera> <zone> <layout id=" {layout-id}"> <window cameraid=" {camera-id}",> <position x="{left}" w="{width)" h="{~height}"/> </window> </layout> <sequence id=" {sequence-id}"> <layout id=" {layout-id}"> <time> {layout-time} </time> </layout> sequence> </login> </nvr-response> 693277.doc -18- The login response comprises a status code, which is set to "ok" if the login request was processed successfully. Each successful login response contains a client ID, which is

U

Sa unique number designed to identify a client to the storage server 3200. The login Sresponse also contains a list of cameras 112-115 organised in a hierarchy of zones, a list of layouts and a list of layout sequences.

The login process 400, as executed at step 303, will now be described with Sreference to Fig. 4. The process 400 may be implemented as software resident in the hard Sdisk drive 3210 of the storage server 3200 and being controlled in its execution by the processor 3205. The process 400 is executed whenever a "login" message is received by the processor 3205 from the client the viewer 3100A).

The method 400 begins at step 401, where the processor 3205 selects a number to be assigned to the client as a client ID. The selected number is unique and may be generated by a random number generation algorithm. The selected number is written in a "clientid" element in the login response to be sent to the client. At the next step 403, the processor 3205 generates the list of cameras 112-115, layouts and layout sequences present in the system 100. The list is appended to the login response.

Then at step 405, the processor 3205 determines the initial layout that will be used by the client, and adds all of the cameras 112-115 in the initial layout to the client view list for the client. Also at step 405, the processor 3205 increments the camera view counts for each of the cameras 112-115 in the initial layout. The process 400 concludes at the next step 407, where the processor 3205 sends the login response to the client that generated the login request the viewer 3100 OA).

1.2.1.2 Add cameras message The add cameras message comprises an add cameras request and an add cameras response. The add cameras message is sent to the storage server 3200 to register a list of cameras 112-115 for pre-event buffering.

693277.doc 19- The form of the XML document describing an add cameras request is as follows:

O

d) <nvr-command> <clientid>{client-id}</clientid> <addcameras replace="{replace:true false}"> <camera id="{camera-id}"/>

(N

</addcameras> </nvr-command> C 10 A "clientid" element of the add cameras request comprises a client ID which was previously generated in the login process 400. An "addcameras" element comprises information specific to the add cameras message.

A "replace" attribute of the add cameras request instructs the storage server 3200 to delete the client view list associated with the client that sent the add cameras request and replace the client view list with only the cameras 112-115 listed in the add cameras request.

An "addcameras" element of the add cameras request may comprise a number of "camera" elements which are used to refer to cameras 112-115 which are to be added to the client view list for the client.

The form of the XML document describing the add cameras response is as follows: <nvr-response> <addcameras status="{status:okl*}"> <camera id="{camera-id}" status="{status:ok|*}"/> </addcameras> </nvr-response> 693277.doc The add cameras response comprises a status code, which is set to "ok" if the add ri cameras request was processed successfully. Additionally, each camera 112-115 in the

U

original request is listed with a status code, which is set to "ok" if that camera server 109- 111 was successfully added to the view list for the client.

The add cameras process 500, as executed at step 304, will now be described with Nreference to Fig. 5. The process 500 may be implemented as software resident in the hard Sdisk drive 3210 of the storage server 3200 and being controlled in its execution by the In Sprocessor 3205. The process 500 is executed whenever an "add cameras" request is received by the processor 3205 from the client the viewer 3200).

The process 500 begins at step 501, where if the processor 3205 determines that the "replace" flag of the add cameras message is set then the process 500 proceeds to step 503. Otherwise, the process 500 proceeds to step 505.

At step 503, the processor 3205 removes all cameras 112-115 from the client view list for the client. Also at step 503, the processor 3205 decrements the camera view counts of each camera 112-115 that is removed from the client view list. Then at step 505, the processor 3205 adds each camera 112-115 that is listed in the add cameras request to the client view list for the client that sent the add cameras request. The processor 3205 also increments the camera view counts of each camera 112-115 that is included in the add cameras request.

1.2.1.3 The remove cameras message The remove cameras message comprises a remove cameras request and a remove cameras response. The remove cameras message is sent to the storage server 3200 to stop pre-event buffering for one or more cameras 112-115.

The form of the XML document describing a remove cameras request is as follows: <nvr-command> 693277.doc <clientid>{client-id}<clientid> <removecameras all="{true false}"> <camera id="{camera-id}"/> </removecameras> </nvr-command> 0 A "removecameras" element of the remove cameras message comprises CNi information specific to the remove cameras message. An "all" attribute instructs the Sstorage server 3200 to delete the entire client view list associated with the client that

(N

transmitted the remove cameras message.

The form of an XML document describing the remove cameras response is as follows: <nvr-response> <removecameras status="{status:okl*}"/> </nvr-response> The remove cameras response contains a status code, which is set to "ok" if the remove cameras request was processed successfully.

The remove cameras process 600, as executed at step 305, will now be described with reference to Fig. 6. The process 600 may be implemented as software resident in the hard disk drive 3210 of the storage server 3200 and being controlled in its execution by the processor 3205.

The process 600 is executed whenever a "remove cameras" message is received by the processor 3205 from the client the viewer 3200). The process 600 begins at step 601, where if the processor 3205 determines that the "all" attribute is set in the remove cameras request then the process 600 proceeds to step 603. Otherwise, the process 693277.doc -22proceeds to step 605. At step 603, the processor 3205 removes all cameras 112-115 from

O

C the client view list of the client that sent the remove cameras message. Also at step 603,

U

Sthe processor 3205 decrements the camera view counts of each camera 112-115 that is removed from the client view list at step 603 and the process 600 concludes.

At step 605, the processor 3205 removes all cameras 112-115 that are listed in the ("i Sremove cameras request from the client view list for the client. Also at step 605, the processor 3205 decrements the camera view counts of each camera 112-115 that is Sremoved from the client view list for the client.

(N

1.2.1.4 The trigger recording message The trigger recording message comprises a trigger recording request and a trigger recording response. The trigger recording request is sent to the storage server 3200 to instruct the storage server 3200 to record video image data for a specified period of time.

The form of the XML document describing a trigger recording request is as follows: <nvr-command> <clientid>{client-id}</client-id> <trigger> <camera id="{camera-id}"/> <time>{time:int}</time> </trigger> </nvr-command> A "trigger" element of the trigger recording request comprises a "camera" subelement, which indicates the camera 112-115 which is to be triggered. The camera 112- 115 indicated by the camera sub-element will be referred to as "the current camera" for the purposes of the process 700.

693277.doc -23 The "trigger" element also contains a "time" sub-element, which indicates the time, N in milliseconds, that recording should take place for the current camera 112-115 indicated

O

by the camera sub-element. If the time sub-element contains a value of zero then the Strigger recording message is interpreted as an instruction to stop any currently executing recording triggered for the current camera 112-115 by the client sending the trigger recording message.

The form of an XML document describing the trigger recording response is as follows: <nvr-response status="{status:okl*}"/> The trigger recording response contains a status code, which is set to "ok" if the trigger recording request was processed successfully.

The trigger recording process 700, as executed at step 306, will now be described with reference to Fig. 7. The process 700 may be implemented as software resident in the hard disk drive 3210 of the storage server 3200 and being controlled in its execution by the processor 3205. The process 700 is executed whenever a "trigger recording" request is received by the processor 3205 from the client the viewer 3200).

The storage server 3200 maintains one operator triggered recording per camera 112- 115 per client 3100A, 3100B and/or 3100C), so that if such a recording is already running, the recording timer associated with the given camera 112-115 and client may be changed in accordance with the trigger recording request received from the client.

The process 700 begins at step 701, where the processor 3205 selects one or more frames in a frame buffer configured within memory 3206. In particular, the processor 3205 selects one or more frames whose time stamps fall within a predetermined pre-event time for operator triggered recordings. The predetermined pre-event time for operator triggered recordings may be set equal to thirty (30) seconds, for example. The processor 693277.doc -24- 3205 supplies the selected frames to the video file management module 209 of the storage N server software application 200. The video file management module 209 writes the

U

Sselected frames to the hard disk drive 3210. These selected frames are also removed from Sthe frame buffer once the frames are written to the hard disk drive 3210.

The process 700 continues at the next step 702, where if the processor 3205 determines that the current camera 112-115 the camera indicated by the camera sub- Selement of the trigger element) is already present in the client record list for the client, then the process 700 proceeds to step 707. Otherwise, the process 400 proceeds to step 705. At step 705, the processor 2305 adds the current camera 112-115 to the client record list for the current camera 112-115. Also at step 705, the processor 2305 increments the client record count variable for the current camera 112-115.

At step 707, the process 700 disables any recording timer associated with the current camera 112-115 and client, and starts a new timer to expire after the time period specified in the trigger recording request. The new timer may be managed by the operating system of the storage server 3200. When the new timer expires, the operating system calls a "recording timer expiry" process 1200, which will be described in detail below with reference to Fig. 12. The process 1200 removes the current camera 112-115 associated with the expired timer from the client record list of the client that sent the trigger recording request, as will be described in detail below.

1.2.1.5 The logout message The logout message comprises a logout request and a logout response. The logout message is sent to the storage server 3200 to instruct the storage server 3200 that a client is terminating its connection to the storage server 3200.

The form of the XML document describing a logout request is as follows: <nvr-command> <clientid>{client-id} </client-id> 693277.doc <logout continue="true false"> c </nvr-command> C The logout request comprises a "continue" flag which indicates whether the storage server 3200 should continue any operator triggered recordings initiated by the client.

CN The form of the XML document describing a logout response is as follows: C) <nvr-response status="{status:ok|*}"/> c- 10 The logout response comprises a status code, which is set to "ok" if the logout request was processed successfully. After a successful logout message has been processed by the storage server 3200, the client ID used in the logout message can no longer be used in any subsequent messages.

The logout process 800 will now be described with reference to Fig. 8. The process 800 may be implemented as software resident in the hard disk drive 3210 of the storage server 3200 and being controlled in its execution by the processor 3205. The process 800 is executed whenever a "logout" request is received by the processor 3205 from the client the viewer 3200).

The process 800 begins at step 801, where if the processor 3205 determines that a "continue" flag is set in the logout message then the processor 3205 proceeds to step 805.

Otherwise, the processor 3205 proceeds to step 803. At step 803, the processor 3205 removes all of the cameras 112-115 from the client record list of the client. The process also decrements the record count for each of the removed cameras 112-115. At step 805, the processor 3205 removes all of the cameras 112-115 from the client view list of the client. The processor 3205 also decrements the view count for each of the cameras 112- 115 removed from the client view list of the client that sent the logout message. The process 800 concludes at the next step 807, where the processor 3205 discards the client 693277.doc -26- ID of the client that sent the logout message so that subsequent requests using the same C client ID will fail.

U

1.2.2 Storage Server Control The storage server control module 213 comprises a scheduling module 217 and a frame buffer module 219.

1.2.2.1 Scheduling module SThe storage server 3200 is controlled by the scheduling module 217. Each camera S112-115 registered with the storage server 3200 has an associated set of schedules.

Schedules are split into weekdays so that a different schedule may be set for each day of the week. As an example, Fig. 9 shows a sample schedule 900 for Monday with three schedule items 901, 903 and 905 having non-overlapping periods. Each schedule item 901, 903 and 905 specifies whether recording of video data is to be performed for an associated camera 112-115 during the specified time period. A schedule item 901, 903 and 905 may also specify alternate recording settings to be applied when a certain event is activated by one of the electronic sensors connected the camera server 109-111 associated with the associated camera 112-115. The sensor event settings may also contain a preevent recording time. The pre-event recording time indicates the number of seconds of video image data preceding the event to be recorded.

Fig. 10 is a flow diagram showing a set camera schedule process 1000 for setting up the scheduling component 217 to interact with a single camera 112-115. This single camera 112-115 will be referred to as "the current camera" for the purposes of Fig. 10 and related processes. The process 1000 may be implemented as software resident in the hard disk drive 3210 and being controlled in its execution by the processor 3205. The process 1000 is executed for each camera 112-115 when the storage server 3200 begins execution. The process 1000 is also executed when a schedule item timer expires, as will be described below with reference to Fig. 11.

693277.doc -27- The process 1000 begins at step 1001, where the processor 3205 determines the CK, schedule item that should be active for the current camera 112-115 at the current time.

U

The processor 3205 may obtain the current time from the operating system of the storage server 3200. The processor 3205 may determine a schedule item with a start time that is less than or equal to the current time and an end time that is greater than or equal to the current time.

At the next step 1003, if a suitable schedule item was found for the current camera S112-115 by the processor 3205, the process 1000 proceeds to step 1005. Otherwise, the process 1000 proceeds to step 1007. At step 1005, the process 1000 instructs the operating system of the storage server 3200 to set up a schedule item timer which expires at the end of the current schedule item and the process 1000 proceeds to step 1009.

At step 1007, the process 1000 determines the next schedule item that is due to occur for the current camera 112-115 and instructs the operating system of the storage server 3200 to set up a schedule item timer for the current camera 112-115. This schedule item timer expires at the beginning of the next schedule item. At step 1009, the process 1000 calls a "set video image data acquisition" process 1300 to determine whether video image data should be acquired for the current camera 112-115. The process 1300 will be described below with reference to Fig. 13.

Fig. 11 is a flow diagram showing a schedule item timer process 1100. The process 1100 may be implemented as software resident on the hard disk drive 3210 and being controlled in its execution by the processor 3205. The process 1100 is executed by the storage server 3200 upon the storage server 3200 being notified by the operating system that one of the schedule item timers has expired. At step 1101, the process 1100 calls the set camera schedule process 1000 as described above, to set up the scheduling component 217 to interact with the current camera 112-115.

693277.doc -28- The recording timer expiry process 1200, as executed at step 707, will now be Sdescribed with reference to Fig. 12. The process 1200 may be implemented as software

U

Sresident in the hard disk drive 3210 and being controlled in its execution by the processor S3205. The process 1200 is executed by the storage server 3200 upon the storage server 3200 being notified by the operating system of the storage server 3200 that one of the operator recording timers has expired.

SThe process 1200 begins at step 1201, where if the processor 3205 determines that Sthe client record list for the current camera 112-115 the camera indicated by the camera sub-element of the trigger element of the trigger recording message) is still present in memory 3206, then the process 3200 proceeds to step 1205. Otherwise, the process 3200 proceeds to step 1203. At step 1203, the process 1200 removes the current camera 112-115 from the client record list for the client. At step 1205, the process 1200 decrements the client record count for the current camera 112-115. Then at the next step 1207, the process 1200 calls the set video data acquisition process 1300 for the current camera 112-115, to determine whether video image data should be acquired from the camera server 109-111 for the current camera 112-115, as will be described below with reference to Fig. 13.

The process 1300 may be implemented as software resident in the hard disk drive 3110 and being controlled in its execution by the processor 3205. The process 1300 is executed by the storage server 3200 in order to determine whether video image data should be acquired from the camera server 109-111 for a current camera 112-115.

The process 1300 begins at step 1301, where if the processor 3205 determines that operator related acquisition is required for the current camera 112-115 then the process 1300 proceeds to step 1315. The current camera 112-115 for the purposes of the process 1300 is the camera 112-115 being processed at step 310 of the process 300, the camera 693277.doc -29for which the camera schedule was set in accordance with the process 1000, or the camera 112-115 whose associated recording timer expired in accordance with the process 1200.

U

SThe determination is made at step 1301 based on the client view count for the current camera 112-115. If at least one of the client view count or client record count variables for the current camera 112-115 is greater than zero then the processor 3205 determines that operator related acquisition is required. Otherwise, the process 1300 proceeds to step 1303.

SAt step 1303, the processor 3205 selects a schedule item 900) which is active for the current camera 112-115 at the current time. The schedule item is selected at step 1303 by obtaining the current time from the operating system of the storage server 3200, and determining a schedule item with a start time that is less than or equal to the current time. The processor 3205 also determines an end time that is greater than or equal to the current time. The schedule item selected at step 1303 will be referred to as "the current schedule item". At the next step 1305, if no suitable schedule item was found at step 1303, then the process 1300 proceeds to step 1317. Otherwise, the process 1300 proceeds to step 1307.

At step 1307, if the processor 3205 determines that the selected schedule item specifies continuous recording then the process 1300 proceeds to step 1315. Otherwise, the process 1300 proceeds to step 1309.

At step 1309, the process 1300 enters a loop, processing each individual sensor that is present in the current schedule item. The sensor that is currently being processed in steps 1309 to 1313 will be referred to as "the current sensor". When the loop terminates, the process 1300 proceeds to step 1317. At step 1309, if the processor 3205 determines that there are more sensors required to be processed then the process 1300 proceeds to step 1311. Otherwise, the process 1300 proceeds to step 1317.

693277.doc At step 1311, if the processor 3205 determines that the current sensor is currently Striggered then the process 1300 proceeds to step 1315. Otherwise, the process 1300 proceeds to step 1313. At step 1313, if the processor 3205 determines that the current C schedule item provides a positive pre-event recording time for the current sensor then the process 1300 proceeds to step 1315. Otherwise, the process 1300 returns to step 1309.

At step 1315, the processor 3205 calls a "begin video data acquisition" process 1400, in order to begin video data acquisition for the current camera 112-115. The Sprocess 1400 is executed in order to ensure that video image data is being recorded for the current camera 112-115. The process 1400 will be explained in detail below with reference to Fig. 14. The process 1300 then concludes following step 1315.

At step 1317, the process 1300 calls a "stop video data acquisition process" 1500 to ensure that video image data is not being recorded for the current camera 112-115. The stop video data acquisition process 1500 will be described in detail below with reference to Fig. 15. The process 1300 then concludes following step 1317.

The begin video data acquisition process 1400 may be implemented as software resident on the hard disk drive 3210 and being controlled in its execution by the processor 3205. As described above, the process 1400 is executed by the storage server 3200 to ensure that video data is being acquired for the current camera 112-115 the camera indicated by the camera sub-element of the trigger element of the trigger recording message).

The process 1400 begins at step 1401, where the processor 3205 determines whether video data is currently being acquired for the current camera 112-115. Step 1401 may be implemented by determining if a video data acquisition process 1600, as seen in Fig. 16, is currently being executed. If video image data is currently being acquired for the current camera 112-115 then the process 1400 concludes. Otherwise, the process 1400 proceeds to step 1403. At step 1403, the processor 3205 sends a message to the 693277.doc i -31 camera server 109-111 associated with the current camera 112-115 in order to generate a r stream of video image data. The message sent at step 1403 may be an HTTP request with

U

a resource named "GetVideo". The process 1400 continues at the next step 1405, where N the processor 2305 executes the video data acquisition process 1600 for the current camera 112-115, which will be described in detail below with reference to Fig. 16.

(The stop video data acquisition process 1500 may be implemented as software resident on the hard disk drive 3210 and being controlled in its execution by the processor S3205. As described above, the process 1500 is executed by the storage server 3200 to ensure that video image data is not currently being acquired for the current camera 112- 115 the camera indicated by the camera sub-element of the trigger element of the trigger recording message).

The process 1500 begins at step 1501, where if the processor 3205 determines that video image data is currently being recorded for the current camera 112-115, then the process 1500 proceeds to step 1503. Otherwise, the process 1500 concludes. Step 1501 may be implemented by determining if a video data acquisition process 1600, as seen in Fig. 16, is currently being executed. At the next step 1503, the process 1500 terminates the video data acquisition process 1500. Then at step 1505, the process 1500 closes the network connection used for acquiring the video image data from the current camera 112- 115.

1.2.2.2 Frame buffer module The frame buffer module 219 of the storage server control module 213 manages the video image data received from camera servers 109-111. A separate frame buffer may be configured within memory 3210 for each of the cameras 112-115 in order to store video image data representing one or more frames of video image data captured by each of the cameras 112-115. Each of these separate frame buffers provides a temporary circular buffer for a particular camera 112-115, as will be described below. If a camera 112-115 693277.doc -32is currently configured to record video image data, the video image data representing image frames may be written to the hard disk drive 3210. If the camera 112-115 is not

O

currently configured to record video image data, the frame buffer module 219 may preserve all video image data, which falls within the bounds of any current pre-event recording requirements. This video image data may be preserved in a frame buffer associated with the camera 112-115, The video data acquisition process 1600 is executed by the storage server 3200 as an independent process for each of the cameras 112-115. The process 1600 receives video image data captured by the current camera 112-115 the camera indicated by the camera sub-element of the trigger element of the trigger recording message) from a camera server 109-111 associated with the current camera 112-115. The process 1600 stores the video image data captured by the current camera 112-115 in a frame buffer associated with the current camera 112-115. The process 1600 then writes the video image data stored in the frame buffer to the hard disk drive 3210 depending on whether continuous recording is enabled for the current camera 112-115, whether sensor triggered recording is enabled for the current camera 112-115 or whether operator triggered recording is enabled for the current camera 112-115. Thus, each of the frame buffers essentially acts a temporary circular buffer for a particular camera 112-115.

The process 1600 may be implemented as software resident on the hard disk drive 3210 and being controlled in its execution by the processor 3205. The process 1600 begins at step 1601, where the processor 3205 receives a header from the camera server 109-111 associated with video image data captured by the current camera 112-115. At the next step 1603, the processor 3205 waits to detect an incoming frame of video image data on the communications network 3120. Upon detecting such an incoming frame, at the next step 1605, the processor 3205 reads the size and time stamp of the incoming frame of video image data and allocates a block of the frame buffer configured within 693277.doc -33 memory 3206 for the frame of video image data. The allocated block of the frame buffer N is large enough to hold the incoming frame of video image data. The frame buffer may be

U

enlarged if necessary.

At the next step 1607, the processor 3205 reads the data for the incoming frame of video image data and writes the frame of video image data to the allocated memory block reserved for the frame. Then at the next step 1609, if the processor 3205 determines that Sthe schedule item selected at step 1303 the current schedule item) for the current camera 112-115 has continuous recording enabled, the process 1600 proceeds to step 1615. Otherwise, the process 1600 proceeds to step 1611.

At step 1611, if the processor 3205 determines that the current schedule item for the current camera 112-115 has an entry corresponding to a currently triggered electronic sensor with continuous recording enabled, then the process 1600 proceeds to step 1615.

Otherwise, the process 1600 proceeds to step 1613.

At step 1613, if the processor 3205 determines that the client record count variable for the current camera 112-115 is greater than zero then the process 1600 proceeds to step 1615. Otherwise, the process 1600 proceeds to step 1617.

At step 1615, the process 1600 writes the video image data representing the incoming frame of video image data received at step 1603 from the frame buffer to the hard disk drive 3210 using the video file management module 209. Then at the step 1617, the process 1600 determines the maximum of the following values and (ii): pre-event times for all sensor events in the current schedule; and (ii) pre-event time for operator triggered recording if the client view count is greater than zero The process 1600 then removes from the frame buffer, all frames of video image data whose time stamp is earlier than the pre-event period indicated by the maximum pre- 693277.doc -34event time determined at step 1617. Accordingly, each of the frame buffers acts a c temporary circular buffer for a particular camera 112-115.

U

1.2.3 Camera server communications The camera server communications module 211 is used by the storage server 3200 to communicate with camera servers 109-111 that are present on the communications network 3120.

SFig. 17 is a flow diagram showing an event notification process 1700. The process (Ni 1700 may be implemented as software resident in the hard disk drive 3210 and being controlled in its execution by the processor 3205. The process 1700 may be executed independently for each of the camera servers 109-111. The process 1700 receives notifications of changes in the status ('status notifications') of any external electronic sensors attached to a current camera server 109-111 as a response to the sent HTTP request.

The process 1700 begins at step 1701, where the process 1700 sends a request to the camera server 109-111 to begin sending status notifications on the communications network 3120. Step 1701 may be implemented by sending an HTTP request with a resource named "GetNotice". At the next step 1703, the processor 3205 waits for a new status notification to be sent by the current camera server 109-111. The new status notification will be referred to as 'the current status notification' for the purposes of the process 1700.

Then at the next step 1705, if the processor 3205 determines that the current status notification indicates an "off to on" transition in sensor status, then the process 1700 proceeds to step 1707. Otherwise, the process 1700 proceeds to step 1709. At step 1707, the processor 3205 selects all frames of video image data in the frame buffer associated with a current camera 112-115 the camera 112-115 associated with the current camera server 109-111) whose time stamp falls within a pre-event time specified in the 693277.doc settings for the current sensor in the current schedule item. The processor 3205 supplies N, the selected frames of video image data to the video file management module 309. The

U

Svideo file management module 309 writes the selected frames of video image data to the hard disk drive 3210. Whenever pre-event video image data is written to the hard disk s drive 3210, all frames of video image data corresponding to the written frames of video image data are removed from the frame buffer. The process 1300 continues at the next step 1709, where the process 1700 updates the sensor status to correspond to the status Sdescribed in the status notification received at step 1703. Then at the next step 1711, the process 1700 calls the set video data acquisition process 1300 to change the video data acquisition status as a result of the sensor status change. The process 1700 then proceeds back to step 1703.

1.2.4 Configuration management module The configuration management module 207 extracts information from configuration files stored on the hard disk drive 3210 in order to run the storage server 3200. This information may include: a list of camera servers 109-111; (ii) a list of cameras 112-115 present on each camera server 109-111; (iii) a list of schedule items 900); (iv) a list of layouts; and a list of layout sequences.

1.2.5 Video file management module The video file management module 209 writes video image data to the hard disk drive 3210 of the storage server 3210. The video file management module 209 may interact with the file system module 201 by calling functions provided by the operating system of the storage server 3200.

1.2.6 Variable frame rates 693277.doc -36- Each of the storage server schedule items may specify a frame rate associated with

C

each of continuous and sensor triggered recording settings. For example, Fig. 18 shows a

U

Ssample schedule item 1800 with frame rates 1801, 1803 and 1805 specified. Frame rates Care expressed in thousandths of frames per second. Thus, a frame rate of 25fps may be expressed as 25000.

SFor variable frame rates, the trigger recording message described above may Ccontain an additional sub-element to indicate the frame rate at which an operator triggered Srecording is to occur. The form of the XML document describing a trigger recording request specifying a frame rate is as follows: <nvr-command> <clientid>{client-id </client-id> <trigger> <camera id="{camera-id}"/> <time>{time:int}</time> <rate>{rate:int}</time> </trigger> </nvr-command> The "rate" sub-element expresses the recording frame rate in thousandths of frames per second. As described above, a frame rate of 25fps may be expressed as 25000.

The add cameras message described above may also contain an additional subelement to indicate the frame rate at which pre-event buffering should occur. The form of the XML document describing the add cameras message request specifying a frame rate is as follows: <nvr-command> <clientid>{client-id}</clientid> <addcameras replace="{replace:true|false}"> 693277.doc -37- <camera id="{camera-id}"> CK1 <rate>{rate:int}</rate> </camera> </addcameras> </nvr-command> SSimilarly, the "rate" sub-element of the add cameras message expresses the acquisition CN frame rate in thousandths of frames per second. The desired acquisition frame rate may Sbe specified as an argument to the "GetVideo" message sent to a camera server 109-111.

For example, a 25fps video stream may be requested from the camera server 109-111 by requesting an HTTP resource of the form "GetVideo?framerate=25000".

When determining the rate of data acquisition, the storage server 3200 determines the rates of all currently active recording and pre-event buffering settings and selects the maximum of these rates.

When determining the rate of data recording, the storage server 3200 determines the rates of all currently active recording settings and selects the maximum of these rates. If video data acquisition is being performed at a rate higher than the current recording rate, then the storage server writes enough frames in order to achieve a recording frame rate that is approximately equal to the desired recording frame rate.

1.3 Viewer As described above, the viewer 3100 may be used by a user for processing and displaying the video image data, using a display device 3114 (see Fig. 31) configured with the viewer 3100. The viewer 3100 may be controlled by a viewer application process 2000 (see Fig. 20) together with an operating system. The viewer 3100 may also be configured to receive the video image data directly from one or more camera servers 109-111 and to present the video image data to a user using the display device 3114. The 693277.doc -38viewer 3100 may also be used to query the status of electronic sensors connected to the camera servers 109-111.

U

The viewer software application may accept user input in various forms, which will be described below. Each type of user input generates a type of event. Fig. 19 shows a user interface 1900 for use by the viewer software application. The user interface 1900 comprises a layout view 1901, a camera list 1902 and a menu 1903.

1.3.1 Layout view SThe layout view 1901 displays one or more streams of video image data for viewing by a user of the system 100. Each stream of video image data may be associated with a video window 1904, 1905 and 1906. Each video window 1904, 1905 and 1906 is given a set of co-ordinates within the layout view 1901. Each video window 1904, 1905 and 1906 has controls which allow the video window to be resized. Each video window 1904, 1905 and 1906 also has a context menu, which may be accessed by clicking on the particular video window using the mouse 3103 in a conventional manner by clicking the right button of the mouse 3103). The context menu of each video window 1904, 1905 and 1906 may contain the following items: Trigger recording, which generates a "Trigger recording" event typically with a time of 60000 milliseconds; (ii) Stop triggered recording, which generates a "Trigger recording" event with a time of 0 milliseconds; and (iii) Delete video window from layout, which generates a "Delete video window from layout" event.

The viewer 3100 maintains a list of layouts, each of which contains a list of video windows 1904) and co-ordinates of the windows user interface 1900. When a layout is activated, all video windows currently present in the layout view 1901 are removed and 693277.doc -39new video windows 1904) corresponding to the activated layout are created in the r positions indicated in the layout description associated with the activated layout.

U

The viewer 3100 additionally maintains a list of layout sequences. A layout N sequence comprises a list of previously defined layouts together with the duration for each layout. When a layout sequence is selected, the viewer software application displays N a first layout of the sequence in the layout view 1901, and periodically changes the layout displayed in the layout view 1901 as described in the selected layout sequence. Upon (Ni Sreaching the end of the layout sequence, the viewer software application continues from the beginning of the layout sequence.

The lists of layouts and layout sequences may be downloaded by the viewer 3100 as part of the login process 400 described above.

1.3.2 Camera list The camera list 1902 displays the cameras 112-115 in the system 100. The list of cameras 1902 may be downloaded by the viewer 3100 as part of the login process. Each camera 112-115 is displayed as an icon 1907). Each icon contains a thumbnail image representative of a scene which is being viewed by an associated camera 112-115.

Using the mouse 3103 in a conventional manner, the user may drag any of the icons 1907 in the camera list 1902 to the layout view 1901 in order to add a new video window 1904 corresponding to the selected camera 112-115 to a currently selected layout at a position selected by the user. Dragging the icons 1907 in this manner results in the generation of a "drag camera thumbnail to layout" event.

1.3.2 Menu The menu 1903 may be used to generate user input events. The following items may be present in the menu 1903: A list of layout sequences, each of which generates an "activate layout sequence" event for the selected layout sequence; 693277.doc (ii) A list of layouts, each of which generates an "activate layout" event for the Sselected layout; and

U

(iii) An option entitled "Resume paused layout sequence".

(,i 1.3.4 Pre-event buffering for operator triggered recording Fig. 30 is a flow diagram showing a process 3000 for providing pre-event buffering for operator recording. The process 3000 may be implemented by the processor 3105 of the viewer 3100 and may be resident in the hard disk drive 3110.

SThe process 3000 begins at step 3001, where the processor 3105 establishes a connection between the viewer 3100 and the storage server 3200. This connection may be established by the viewer 3100 sending a "login" message to the storage server 3200. At the next step 3003, an initial layout is selected by the viewer 3100. If the viewer 3100 is currently running a layout sequence, the initial layout is the first layout of the layout sequence. The initial layout will be referred as "the current layout" for the purposes of the process 3000 and related processes.

Then at the next step 3005, the processor 3105 sends a list of all cameras 112-115 in the current layout to the storage server 3200. Step 3005 may be implemented by sending an add cameras message with the "replace" flag set to the storage server 3200. As described above, the add cameras message lists all cameras 112-115 in the current layout.

In response to receiving the add cameras message, the storage server 3200 stops buffering storing) video image data from cameras 112-115 not present in the current layout and begins buffering video image data from cameras 112-115 that are present in the current layout.

The process 3000 continues at the next step 3009, where if the viewer 3100 determines that new video windows 1904) have been added to the current layout since the last time the viewer 3100 checked, the process 3000 proceeds to step 3011.

Otherwise, the process 3000 proceeds to step 3013.

693277.doc -41- At step 3011, the processor 3105 sends a list of all cameras 112-115 added to the Scurrent layout, to the storage server 3200. Step 3011 may be implemented by the

U

processor 3105 sending an "add cameras" message. Such an add cameras message has N the "replace" flag cleared and lists the newly added cameras 112-115, to the storage server 3200. In response to receiving the add cameras message, the storage server 3200 begins buffering video image data from the cameras 112-115 corresponding to the newly Sadded video windows 1904). Following step 3011, the process 3000 then returns to Sstep 3009.

At step 3013, if the viewer 3100 determines that any video windows have been removed from the current layout since the last time the viewer 3100 checked, then the process 3100 proceeds to step 3015. Otherwise, the process 3100 proceeds to step 3017.

At step 3015, the processor 3105 sends a list of all cameras 112-115 removed from the current layout to the storage server 3200. Step 3015 may be implemented by the processor 3105 sending a remove cameras message to the storage server 3200. Such a remove cameras message has the "all" flag cleared and lists the newly removed cameras 112-115. In response to receiving the remove cameras message, the storage server 3200 ceases buffering from the cameras 112-115 removed from the current layout. Following step 3015, the process 3000 returns to step 3009.

At step 3017, if the viewer 3100 determines that the current layout has been changed as a result of user intervention or layout sequence operation, the process 3000 returns to step 3005. Otherwise, the process 3000 proceeds to step 3019. At step 3019, if the processor 3205 determines that the current layout sequence has been changed as a result of user intervention, then the process 3000 returns to step 3003. Otherwise, the process 3000 proceeds to step 3021.

At step 3021, in response to the user of the system 100 signaling the viewer 3100 to exit, the viewer 3100 sends a logout message to the storage server 3200 to terminate the 693277.doc -42connection between the viewer 3200 and the storage server 3200. As a result of the r connection being terminated, the storage server 3200 ceases buffering video image data

U

from all cameras 112-115 present in the current layout.

SFig. 20 is a flow diagram showing the viewer application process 2000, as executed by the viewer 3100. The viewer application process 2000 may be resident in the hard disk drive 3110 and being controlled in its execution by the processor 3105.

SThe process 2000 begins at step 2001, where the processor 3105 logs the viewer (,1 3100 in with the storage server 3200. Step 2001 may be implemented by sending a "login" request to the storage server 3200. At the next step 2003, the processor 3105 receives the login response including a list of cameras, layouts and layout sequences from the storage server 3200. The list of cameras is used to populate the camera list 1902 of the user interface 1900. The lists of layouts and layout sequences are used to populate the menu 1903. Then at the next step 2005, if the processor 3105 determines that the storage server 3200 indicated that an initial layout sequence should be started, then the process 2000 proceeds to step 2009. Otherwise, the process 2000 proceeds to step 2007. At step 2009, the processor 3105 activates the initial layout sequence. The initial layout sequence may be activated by calling an "activate layout sequence" process 2300, which will be described in detail below with reference to Fig. 23. At step 2007, the process 2000 activates the initial layout. The initial layout may be activated by calling an "activate layout" process 2400, which will be described in detail below with reference to Fig. 24.

The process 2000 then proceeds to step 2011, where the process 2000 calls a "viewer event handler" process 2100, which will be described in detail below with reference to Fig. 21. The process 2100 processes user input events.

The process 2100 may be implemented as software resident in the hard disk drive 3110 and being controlled in its execution by the processor 3105. The process 2100 begins at step 2101, where the process 2100 awaits a new event. If the event is the expiry 693277.doc -43 of a layout sequence pre-emption timer, the process 2100 proceeds to step 2103. If the

O

N event is the expiry of a layout sequence change timer, the process 2100 proceeds to step 2105. If the event is caused by user input, the process 2100 proceeds to step 2113. If the Sevent is an indication that the viewer application process 2000 is about to exit, then the process 2100 proceeds to step 2109.

At step 2105, the processor 3105 selects the next layout in the current layout Ssequence the layout sequence activated at step 2009). Then at the next step 2107, the Sprocessor 3105 switches to the next layout by calling the activate layout process 2400.

At step 2103, the process 2100 calls a resume layout sequence process 2500 to set up timers and pre-emption information, which will be described in detail below with reference to Fig. 25. The process 2500 ensures that the layout sequences are active.

Following step 2103, the process 2100 returns to step 2101.

At step 2113, the process 2100 calls a "user input handler" process 2200 to process user input events as will be described in detail below with reference to Fig. 22. Following step 2113, the process 2100 returns to step 2101. The process 2200 handles any user input events generated by the viewer event handler process 2100.

At step 2109, the process 2100 prompts the user as to whether any recordings initiated by the user should continue running. This prompt may be implemented using a "Yes/No/Cancel" dialog, displayed on the display 3114, for example. The process 2100 then continues at the next step 2111, where the processor 3105 creates and sends a "logout" message to the storage server 3200 to terminate the connection with the storage server 3200. The "continue" flag is set in the logout message to correspond with the answer provided by the user in step 2109.

As described above, the process 2200 executed at step 2113 of the process 2100 handles any user input events generated during the viewer event handler process 2100.

693277.doc 0 -44- The process 2200 may be implemented as software resident in the hard disk drive 3110

O

rand being controlled in its execution by the processor 3105.

The process 2200 begins at step 2201, where the processor 3105 examines the type (of user input event being processed. If the processor 3105 detects an "activate layout sequence" event indicating that the user has activated a layout sequence, the process 2200 proceeds to step 2203. If the processor 3105 detects a "resume paused layout sequence" Sevent indicating that the user has activated a paused layout, the process 2200 proceeds to Sstep 2205. If the processor 3105 detects an "activate layout" event indicating that the user has activated a new layout, the process 2200 proceeds to step 2207. This new layout will be referred to as the current layout for the purposes of the process 2200 and related processes. If the processor 3105 detects a "drag camera thumbnail to layout event" indicating that the user has dragged one of the thumbnails 1907) into the layout view 1901, the process 2200 proceeds to step 2209. If the processor 3105 detects a delete video window from layout event indicating that the user has deleted one of the video windows 1904) from the layout view 1901, the process 2200 proceeds to step 2211.

If the processor 3105 detects a trigger recording event indicating that the user has triggered a recording of video image data from one of the cameras 112-115 for a specified period of time, the process 2200 proceeds to step 2213.

At step 2203, the process 2200 calls the activate layout sequence process 2300 to activate the layout sequence for the current layout. The process 2200 then concludes.

At step 2205, the process 2200 calls the resume layout sequence process 2500 to set up timers and pre-emption information based on a currently paused layout. The process 2200 then concludes.

At step 2207, the process 2200 stops the current layout sequence, if one is currently active, by killing the layout sequence pre-emption and change timers associated with the 693277.doc current layout. At the next step 2215, the process 2200 calls the activate layout process S2400 to activate the current layout selected by the user. The process 2200 then concludes.

U

At step 2209, the process 2200 calls an "add window to layout" process 2700, which will be described in detail below with reference to Fig. 27, to add a window 1904) to the layout view 1901 based on the thumbnail 1907) dragged by the user to the layout view 1901.

SAt step 2211, the process 2200 calls a "remove window from layout" process 2600, Swhich will be described below with reference to Fig. 26, to remove the deleted window from the layout view 1901 of the current layout.

At the next step 2213, the process 2200 calls a "trigger recording" process 2800, which will be described in detail below with reference Fig. 28. The trigger recording process 2800 instructs the storage server 3200 to record video image data from a camera 112-115 for a specified period of time.

The activate layout sequence process 2300, as executed at step 2203, is called to activate each new layout sequence. The process 2300 may be implemented as software resident in the hard disk drive 3110 and being controlled in its execution by the processor 3105.

The process 2300 begins at step 2301, where the process 2300 sets the current layout sequence to the layout sequence activated by the user at step 2203. At the next step 2303, the processor 3105 selects the first layout in the activated layout sequence to be displayed on the display 3114. This first layout will be referred to as the current layout for the purposes of the process 2300.

The process 2300 continues at the next step 2305, where the process 2300 calls the activate layout process 2400 to activate the current layout. At the next step 2307, the process 2300 calls the resume layout sequence process 2500 to set up timers and preemption information for the current layout.

693277.doc -46- The activate layout process 2400, as executed at step 2207, is called whenever a new layout is activated. The new layout will be referred to as the "current layout" for the

U

purposes of the process 2400. The process 2400 may be implemented as software resident in the hard disk drive 3210 and being controlled in its execution by the processor 3205.

The process 2400 begins at step 2401, where the processor 3105 closes all camera Sserver connections that are currently open. At the next step 2403, the process 2400 removes all camera windows 1904) currently present in the layout view 1901. Then at step 2405, the process 2400 sends a list of all cameras 112-115 in the current layout to the storage server 3200. Step 2405 may be implemented by the processor 3205 generating an add cameras message with the "replace" flag set and listing the cameras 112-115 in the current layout. The process 2400 may then send the add cameras message to the storage server 3200. At the next step 2407, the process 2400 opens a connection to each camera 112-115 in the current layout and begins receiving streamed video image data from each connection. Then at the next step 2409, the process 2400 creates a video window 1904) for each camera 112-115 in the current layout.

The process 2500 is called to ensure that the layout sequence timers are active. The process 2500 may be implemented as software resident on the hard disk drive 3110 and being controlled in its execution by the processor 3105. The process 2500 is executed if a layout sequence is currently active.

The process 2500 begins at step 2501, where if the processor 3105 determines that the time that is remaining before the next layout switch is less than or equal to a preemption time a constant value of 30 seconds), then the process 2500 proceeds to step 2503. Otherwise, the process 2500 proceeds to step 2511. The time that is remaining before the next layout switch is determined by subtracting the time that the current layout has been active from the duration of the current layout as specified in the layout sequence.

693277.doc -47- At step 2503, the processor 3105 creates a list of cameras 112-115 that are Sreferenced in the next layout in the current layout sequence. Then at the next step 2507,

U

the process 2500 sends the list of cameras 112-115 to the storage server 3200. Step 2507 may be implemented by the processor 3105 creating an "add cameras" message containing the list of cameras 112-115 and sending the add cameras message to the storage server 3200. In this instance, the "replace" flag in the add cameras message is not Sset, since the cameras 112-115 in the current layout remain in the client view list for the viewer 3100. At the next step 2509, the processor 3105 starts the change timer associated with the current layout sequence to expire at the time when the layout sequence is to be changed. Following step 2509, the process 2500 concludes.

At step 2511, the processor 3105 starts the pre-emption timer associated with the cutrrent layout sequence to expire at the pre-emption time for the current layout in the current layout sequence. The pre-emption time may be set to thirty (30) seconds before the layout sequence change time.

The remove window from layout process 2600 may be implemented as software resident in the hard disk drive 3110 and being controlled in its execution by the processor 3105. The process 2600 is called when the viewer 3100 needs to remove a single video window 1904) from the currently viewed layout.

The process 2600 begins at step 2601, where the process 2600 calls a "pause layout sequence" process 2900, which stops all timers related to the current layout sequence.

The process 2900 will be described in detail below with reference to Fig. 29. At the next step 2603, the process 2900 sends a reference to the camera 112-115 that is being removed to the storage server 3200. Step 2603 may be implemented by sending a "remove cameras" message to the storage server 3200. The remove cameras message contains a reference to the camera 112-115 that is being removed.

693277.doc -48- Then at the next step 2605, the process 2900 closes the network connection to the Scamera 112-115 that is being removed. The process 2600 concludes at the next step 2607,

U

d where the process 2600 removes the video window 1904) from the layout view 1901.

The add window to layout process 2700 may be implemented as software resident Sin the hard disk drive 3110 and being controlled in its execution by the processor 3105.

SThe process 2700 is called when the viewer 3100 needs to add a single video window 1904) to the currently viewed layout.

The process 2700 begins at step 2701, where the process 2700 calls the pause layout sequence process 2900 to stop all timers related to the current layout sequence. At the next step 2703, the processor 3105 sends a reference to the camera 112-115 that is being added to the storage server 3200. Step 2703 may be implemented by the processor 3105 sending an "add cameras" message to the storage server 3200 containing a reference to the camera 112-115 that is being added. Then at the next step 2705, the process 2700 opens a network connection to the camera 112-115 that is being added and begins receiving streamed video image data. The process 2700 concludes at the next step 2707, where the process 2700 adds the video window to the layout view 1901.

The trigger recording process 2800 is called when the viewer 3100 needs to trigger a recording on the storage server 3200. The process 2800 may be implemented as software resident on the hard disk drive 3110 and being controlled in its execution by the processor 3105.

The process 2800 calls the pause layout sequence process 2900 to stop all timers related to the current layout sequence. Then at the next step 2803, the process 2800 sends a time corresponding to the time associated with the trigger recording user input event to the storage server 3200. Step 2803 may be implemented by the processor 3105 to the 693277.doc -49- "trigger recording" message to the storage server 3200, containing a time corresponding Sto the time associated with the trigger recording user input event.

The pause layout process 2900 is called when a layout sequence needs to be paused.

(A layout sequence may need to be paused in response to user input such as adding or removing cameras 112-115 in the currently selected layout. The process 2900 may be Simplemented as software resident in the hard disk drive 3110 and being controlled in its Sexecution by the processor 3105.

(Ni SThe process 2900 begins at step 2901, where the process 2900 stops the layout sequence pre-emption timer, if one is currently running. The process 2900 concludes at the next step 2903, where the process 2900 stops the layout sequence change timer, if one is currently running.

The aforementioned preferred method(s) comprise a particular control flow. There are many other variants of the preferred method(s) which use different control flows without departing the spirit or scope of the invention. Furthermore one or more of the steps of the preferred method(s) may be performed in parallel rather sequentially.

Industrial Applicability It is apparent from the above that the arrangements described are applicable to the computer and data processing industries.

The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive.

In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including", and not "consisting only of'. Variations of the word "comprising", such as "comprise" and "comprises" have correspondingly varied meanings.

693277.doc

Claims (4)

1. A system for storing video image data, said system comprising: one or more video cameras for capturing video image data; display means for displaying the video image data captured by at least a sub-set of said video cameras, the video image data being displayed according to one or more 0 layouts, each of said layouts referencing at least one of the sub-sets of said video cameras; and buffer means for temporarily storing therein the video image data captured by the sub-set of video cameras referenced by the layout currently being used by the display means to display the video image data, wherein at least a portion of the video image data stored in the buffer means is further stored in a non-volatile storage means depending on a recording configuration of the current sub-set of video cameras.

2. The system according to claim 1, wherein said portion of video image data is copied to said non-volatile storage means in response to an event.

3. The system according to claim 2, wherein said event is a message received by said buffer means from said display means.

4. The system according to claim 3, wherein said message is sent by said display means in response to user input. The system according to claim 1, wherein said buffer means further stores therein video image data captured by one or more of the video cameras referenced by a subsequent layout to the layout currently being used by said display means.

693277.doc -51 00 S6. The system according to claim 5, wherein said subsequent layout is scheduled to be Sused within a predetermined period. \O 7. The system according to claim 1, wherein the current layout identifies the sub-set of video cameras currently being used to display video image data. 8. The system according to claim 1, wherein the current layout defines the position of video image data windows currently being used by said display means to display the video image data. 9. The system according to claim 1, wherein the video image data stored within said buffer means is deleted after a predetermined period of time. 10. The system according to claim 1, wherein the video image data stored within said buffer means is deleted depending on the amount of video image data stored within said buffer means. 11. The system according to claim 1, wherein said buffer means is a circular buffer. 12. A method for storing video image data, said method comprising the steps of: capturing the video image data using one or more video cameras; displaying the video image data captured by at least a sub-set of said video cameras, the video image data being displayed according to one or more layouts, each of said layouts referencing at least one of the sub-sets of said video cameras; and 693277.doc -52- 00 0 storing temporarily in a buffer the video image data captured by the sub-set of video N, cameras referenced by the layout currently being used to display the video image data, Swherein at least a portion of the video image data stored in the buffer is further stored in a non-volatile memory depending on a recording configuration of the current sub-set of video cameras. 13. The system according to claim 12, further comprising the step of copying said portion of the video image data to said non-volatile memory in response to an event. 14. The system according to claim 13, wherein said event is a message received by said buffer. The system according to claim 14, further comprising the step of sending said message in response to user input. 16. The system according to claim 12, further comprising the step of storing, in the buffer, further video image data captured by one or more of the video cameras referenced by a subsequent layout to the layout currently being used to display video image data. 17. The system according to claim 16, wherein said subsequent layout is scheduled to be used within a predetermined period. 18. The system according to claim 12, wherein the current layout identifies the sub-set of video cameras currently being used to display video image data. 693277.doc -53- 00 0 19. The system according to claim 12, wherein the current layout defines the position of N, video image data windows currently being used to display the video image data. IND The system according to claim 12, further comprising the step of deleting the video image data stored in said buffer after a predetermined period of time. 21. The system according to claim 12, wherein the video image data stored within said buffer is deleted depending on the amount of video image data stored within said buffer. 22. The system according to claim 12, wherein said buffer is a circular buffer. 23. An apparatus for storing video image data, said apparatus comprising: one or more video cameras for capturing video image data; display means for displaying the video image data captured by at least a sub-set of said video cameras, the video image data being displayed according to one or more layouts, each of said layouts referencing at least one of the sub-sets of said video cameras; and buffer means for temporarily storing therein the video image data captured by the sub-set of video cameras referenced by the layout currently being used by the display means to display the video image data, wherein at least a portion of the video image data stored in the buffer means is further stored in a non-volatile storage means depending on a recording configuration of the current sub-set of video cameras. 24. A computer program for storing video image data, said program comprising: code for capturing the video image data using one or more video cameras; 693277.doc -54- 00 code for displaying the video image data captured by at least a sub-set of said video cameras, the video image data being displayed according to one or more layouts, each of Ssaid layouts referencing at least one of the sub-sets of said video cameras; and code for storing temporarily in a buffer the video image data captured by the sub-set of video cameras referenced by the layout currently being used to display the video image data, wherein at least a portion of the video image data stored in the buffer is further stored in a non-volatile memory depending on a recording configuration of the current sub-set of video cameras. 25. A system for storing video image data, said system being substantially as herein before described with reference to any one of the embodiments as that embodiment is shown in the accompanying drawings. 26. A method for storing video image data, said method being substantially as herein before described with reference to any one of the embodiments as that embodiment is shown in the accompanying drawings. 27. An apparatus for storing video image data, said apparatus being substantially as herein before described with reference to any one of the embodiments as that embodiment is shown in the accompanying drawings. 693277.doc 00 O c 28. An computer program for storing video image data, said program being Ssubstantially as herein before described with reference to any one of the embodiments as Sthat embodiment is shown in the accompanying drawings. DATED this fifth Day of February 2008 CANON KABUSHIKI KAISHA Patent Attorneys for the Applicant SPRUSON&FERGUSON 693277.doc