WO2012030205A1

WO2012030205A1 - System and method for modular intelligent surveillance

Info

Publication number: WO2012030205A1
Application number: PCT/MY2010/000284
Authority: WO
Inventors: Hock Woon Hon; Saw Meng Soo; Yen San Yong; Sing Wai Yuen
Original assignee: Mimos Berhad
Priority date: 2010-08-30
Filing date: 2010-11-22
Publication date: 2012-03-08
Also published as: MY168870A

Abstract

The present invention relates generally to a system and method for modular surveillance system wherein the capture and display module (4) is separated from the processing module (6).

Description

SYSTEM AND METHOD FOR MODULAR INTELLIGENT SURVEILLANCE . TECHNICAL FIELD OF THE INVENTION

The present invention relates generally to a system and method for modular surveillance wherein the capture and display module is separated from the processing module. . BACKGROUND OF THE INVENTION

Intelligent surveillance technology has become one of the vital components in security infrastructure. It has become indispensable where the system architecture assumes a high level of importance. Intelligent surveillance system typically consists of the capturing, processing and displaying modules all integrated as one. Different processes in each module may be written using different programming languages to optimise its performance. For instance, the capture and display module requires powerful graphics display and user friendly graphical user interface design while the processing module requires application of programming language. Problem arises when the modules need to communicate with each other as the modules are written in different programming languages. Also from time to time researchers may develop new and more sophisticated processing components. Upgrading the whole system is not only expensive but time consuming as it involves recoding the whole system.

It would hence be extremely advantageous if the above shortcoming is alleviated by separating the processing module from the main framework of the surveillance system, that is the capture and display module so that the user can easily configure and implement newly developed processing components without having to recode the system and the user can choose to turn on and off the type of processing components required for a required task or action by merely modifying the system configuration file. Communication problems between the processing module and the capture and display module typically encountered in conventional surveillance systems is overcome by utilising an interface layer in this two-part surveillance system. SUMMARY OF THE INVENTION

Accordingly, it is the primary aim of the present invention to provide a system and method for Modular Intelligence Surveillance which is separated into two-parts namely the processing module and the capture and display module which forms the main framework of the intelligent surveillance system to provide versatility. It is yet another object of the present invention to provide a system and method for Modular Intelligence Surveillance wherein the user can easily configure the manner of implementing newly developed processing components into the surveillance system which then becomes integrated to the intelligent surveillance system without having to recode the system.

It is a further object of the present invention to provide a system and method for Modular Intelligence Surveillance wherein the user can choose to turn on and off the processing component by merely modifying the system configuration file thereby allowing a plug and play concept to operate.

Yet another object of the present invention is to provide a system and method for Modular Intelligence Surveillance wherein communication between the modules in the said system is made possible and facilitated by means of an interface layer.

Yet a further object of the present invention is to provide a system and method for Modular Intelligence Surveillance which allows the processing components to be written in programming languages different from the capture and display module. Other and further objects of the invention will become apparent with an understanding of the following detailed description of the invention or upon employment of the invention in practice.

According to a preferred embodiment of the present invention there is provided,

A Modular Intelligence Surveillance System comprising; at least a capture and display module (4); at least a processing module (6); at least a communication server (8); characterised in that the said processing module (6) comprising at least a processing component (7) to process data and detect events is only implemented and integrated to the said system as and when required.

In another aspect there is provided, A method of communicating between the capture and display module (4) and the processing module (6) of the Modular Intelligence Surveillance System comprising steps of, establishing a relationship between the capture and display module (CDM) (4) and at least one processing component (7) in the processing module

(6); sending at least an Initialize Request together with required information wrapped with a defined interface layer (10) from CDM to at least one processing component (7) in the processing module (6) via communication server (8); initializing at least one processing component (7) in the processing module (6); capturing and extracting raw image data using the CDM; sending Execute Request together with the required information wrapped with the said interface layer (10) from the CDM to at least one processing component (7) via communication server (8); processing the raw image data using at least one processing component sending acknowledgement together with processed information wrapped with the said interface layer (10) from the processing module (6) to the said CDM (4) via communication server (8); sending Terminate Request together with required information wrapped with the said interface layer (10) from the CDM (4) to at least one processing component (7) of the processing module (6) if the user decides to abandon application or an error occurs.

4. BRIEF DESCRIPTION OF THE DRAWINGS

Other aspect of the present invention and their advantages will be discerned after studying the Detailed Description in conjunction with the accompanying drawings in which:

FIG. 1 shows the modular intelligent surveillance system architecture of the present invention.

FIG. 2 shows the data flow between the capture and display module and a component in the processing module such as an analytic engine in the preferred embodiment of this invention. FIG. 3 shows the data flow between the capture and display module and a component in the said analytic engine in another embodiment of the present invention. DETAILED DESCRIPTION OF THE DRAWINGS

In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those or ordinary skill in the art that the invention may be practised without these specific details. In other instances, well known methods, procedures and/ or components have not been described in detail so as not to obscure the invention.

The invention will be more clearly understood from the following description of the embodiments thereof, given by way of example only with reference to the accompanying drawings which are not drawn to scale.

Referring to Figure 1, there is shown the modular intelligent surveillance system architecture of the present invention consisting of two parts namely the capture and display module [CDM] (4) and the processing module (6) preferably an analytic engine [AE]. For purposes of this document processing module (6) includes AE and vice versa. The CDM (4) is responsible for capturing data including video streams, image sequence and audio streams through the IP (Internet Protocol) network from sensing devices preferably cameras to display on the monitor or play with speaker. For purposes of this document sensing devices include cameras and vice versa. A comprehensive surveillance system would employ various technological and process components to achieve its objective of providing enhanced surveillance and security. Therefore the AE (6) consists of many smaller processing components developed to perform processing tasks including image, video or audio processing such as transformation, event detection and etcetera. As the many smaller processing components of the AE (6) in this present invention is operable based on the concept of plug and play, these smaller processing components shall be hereinafter referred to as "Plug and Play" components [PnP components] (7). The PnP components (7) allow versatility as the user can select any type of component in the AE (6) based on the processing task or result required. This present invention uses the concept of server-client. The communication server (8) is just a communication centre to synchronise and communicate with each client. The communication server (8) can be on a single piece of central processing unit (CPU) or multiple CPUs. The client could be the CDM (4) or any PnP component (7) in the AE (6) as illustrated in Figure 1. In addition to this architecture the PnP component (7) can be written in any programming language even if different from the CDM (4) thereby providing more freedom for a developer to code and optimise their algorithm. This freedom is enabled by the use of an interface layer (10) which facilitates communication between the CDM (4) and the AE (6) in particular the PnP components (7) and vice versa although both may be written in different programming languages. This CDM-AE interface layer (10) is a defined interface and advantageously comprises the following which is not exhaustive:-

(i) original data information (for example data source pointer and data properties or data storage location);

(ii) processed information (for example data source pointer and data properties or data location);

(iii) segmented data information (for example audio segment pointer and properties, snapshot image pointer and properties, data storage location);

(iv) event information (event ID, event location such as bounding box, centroid);

(v) status information (successful or error ID with error message);

(vi) control information (PTZ control signal)

Figure 2 shows the data flow between CDM (4) and a PNP component (7) in the AE (6). There are three types of request involved in the CDM (4) namely to initialize, execute and terminate. Each request which is independently sent from the CDM is a command requiring the AE (6) to perform a different task which may include the following:-

- initializing the PnP component (7) once, executing the PnP component (7) once or more times and terminating the PnP component (7) once; or in cases where an error has occurred during the initialization request that does not allow the Initialize Request to proceed to the PnP Component execution stage (20) in the AE (6), initializing the PnP component (7) once and terminating the PnP component (7) once; or in cases where an error has occurred at the execution request or in the middle of the execution stage (14), initializing the PnP component (7) once, executing the PnP component (7) once or more times and terminating the PnP component (7) once.

The request issued by the CDM (4) after undergoing the initialization stage (12), the execution stage (14) or the termination stage (16) to initiate, execute or terminate respectively are the commands requesting the AE to perform a particular task. In the initialization stage (12), the CDM (4) reads the configuration file in order to set up a relationship between at least one PnP component (7) of the AE (6) and the CDM (4) as indicated in the first CDM initialization step (12A). Then it gathers the required information for initialization and wraps the said information with a defined interface layer (10) as indicated in the second CDM initialization step (12B). Next, it invokes one of the PnP components (7), the required PnP Component A (7A) in the AE (6) by sending a request for initialization ("Initialization Request") and the information to the required PnP Component A (7A) via the communication server (8) as indicated in the third CDM initialization step (12C) and awaits acknowledgement from the required PnP Component A (7A).

Upon the PnP Component A (7 A) receiving the Initialization Request from the CDM (4), the PnP Component A (7A) will proceed to verify whether it is an Initialization Request or not. If the answer is in the affirmative then the PnP Component A initialization stage (18) is activated. The PnP Component A (7A) will proceed to extract the relevant information from the defined interface layer (10) as illustrated in the first PnP Component A initialization step (18A) and thereafter perform component initialization as illustrated in the second PnP Component A initialization step (18B). At the end of the PnP Component A initialization stage (18), an acknowledgement to initialize ("Initialize Acknowledgement") feedback is sent to the CDM (4) as illustrated in the third PnP Component A initialization step (18C) via the communication server (8).

If the Initialize Acknowledgement feedback from the PnP Component A (7A) is received and the CDM (4) intends to proceed then the CDM (4) will activate the next stage that is the execution stage (14). In this CDM execution stage (14), the CDM falls into a loop which involves three main steps namely the first CDM execution step (14A), the second CDM execution step (14B) and the third CDM execution step (14C). The first CDM execution step (14A) captures image and extracts raw image data. The second CDM execution step (14B) gathers required information for execution and wraps the information with the defined interface layer (10). The third CDM execution step (14C) will send a request to execute ("Execute Request") to the PnP Component A (7A) and awaits acknowledgement from the said component until the user terminates the application.

If the Execute Request from the CDM (4) is sent to the AE (6) the selected PnP Component A in the AE (6) will verify the type of request that is required by the CDM (4). If it is not an Initialize Request then it will proceed to verify whether it is an Execute Request. If it is an Execute Request the PnP Component A execution stage (20) is activated whereupon the required PnP Component A (7 A) will proceed to extract information from the defined interface layer (10) in the first PnP Component A execution step (20 A). Thereafter processing of raw image data or information will be performed in the second PnP Component A execution step. Next the processed data or information is gathered and wrapped with the defined interface layer (10) in the third PnP Component A execution step (20C). Finally the processed data is sent back to CDM (4) as illustrated in the fourth PnP Component A execution step (20D) via the communication server (8) as the acknowledgement to execute ("Execute Acknowledgement") feedback.

If the Execute Acknowledgement feedback from the PnP Component A (7A) is received and the system decides to proceed further then the whole process of the CDM execution stage (14) and PnP Component A execution stage (20) is repeated. If the system does not intend to proceed in instances where an error occurs which prevents the application from proceeding or the user requests to terminate the application, the CDM termination stage (16) will be activated by the system to send a request to terminate ("Terminate Request") to the activated PnP Component A (7A). If the Termination Request from the CDM (4) is sent to the AE (6) the selected PnP Component A (7A) in the AE (6) will verify the type of request that is required by the CDM (4). If it is not an Initialize Request then it will proceed to verify whether it is an Execute Request. If it is also not an Execute Request but a Termination Request, the activated PnP Component A (7 A) will trigger the PnP Component A termination stage (22) and proceed to terminate the said activated component in the first PnP Component A termination step (22A) and thereafter causes an acknowledgement to terminate ("Terminate Acknowledgement") feedback to be sent to the CDM (4) in the second PnP Component A termination step (22B). The Terminate Acknowledgement feedback is sent to the CDM (4) via the communication server (8). Upon receiving the Terminate Acknowledgement feedback from the activated PnP Component A (7A), the CDM (4) will end the process.

The same process is applicable if other PnP components such as PnP Component B (7B) or PnP Component C (7C) in the AE (6) are selected to be implemented and integrated in the surveillance system. The data flow is similar to that in PnP Component A save for the different technology incorporated in the different PnP components (7). Although only three PnP components (7) are illustrated and described in Figure 2, it is to be understood that any number of PnP components (7) can be plugged in and implemented into the system when required and plugged out when not required as what is advantageous is the versatility afforded by the present invention and the benefit of not having to recode the system each time a new processing component is developed. Alternatively another embodiment of the present invention is provided where the data flow in the CDM initialization stage (12) is provided with an additional step of capturing data including images, videos and audios and extracting raw data or information after a relationship between the CDM (4) and the individual component in the AE (6) had been established. This is illustrated in FIG. 3 which shows the data flow between CDM (4) and a PNP component (7) in the AE (6). The process in this second embodiment are similar to that in Figure 2 save for the initialization stage (12) of this second embodiment being provided with an additional step of capturing data including images, videos and audios and extracting raw data

(12D).

Alternatively after the CDM (4) has received the Execute Acknowledgement from the AE (6), the CDM (4) is programmed to take post- event actions based on the type of event action required such as triggering an alarm, sending notification to a mobile phone, drawing bounding boxes on the display screen and others.

While the preferred embodiment of the present invention and its advantages has been disclosed in the above Detailed Description, the invention is not limited thereto but only by the spirit and scope of the appended claim.

Claims

WHAT IS CLAIMED IS:

1. A modular Intelligence Surveillance System comprising; at least a capture and display module (CDM) (4); at least a processing module (6); at least a communication server (8); characterised in that the said processing module (6) comprising at least a processing component (7) to process data and detect events is only implemented and integrated to the said system as and when required.

2. A modular Intelligence Surveillance System as claimed in Claim 1 wherein an interface layer (10) is provided to the system for communication between the CDM (4) and processing module (6).

3. A modular Intelligence Surveillance System as claimed in Claim

1 wherein the processing module (6) is an analytic engine.

4. A modular Intelligence Surveillance System as claimed in Claim 1 or 3 wherein the processing module (6) comprises at least a PnP component (7) which is operable using the plug and play concept.

5. A modular Intelligence Surveillance System as claimed in Claim 1 or 3 wherein the PnP components (7) can be written in any programming language even if different from the CDM (4).

6. A method of communicating between the CDM (4) and the processing module (6) in the modular Intelligence Surveillance System comprising steps of, establishing a relationship between the CDM (4) and at least one processing component (7) in the processing module (6); sending at least an Initialize Request together with required information wrapped with a defined interface layer (10) to at least one processing component (7) in the processing module (6) via communication server (8); initializing at least one processing component (7) in the processing module (6); capturing and extracting raw data using the CDM (4); sending Execute Request together with the required information wrapped with the said interface layer (10) from the CDM (4) to at least one processing component (7) via communication server (8); processing the raw image data using at least one processing component

(7); sending acknowledgement together with processed information wrapped with the said interface layer (10) from the processing module (6) to the said CDM (4) via communication server (8); sending Terminate Request together with required information wrapped with the said interface layer (10) from the CDM (4) to at least one processing component (7) of the processing module (6) if the user decides to abandon application or an error occurs.