AU2004306552A1

AU2004306552A1 - Augmented video surveillance system

Info

Publication number: AU2004306552A1
Application number: AU2004306552A
Authority: AU
Inventors: Colin Andrew; Gregory Cox
Original assignee: De Beers Consolidated Mines Ltd
Current assignee: De Beers Consolidated Mines Ltd
Priority date: 2003-10-09
Filing date: 2004-10-11
Publication date: 2005-04-21
Also published as: WO2005036881A3; WO2005036881A2; AP2006003573A0; CA2541880A1; EP1673938A2; IL174857A0

Description

WO 2005/036881 PCT/IB2004/003304 -1 AUGMENTED VIDEO SURVEILLANCE SYSTEM BACKGROUND TO THE INVENTION THIS invention relates to an augmented video surveillance system. Video surveillance or CCTV (closed circuit television) surveillance systems usually make use of a number of video cameras and communication links to transfer camera-captured analogue images to remote video monitors where they can be observed by human operators or to video image recordal equipment which records them for subsequent viewing by human operators. Such systems are entirely reliant upon the operator's ability to discern, from the images viewed, whether a particular event has taken place and whether any particular action is required. Recognised problems with known video surveillance systems, whether for security or other purposes, are boredom and inefficient or improper reaction to specified events by surveillance operators. A further problem is that there are only limited methods for measuring or monitoring the performance of surveillance operators. The present invention seeks to address these problems. SUMMARY OF THE INVENTION According to the invention there is provided an augmented video surveillance system in which a surveillance video is augmented by insertion, into live video images seen by an operator, of synthetic or realistic objects and/or by visual enhancement of predetermined events taking place in the live video image seen by the operator, such insertion requiring specific action by the operator, and monitoring the action taken by the operator in response to such insertion.

WO 2005/036881 PCT/IB2004/003304 -2 The objective of the invention is to improve the performance and effectiveness of the operator and to measure his performance online. In accordance with the invention this object is addressed by engaging the operator's attention by interaction with the live video with the result that the operator's levels of vigilance and interest are elevated. The system proposed by the invention may be used to provide on-line operator training. The invention also provides apparatus for use in the system summarized above. BRIEF DESCRIPTION OF THE DRAWING The invention will now be described in more detail, by way of example only, with reference to the accompanying block diagram which illustrates the principles of the invention. SPECIFIC DESCRIPTION Referring to the diagram, the numeral 10 indicates an event controller which controls and implements video augmentation according to the invention. The event controller 10 can be programmed to augment a live video seen by an operator in a number of different ways which may be implemented alone or in combination with one another. In one version live video events are brought to the attention of the operator. These are actual live events taking place in the scene observed by the operator. Examples of typical events include entry of a person into the observed scene, addition or removal of inanimate objects to the scene. This is indicated by the numeral 11 in the diagram. As described in a simultaneously filed, co-pending patent application filed by the present applicant and entitled "Enhanced Video Based Surveillance WO 2005/036881 PCT/IB2004/003304 -3 System" a background image or model is generated and, during the course of live video monitoring, the overall image is segmented into respective background and foreground segments and video enhancement techniques are employed visually to enhance all or selected foreground features or events, thereby drawing the operator's attention to such features and events. The system described above may be interfaced with a video motion detection system also described in the aforementioned co-pending patent application, in which case the operator's attention is drawn, by camera switching and/or image enhancement to specified motion events in the observed image. This is indicated by the numeral 13 in the diagram. The operator is trained to respond to specified foreground features or events, in each case in a predetermined manner. The system may be used to measure the operator's efficiency in responding in the appropriate manner to specified events. The invention proposes the insertion of certain synthetic or realistic objects or events into the image viewed by the operator. In one example the event controller 10 acts in response to a rules-based event generation module 12 to initiate the generation, by a synthetic object generator 14, of a synthetic event or object and augments the raw, live video image 16 by inserting such event or object into the image, thereby forming an augmented video image 18 which is viewed by the operator. In each case the controller 10 controls all parameters relating to the insertion, e.g. the frequency of insertion, the duration of the insertion and so on. Synthetic objects are objects which are inserted into the viewed image and are classified as synthetic because they do not blend into the imaged scene and can be clearly identified by the operator as artificially inserted objects. Typical examples are fanciful shapes such as stars, squares, circles, still pictures of persons or objects, or animated sequences of person and/or objects. In each case the insertion may require specific action by the WO 2005/036881 PCT/IB2004/003304 -4 operator, as indicated by the numeral 20. The operator may, for example, be required to operate a mouse pointer and to click on the insertion in order to remove it. The operator's reaction time and ability to carry out the required action can be monitored to provide an indication of the operator's efficiency and training and can, if necessary, be logged as part of the operator's performance record. Associated with the insertion of synthetic objects can be certain specific tasks that need to be performed by the operator, such tasks requiring higher levels of cognitive engagement than a simple response of acknowledging the presence of the synthetic object by clicking on it. For example, this could be a discrimination task whereby the operator is required to distinguish between different categories of inserted synthetic objects. For example, still pictures or images of persons may show such persons displaying normal, unsuspicious behaviour and may require a first type of response by the operator. On the other hand, pictures of persons displaying suspicious behaviour may require a totally different type of response by the operator. Once again, the operator's reaction to the inserted object(s) can be used to provide a measure of ability and efficiency, with the accuracy of completion of the associated task providing an additional level of information on operator performance. The object insertion technique described in the preceding paragraph also serves a valuable function in on-line training of the operator in that it can raise the operator's awareness of certain types of personal behaviour which may be considered undesirable from a security point of view, for example a person assuming a certain bodily posture in the vicinity of a high-risk area such as a cash-counting or gem sorting station. Through these types of synthetic objects, operators are sensitised to the types of behaviours/postures . that they should detect, thereby increasing the likelihood of detecting incidents associated with such behaviours/postures when they actually do occur. Often such incidents have a fairly low frequency of occurrence.

WO 2005/036881 PCT/IB2004/003304 -5 The invention also envisages the augmentation of the live video by insertion of realistic objects or events, under the control of the event controller 10 and in accordance with the rules-based event generation module 12. In this case, realistic video objects, such as animated sequences extracted from previously recorded images of the scene under surveillance, as indicated by the block 22 in the diagram, are inserted back into the live video image 16 to produce the augmented image 18 in such a manner that the operator is unable to distinguish between artificially inserted objects and real objects of the scene. So, for instance, live video footage of a person moving in the surveilled scene can be inserted to produce the augmented image. The operator will once again be required to react in a certain way to the augmented video which he sees. Where, for instance, augmentation involves insertion of video footage showing an act of theft, the technique can be used to monitor the operator's ability to recognize the act and to react accordingly. Thus, where performance on inserted synthetic object(s) provides a proxy on operator performance, realistic insertion provide a direct measure of operator performance on completely realistic incidents. Combining both approaches in operator performance monitoring also prevents unrealistic estimates of performance of operator who simply concentrate on elevating their performance on inserted synthetic objects at the expense of detecting real incidents in the surveillance video. In each of the examples described above, the operator will be required to interact, in accordance with his training, via an interface generator 24. Where a synthetic object has been inserted, the operator may for example be required to click once, with a mouse pointer, on an image such as a fanciful image or a realistic image of non-threatening type or to click twice on an image representing a security threat. The event controller may be programmed to give feedback to the operator, via the interface 24, to indicate that the correct action has been taken. The inserted object may for instance be caused to flash and/or to disappear from the image, thereby acknowledging the correctness of the operator's action.

Claims

1. An augmented video surveillance system in which a surveillance video is augmented by insertion, into live video images seen by an operator, of synthetic or realistic objects and/or by visual enhancement of predetermined events taking place in the live video in-age seen by the operator, such insertion requiring specific action by the operator, and monitoring the action taken by the operator in response to such insertion.

2. An augmented video surveillance system according to claim 1 comprising a programmable event . controller for controlling the augmented video image seen by the operator.

3. An augmented video surveillance system accord ing to claim 2 comprising a rules-based event generation module for initiating the generation of an augmented image.

4. An augmented video surveillance system according to any one of claims 1 to 3 comprising a synthetic object generator for generating the synthetic object or event for insertion into the live video image seen by the operator.

5. An augmented video surveillance system according to claim 4 wherein the synthetic object is a fanciful shape or a still picture of a person.

6. An augmented video surveillance system according to claim 5 wherein the fanciful shape is a star, a square or a circle.

7. An augmented video surveillance system according to claim 5 wherein the still picture of a person shows such person displaying normal, unsuspicious behaviour or shows such person displaying suspicious behaviour. WO 2005/036881 PCT/IB2004/003304 -7

8. An augmented video surveillance system according to any one of claims I to 3 comprising real object retrieval whereby the realistic objects or events which are inserted into the augmented video image comprises animated sequences extracted from previously recorded images of the scene under surveillance.

9. An augmented video surveillance system according to claim 8 wherein the realistic event comprises video footage showing an act of theft.

10. An augmented video surveillance system according to any one of the preceding claims comprising an interface generator allowing interaction with the operator.

11. An augmented video surveillance system according to claim 10 wherein the event controller is programmed to give feedback to the operator via the interface generator.