WO2006028368A1

WO2006028368A1 - Retail surveillance system and method

Info

Publication number: WO2006028368A1
Application number: PCT/NL2005/000647
Authority: WO
Inventors: Mark Bloemendaal
Original assignee: Ultrawaves Design Holding B.V.
Priority date: 2004-09-07
Filing date: 2005-09-07
Publication date: 2006-03-16

Abstract

Retail surveillance system for monitoring surveillance sites using surveillance guards, the retail surveillance system comprising a retail server (10). Each surveillance site comprises a shop unit (20) connectable to the retail server (10). Each surveillance guard is equipped with a guard unit (30) connectable to the retail server (10). The retail server (10) communicates data to and from the shop units (20) using a first data link (1), and to and from the guard units (30) using a third data link (3). The retail server (10) is equipped to communicate, after detection of an event, warning data associated with the event to the guard units (30) using the third data link (3), and to allow the shop unit (20) to broadcast video data relating to the event using the second data link (2) between the shop unit (20) and the guard units (30).

Description

Retail surveillance system and method

Field of the invention

The present invention relates to retail surveillance system for monitoring a plurality of surveillance sites using a plurality of surveillance guards.

Prior art

Such a retail surveillance system is known from the prior art, in which shops are monitored using closed circuit television circuits, hi general, the retail surveillance system comprises a central monitoring room, where video signals from various cameras are being monitored and stored.

Summary of the invention

The present invention seeks to provide a retail surveillance system which is easy to set up, and provides video data not only to a central monitoring room, but also to mobile surveillance guards.

According to the present invention, a retail surveillance system according to the preamble defined above is provided, in which the retail surveillance system comprises a retail server, and each surveillance site comprises at least one shop unit connectable to the retail server computer (via a wired or wireless connection). The shop unit comprises one or more surveillance cameras, local processing means connected to the one or more surveillance cameras for processing the camera images, and local storage means connected to the local processing means for storing the processed images. Each surveillance guard is equipped with a guard unit, e.g. in the form of a tablet computer, PDA computer or portable computer, connectable to the retail server computer (e.g. using a wireless connection such as UMTS, WLL, etc.), and comprises a display. The retail server comprises first interface means for cornmunicating data to and from each of the shop units using a first data link, second interface means for communicating data to and from the guard units using a third data link, and processing means connected to the first interface means and second interface means. The processing means are arranged for, after detection of an event, communicating warning data associated with the event to each of the guard units using the third data.link (and logging the event), and allowing the shop unit to broadcast video data relating to the event ( e.g. at 1 frame per second) using the second data link between the shop unit and the guard units. Also, a third data link may be provided between the retail server and the guard units, which may be used as a backup facility or to provide coverage outside the second data link coverage area. In an embodiment of the present retail surveillance system, the first data link, second data link and/or third data link comprises a radio data communication link, such as provided by a wireless local link network, a wireless local area network (WLAN), a UMTS network. Due to the wireless nature, each surveillance guard is able to get information on the event (such as localization data and video data) while remaining mobile. Also, the set up of the total system using wireless techniques is very easy abd flexible. Furthermore, it allows to use bandwidth on demand enabling e.g. least cost data transport and graceful degradation of the retail surveillance system.

The shop unit, in a further embodiment, is arranged to store a predetermined amount of video data (e.g. at the full resolution and full frame rate of the video data sources), and the processing means are arranged for sending a request for video data from a shop unit after detection of an event, in which the request comprises an indication of the time period and frame rate of video data to be broadcast. This allows a surveillance guard e.g. to view the store video data from a period before detection of the event up to the moment of the event. This will allow the surveillance guard to better assess the situation, and to gather information regarding the event, such as suspect descriptions.

In a further aspect, the present invention relates to a shop unit for use in a retail surveillance system according to the present invention. The shop unit comprises at least one camera for capturing video data, local processing means provided with local storage means for processing and storing the video data and interface means connected to the local processing means for data communication with the retail server of the retail surveillance system.

In a further embodiment, the shop unit further comprises connectors connected to the local processing means for connecting additional devices in the shop or shop department, such as a panic button, a theft detection system, a microphone, etc. In an even further embodiment, the shop further comprises output means connected to the local processing means. E.g., a speaker may be connected which may allow a message to be played back, or which may be used for a speech connection with one of the surveillance guards. Also, a video monitor may be connected, which allows e.g. to play commercials in a normal situation, and selected video data in a situation in which a certain event has been detected, e.g. to allow shop personnel to view video of a suspected person.

Short description of drawings

The present invention will be discussed in more detail below, using a number of exemplary embodiments, with reference to the attached drawings, in which

Fig. 1 shows a general overview of a retail surveillance system according to the present invention;

Fig. 2 shows a schematic view of a shop unit according to an embodiment of the present invention as used in the retail surveillance system;

Fig. 3 shows a schematic view of a retail server according to an embodiment of the present invention as used in the retail surveillance system; Fig. 4 shows a schematic view of a guard unit according to an embodiment of the present invention as used in the retail surveillance system; and

Fig. 5 shows a momentary view of a display of the guard unit of Fig. 4.

Detailed description of exemplary embodiments

According to an embodiment of the present invention, a retail surveillance system is described below. The retail security concept aims to provide shopping areas (mall, streets, city centers) with a blanket of effective security. Key innovative aspects are: a wireless communication backbone with the connected shops, wireless communication to mobile guards and intelligence. Figure 1 depicts the overall system overview, and shows a retail server 10, a number of shop units 20 and a number of guard units 30. The retail server 10 is able to communicate with the shop units 20 using a first data communication link 1, and with the guard units using a third data communication link 3. The guard units 30 may receive data from the shop units 20 in a further embodiment, using a second data communication link 2.

A shop in a retail area may be equipped with a shop unit 20, the shop unit e.g. being fitted with three cameras 21 in order to cover the whole shop area. The shop unit 20 includes capture and digital recording of video and audio, links with other alarm equipment, intelligence plus a data communication radio link (see description of Fig. 2 below).

A first communication link 1, provided e.g. as a first radio coverage area including the shop fronts, enables video and audio streaming from/to the shops into an associated first radio network, e.g. using a (secure) connection between the first radio network and a WLL network (wireless local loop). From the WLL network a communication link to any place world-wide through (encrypted) internet communication is possible.

A second communication link 2, e.g. provided as a second radio communication network, enables communication from/to the front shop area for video/audio communication to the guard units 30, e.g. in the form of tablet PCs of the mobile guards.

A third communication link 3 is optionally provided as a third radio communication network to enable communication directly from the retail server 10 to the guard unit 30. This network is intended for coverage outside the second communication network, or as a back up facility.

Fig. 2 shows a schematic diagram of one of the shop units 20. Such a shop unit 20 is provided for each shop or shop department. The shop unit comprises local processing means 22, such as a microprocessor or the like, for processing data and controlling the shop unit 20. The local processor 22 is connected to one or more video cameras 21 (e.g. three surveillance cameras), e.g. using a conventional coaxial cable connection, which cover the shop guarded area. Additionally, other external input sources 26 may be provided and connected to the local processor 22, such as a microphone for audio capture, a panic button for alarm indication by personnel, or regular shop anti theft systems. Also, the local processor 22 may be connected to external output devices, such as a loudspeaker 28 for audio talk back or background music, and a video monitor 29 for live video or prerecorded video display. Furthermore, the shop unit 20 comprises an interface 24 and an external antenna 25 for wireless communication to the shop area radio network, i.e. the first or second communication link 1 , 2 as depicted in Fig. 1.

In Fig. 3, a schematic diagram is shown of the retail server 10, which can be regarded as central control part of the present shop surveillance system. The retail server 10 is a computer system, provided with processing means 15, such as a processor 15 or the like, connected to a storage device 16, such as a hard disc recording device. The processor 15 is connected to a first interface 17, which in turn is connected to a first antenna 18 for communicating with the shop units 20 using the first communication link 1. Furthermore, the processor 15 is connected to a second interface 19, which in turn is connected to a second antenna 9 for communicating with the guard units 30 using the second communication link 2. The processor 15 is furthermore connected to an external interface 12, which allows communication with any remote facility using appropriate communication networks, such as virtual private networks (VPN), the internet (using secure protocols), or a backbone data network of a service provider. As shown in Fig. 3, the processor is also connected to a number of converters 8, which allow to convert the digital video data as stored or received to be shown on monitors 11, e.g. in a central monitoring room.

In Fig. 4, a schematic diagram is shown of an embodiment of a guard unit 30. As mentioned before, the guard unit 30 may be implemented using a tablet PC, or alternatively as a notebook PC or PDA computer, which all are easily portable by the guards monitoring the retail area. Each guard unit 30 comprises a guard unit processor

31 connected to a local storage device 32, such as a hard disc or semiconductor mass memory. Furthermore, the guard unit 30 comprises a display 33 connected to the processor 31, for showing video data and various other control data of the present surveillance system. Preferably, the display 33 is of the touch sensitive type, allowing the guard to easily input data and commands on the display 33 itself. Alternatively, other input means, such as a keyboard or mouse may be provided. For communication with the shop unit 20 over the second communication link 2, and for communication with the retail server 10 using the third communication link, an appropriate interface 34 and antenna 35 are present, which are connected to the guard unit processor 31.

As mentioned above, it is possible for the shop unit 20 and the guard unit 30 to be provided with a single interface and antenna (24, 25; 34, 35) for communicating via different communication links. It is also possible that the shop unit 20 and guard unit 30 are provided with different interfaces and associated antennae for each of the communication links 1, 2, 3. Also, the retail server 10 may be provided with a single interface and antenna as an alternative to the pair of interfaces 17, 19 and associated antennae 18, 9. In a steady state situation, i.e. no detection of any event that requires action of one of the guards, the shop unit 20 provides a number of functions. Primarily, the local processor 22 is arranged to process the video data from the cameras 21, e.g. using digitizing techniques for each camera 21 at 50 frames per second. Also, the local processor 22 is arranged to store a predetermined time period, e.g. 96 hours, of each camera in a local storage 23, such as a hard disc recording device. Also one of the primary functions of the shop unit 20, is to monitor the external inputs 26. When an event is detected, such as the operation of a panic button 26 by personnel, the shop unit 20 changes to an event detected state, which is further explained below. Secondary functions of the shop unit 20 include, but are not limited to play prerecorded video or audio data on the speaker 28 and/or monitor 29, such as advertisements. Also, when a microphone is one of the external input sources 26, the sound signal may be digitized, recorded and/or transmitted. Further enhancements may be included in the local processor 22 of the shop unit 20, such as image analysis relating to surveillance of a shop (detect suspicious behavior of persons, intrusion detection, ...).

The shop unit 20 is an autonomous system with three primary functions in steady state, i.e. capture and digital recording of video and audio signals, listening to external event triggers and communicating status information with the retail server 10. In addition the shop unit 20 can be enabled with two advanced functions.

Video/audio intelligence can offer triggering on the captured video and audio streams. Examples are counting and profiling customer traffic, detection presence of persons at night, glass breaking detection and guarding of specific objects. Furthermore, pre¬ recorded video and audio playback is a possible function of the present shop unit 20. In steady state the shop unit 20 functions autonomously with only an 'ambient' communication (wireless) with the retail area network, i.e. the first communication link 1 in Fig. 1. When an event occurs, for example pushing of the alarm button, or sensor triggering, the shop unit 20 immediately switches into event status mode.

In the event status mode, additional functions are performed by the local processor 22. Apart from digitizing and storing video data from the cameras 21, event trigger information from the external inputs 26 is processed and stored. Live video from the cameras 21 is processed and sent to the first and second communication link 1, 2, i.e. to the central retail server 10 and to the guard units 30. Also, when a microphone 26 and speaker 28 are present, it is possible to provide a live audio communication between the shop unit 20 and the retail server 10 using the first communication link 1, and between the shop unit 20 and the guard units 30 using the second communication link 2.

Key differences between the steady state mode and the event status mode is that the shop unit 20 signals the event status into the Retail network (first, second and third communication links 1, 2, 3 in Fig. 1) which is relayed immediately to the guard units 30 of the guards on duty as well as e.g. a remote control center. Live video and audio is streamed through the same network to the guard unit 30 and the retail server 10. Video and audio recordings in the shop unit 20 (including a programmable interval before the event occurred) are labeled to be maintained for future review. Video frame recording is focused towards cameras related to the event. An audio channel for a talk back channel can be opened between the guard unit 30 and/or retail server 10 to the audio speaker 28. When configured/wanted the shop unit 20 may generate audio and video alarm signals on the monitor 29 and the loudspeaker 28 for deterrence.

The shop unit 20 is thus capable of the functions of video grabbing and digitally recording, motion detection, I/O sensing, alarm interpretation, video to screen streaming, alarm event generation, generating an event report to the retail server 10, and live and buffered video streaming to the retail server 10. The default application of video display sequencer on a monitor is also provided. Alarm event video streams are immediately (while active) passed on to the retail server 10 enabling multiple clients (e.g. the guard units 30) to request streaming data from the retail server 10 about this alarm event. This mechanism allows multiple guard units 30 (or other clients) to receive data simultaneously without overloading the shop unit 20. The retail server 10 therefore also acts as an archive for all past events from all associated shop units 20.

The retail server 10, or better its processor 15 is arranged for the functions of Live View GUI execution (using e.g. monitors 11), alarm event report logging (includes event report archiving and incoming video event streams archiving), alarm event polling from guard units 30, video stream feeding from alarm video event database and live incoming streams (broadcast server), and external communication, such as alarm event pass-through. The guard unit 30, or its guard unit processor 31, is arranged to perform the functions of retail event GUI execution, alarm event list retrieval from the retail server

10, polling (e.g. Ix per sec) for new alarm events from the retail server 10, and receiving alarm event information and video retrieval from the retail server 10. In relation to the possible information shown on the display 33, the guard unit 30 is arranged to allow standard multi-screen viewing of the video in the retail area(s), viewing of the different retail areas, viewing of the different shops, viewing of outdoor cameras, and to manually select cameras and select for live viewing. This may be accomplished using a list box which shows in a selectable manner, the retail areas, shops, cameras 21 in shop, outdoor areas, and cameras 21 in outdoor areas

As a result the guards present at the retail area are both signaled about the type and place of an event and are provided immediately with good quality live video and audio of the scene of the event. As a consequence response can be quickly and adapted to the situation at hand. The retail network infrastructure can be scaled to cater for the simultaneously handling of multiple events (current plans cater for four events). Since during steady state the shop units 20 operate autonomously the retail area can cover an unlimited number of participating shops.

The above may be further clarified with reference to Fig. 5, which schematically shows a display screen 33 of a guard unit 30 during an event status mode. The Event list on the left shows the past shop or area/camera where something happened and the nature of the event. In a passive or steady state mode the guard unit 30 will wait for the next event, only system integrity information is shown plus the past list of events. When in passive mode and when a new event comes in, the guard unit 30 switches to active mode (or event status mode) and automatically selects this last incoming event and sounds an audio alarm. The guard unit 30 also switches to active mode when an event is selected by the operator from the list. When a new event comes in, the video/event window is 'focused' on this incoming event with video and status and background information. Video data shown can be live (e.g. at a rate >=1 frame /sec) or replayed from the event start. Through 'I respond' the retail server 10 (and other guard units 30 and possibly shop units 20) is made aware that this terminal owner will follow- up the event. Through the 'report', information can be filed which is communicated in the network and stored in the event database of the retail server 10. Any other guard unit's view on this event is updated automatically (event database on retail server 10). After entering 'Event done¹, the guard unit 20 (and retail server 10) switches back to passive mode. In a practical implementation of the present retail surveillance system, a "mini shopping center" which is used to train guarding staff and staff of shops has been set up. The area includes a shopping street, a square and a parking lot. In the shopping street a small desk for a guard is foreseen. On the square a second guard desk is available. The square connects to the outside parking lot. Three shops were fitted with a shop unit 20, each comprising:

♦ 4 analog video inputs,

♦ 4 analog fixed dome cameras wired connected to shop unit

♦ 25 frames/sec max for a single cameras (adaptive), total 50 frames/sec

♦ local storage 80 Gbyte (3 mbit/sec total storage = +-48 hrs straight ) ♦ Single monitor : 4 cam split screen 2 frames/sec update

♦ Mono loudspeaker for talk-back

♦ Mono microphone (active on demand) remote listening

♦ 8 Kliz recording of audio on demand/event

♦ alarm sense switch ♦ remote web browser GUI on a PC

Furthermore, a live viewing station is provided with:

♦ 12 overview/split screen slow updating (1 fps)

♦ 4 live view analog monitors (10 fps), selectable from 12 channels.

♦ Single audio listening ♦ Single audio talk back

♦ Central local storage on demand when object selected

♦ Retrieval from shop unit in past

The (mobile) guard in the test set up is provided with a tablet PC or PDA computer, although it is also feasible to use a notebook based computer. The total system is arranged for:

♦ On event relay of shop video : 4 streams / 2-4 Mbit/sec

♦ Handover to different cells ♦ Audio listening

♦ Audio talk back

Also, a remote observation capability is provided based on remote access to live viewing station (using either xDSL or WLL networks, which has the functionality of a mobile guard, and allows Web (https) based 'live' viewing over Internet.

In a further test set up, an audio firmware upgrade has been provided, and also, the shop units 20 (hard- and software) have been modified to include an intelligent video processor enabling the addition of (hardware accelerated) video intelligence features and 8 frames/sec average video rate. Also, test algorithms for object detection and analysis have been installed.

Claims

1. Retail surveillance system for monitoring a plurality of surveillance sites using a plurality of surveillance guards, in which the retail surveillance system comprises a retail server, and. in which each surveillance site comprises at least one shop unit connectable to the retail server, the shop unit comprising one or more surveillance cameras, local processing means connected to the one or more surveillance cameras for processing the camera images, and local storage means connected to the local processing means for storing the processed images, in which each surveillance guard is equipped with a guard unit connectable to the retail server, the guard unit comprising a display, the retail server comprising first interface means for communicating data to and from each of the shop units using a first data link, second interface means for communicating data to and from the guard units using a third data link, and processing means connected to the first interface means and second interface means, the processing means being arranged for: after detection of an event, communicating warning data associated with the event to each of the guard units using the third data link; allowing the shop unit to broadcast video data relating to the event using the second data link between the shop unit and the guard units.

2. Retail surveillance system according to claim 1, in which the first data link, second data link and/or third data link comprises a radio data communication link, such as provided by a wireless local link network, a wireless local area network (WLAN), a UMTS network.

3. Retail surveillance system according to claim 1 or 2, in which the shop unit is arranged to store a predetermined amount of video data, and the processing means are arranged for sending a request for video data from a shop unit after detection of an event, in which the request comprises an indication of the time period and frame rate of video data to be broadcast.

4. Shop unit for use in a retail surveillance system according to any one of claims 1 to 3, in which the shop unit comprises at least one camera for capturing video data; local processing means provided with local storage means for processing and storing the video data; interface means connected to the local processing means for data communication with the retail server of the retail surveillance system.

5. Shop unit according to claim 4, further comprising connectors connected to the local processing means for connecting additional devices.

6. Shop unit according to claim 4 or 5, further comprising output means connected to the local processing means.