AU768275B2 - Wireless video surveillance system - Google Patents

Description

1/AUUU/VUU3O4 Received 11 May 2001

I

WIRELESS VIDEO SURVEILLANCE SYSTEM This invention relates to an apparatus and method for wireless video surveillance and communication. In particular, it relates to an apparatus and method employing commercially available, hand-held portable devices, such as personal information managers and personal digital assistants.

BACKGROUND TO THE INVENTION The use of personal digital assistants (PDA's), such as the 3COM Palm Pilot', is becoming very widespread. Functionality of PDA's varies between manufacturers but all include a liquid crystal display, an input device, memory and a processor unit. Various improvements are constantly being made to increase available memory, improve processing power and extend software applications.

Recently, PDA manufacturers have released devices having colour liquid crystal displays and wireless communication capability.

This innovation has been driven by a desire to provide PDA's with intemrnet access for web surfing and e-mail. Similar market pressures are driving the integration of processing capability into mobile phones to facilitate direct web access for the same purposes.

It is also known to integrate PDA's with mobile phones to provide an integrated system giving full voice and data transfer facilities. One such system is described in United States patent number 5625673, assigned to Lucent Technologies Inc. The patent describes a PDA that includes means for cordless connection to specialised accessories, such as a cellular telephone and a modem.

The capabilities of PDA's have not been extended for application in the video surveillance area. Existing surveillance systems, such as house security systems, do not normally offer a video surveillance capability. When such a capability is provided, it is usually linked to a base station that provides remote manual surveillance.

AMENDED SHEET

IPEAIAU

WO 01/06791 PCT/AU00/00364 2 The majority of existing surveillance systems utilise a local processing centre that packets data for transmittal to the base station.

Transmittal may be by dedicated land-line or may be via a dial-up connection.

OBJECT OF THE INVENTION It is an object of the invention to provide an apparatus and method for wireless video surveillance.

SUMMARY OF THE INVENTION In one form, although it need not be the only, or indeed the broadest form, the invention resides in a wireless video surveillance system comprising: a portable monitor device and first wireless communication means; one or more video recording means for recording video images of a scene; and at least one interface module converting said video images to transmittable data, said interface module incorporating a second wireless communication means for transmission of said transmittable data from said at least one interface module to said portable monitor device, said portable monitor device incorporating means for receiving said transmittable data, converting said transmittable data to said video images and displaying said video images.

In preference, the portable monitor device is a personal digital assistant or similar hand-held processing unit incorporating processor means, memory means and video display means.

The video recording means may be a digital camera or may be a commercially available analogue video camera, such as a CamcorderO.

Suitably, the interface module includes input means for receiving video signals from said video recording means. A suitable input port is a USB port for digital video input. If an analog video recording means is employed, the input means suitably includes a video input port and Substitute Sheet (Rule 26) RO/AU rL I uIUUV/UUv3O Received 20 February 2001 3 analog to digital conversion means.

The interface module preferably also includes processing means for converting said video images to transmittable data and one or more transmission buffers.

The second wireless communication means is suitably a signal transmitting means but is preferably a signal receiving and transmitting means.

The processing means of the interface module is suitably programmed with video and audio compression algorithms. Corresponding 10 audio and video decompression algorithms are suitably programmed in the processor means of the portable monitor device.

The portable monitor device suitably also includes input means for inputting signals for transmitting to said interface module. Preferably said first wireless communication means includes means for transmitting said signals and said second wireless communication means includes means for receiving said signals.

In a further form, the invention resides in a method of providing wireless video surveillance including the steps of: recording a video image of a scene; processing the recorded image to form data for wireless transmission; transmitting the data to a portable monitor device; and processing the data to display the image on the portable monitor device.

Processing the recorded image preferably includes the steps of compressing the image at the interface means and decompressing the image at the portable monitor device.

The method may further include the step of storing said data for later transmission.

BRIEF DESCRIPTION OF THE DRAWINGS Preferred embodiments of the invention are described with reference to the following figures in which: FIG 1 is a block diagram of a first embodiment of the invention; AMENDED SHEET

VIP~AAU

Received 11 May 2001 4 SFIG 2 is a block diagram of a first embodiment of the interface module; FIG 3 is a block diagram of a second embodiment of the interface module; FIG 4 is a block diagram of a third embodiment of the interface module; FIG 5 is a block diagram of the portable video monitor; FIG 6 is a block diagram of a second embodiment of the invention; and FIG 7 is a block diagram of a third embodiment of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT Referring to FIG 1 there is shown a block diagram of a wireless video surveillance system consisting of a video recording means 1, in signal connection with an interface module 2 which is in signal connection with an antenna 3. Signals 4 are transmitted between the antenna 3 and an antenna 5 associated with a portable monitor device 6 which is a personal digital assistant or portable personal computer.

Video images from the recorder 1 are displayed on a display 7 of the monitor 6.

Video and audio signals are transmitted from the interface module to the portable monitor device and control signals may be transmitted from the portable monitor device to the interface module.

The video recording means 1 is a commercially available VHS video recorder having standard video and audio outputs. Other video formats may include PAL, NTSC, SVHS, BETACAM, RGB, SECAM and DVD.

A first preferred embodiment of the interface module 2 is shown in greater detail in FIG 2. In this embodiment the interface module is configured to transmit signals to the portable monitor device but is not configured to receive signals. Video and audio input from the recorder 1 is input through port 8 and converted to digital form in analogue to digital converter 9. The output from the ADC is buffered in dual frame buffer 10 for input to a digital signal processor 11.

AMENDED SHEET

'PEA/AU

Received 20 February 2001 Alternatively a digital camera could be employed as the video recording means 1. In this case, the digital output from the camera can be input directly to the dual frame buffer 10 through USB port 12. Although a USB port is described in the preferred embodiment, it will be appreciated that other interface protocols such as SCSI, IEEE 1394, etc, can be used.

The digital signals are processed for transmission in the digital signal processing chip 11. The required processing will depend on the transmission network being employed and known signal processing and compression algorithms can be used. Suitable algorithms include H.261, H.263, MPEG1, MPEG2, MPEG4, etc. Random access memory 13 is provided for on-board storage of data for signal processing.

Although a software implementation of the signal processing has been described, a hardware implementation is also suitable. In one embodiment, signal processing can be performed using an ASIC (Application specific integrated circuit)/FPGA (Field programming gate array) solution. Altemrnatively, an ASIC front end can be used with a programmable DSP backend. The invention is not limited to any one of these possible solutions.

If a DSP solution is used, the DSP is programmed with available compression software. The invention is not limited to any particular software solution, although the inventor has found that 8-bit and 24-bit coding methods are suitable. Different coding methods may be more suitable for certain colour video displays available in the wide variety of PDA's currently on the market. New coding solutions are regularly becoming available and the inventor recognises that the invention may employ new algorithms in the future.

The selected video compression method will implement three basic stages. The first stage performs temporal decorrelation, also termed interframe coding. This process consists of an optional block based motion compensation step followed by temporal prediction and replenishment. This determines which portions of the image frame have AMENDED SHEET p EAAU WO 01/06791 PCT/AU00/00364 6 changed and selectively sends update information to reproduce the changes. This eliminates the need to retransmit the entire image each time when there have not been extensive change over time. A side product of this stage is that the output can be used to trigger an alarm to be sent to the video monitor when any motion is detected.

Depending on the coding method used, quantisation may be performed at this point.

The next stage decomposes the resultant interframe images from the previous step into basic coding units. The nature, of these units varies depending on the coding method being used. In the classic transform based coding methods these coding units are 2D blocks of spatial frequency coefficients. These may equally be image primitives of another form such as run length vectors or uniform colour blocks. The values of the coding units are then substantially quantised to eliminate unwanted information. This process is known as intraframe coding.

The final stage involves reordering the symbols or values to be coded so as to enhance compressability followed by some form of variable length or statistically based coding to efficiently create a compact bit stream for each frame. This final stage is almost always lossless. This resulting bit stream is then appropriately packaged in a frame packet for insertion into the output data stream.

Audio coding will also be implemented in the DSP stage. In one embodiment a standard ADPCM compression method is used. GSM speech coding will also be useful for use in conjunction with the integrated PDA/mobile phone scenario.

For each frame of video compressed and transmitted an audio frame of equivalent time duration will also be compressed and sent. In this way, excepting the event of system control frames in the data stream, the predominant structure of the transmitted data stream will consist of alternating video and audio frames in an interleaved manner.

The processed data is buffered in transmission buffer 14 before Substitute Sheet (Rule 26) RO/AU r- t/IAUU/uvv v-t Received 11 May 2001 7 being transmitted by transmission module 15. The transmission module can be any suitable wireless transmission system such as CDMA (code division multiple acces), TDMA (time division multiple access), IP (wireless intemet protocol), PHS (personal handyphone system), LMDS (local multipoint distribution service), wireless ATM, GSM, GPRS, HSCSD and other 3G wireless networks. Alternative embodiments are discussed in detail below.

It will be appreciated that the elements of the interface module need not be physical contained in a single device. For example, the transmission module may be separate from the rest of the interface module. For a wireless WAN protocol, the transmission module may be a GSM or CDMA base station connected to the rest of the interface module by a modem or ISDN connection. If the communication protocol is a wireless LAN, the transmission module may be connected to a separate base station or wireless access point by an ethemet.

A second embodiment of the interface module is shown in FIG 3.

This embodiment is configured to transmit signals to the portable monitor device as well as receive control signals. The DSP is replaced with a general purpose CPU 16. As with the first embodiment, video and audio signals are received at input 8, digitised by ADC 9, and buffered in DFB Signals from a digital source can be input directly on input 12.

As well as RAM 13 for local storage, an external storage device 17 is provided. The external storage device allows large volumes of data to be stored which can be accessed by signals from the portable monitor device. Two types of external storage devices may be provided.

Uncompressed analogue video data must be stored in a VCR which is accessed by a sequential, read-only access. Compressed digital data can be stored on any of a number of suitable devices (including magnetic disks drives, digital tape drives, CD ROMs, DVDs, WORM drives, etc) with read-write, random access. There may be multiple external storage AMENDED SHEET

IPEA/AU

r 11 vvr vv-,v Received 20 February 2001 8 devices.

The external storage devices permit a history of video and audio signals to be stored for review at a later time. One application of this arrangement is the removal andstorage of recordings, such as video and audio tapes, for use in legal proceedings that could result from security breaches detected by the invention. Other applications are discussed below.

Signals 4 are sent and received by a transmit/receive means 18. A number of buffers 19 are provided for both transmission and reception. An output means 20 provides analogue or digital signals at port 21 for controlling various other devices in response to signals received from the portable monitor device. For example, a signal from the portable monitor device could be used to turn lights on or off, reset a motion detector or external alarm, or commence video recording. In one application, a signal from the portable monitor device could be used to commence playback of a prerecorded programme stored on the external storage device 17.

A third embodiment of the interface module 2 is shown in FIG 4.

This embodiment differs from the second embodiment by implementing a multiple address data bus 22. This embodiment facilitates the use of multiple external storage devices and multiple output means. It is also possible to implement multiple input sources which may each have a unique address selectable from the portable monitor device. This embodiment provides maximum flexibility to the wireless video surveillance system. For example, the range of devices connected to the interface module may include a number of video cameras, one or more external movement detectors and multiple control lines (such as light switches). Detection of movement would generate a warning signal that is transmitted to the portable monitor device. The userwould then transmit a signal to select the video camera closest to the detected movement. The video is then visible in the LCD 7 of the monitor 6.

The important elements of the portable monitor device 6 are shown in greater detail in FIG 5. The monitor includes a receive/transmit module AMENDED SHEET

IPA/AU

r_l" II L J t-uv i U U. -r Received 20 February 2001 9 23 for receiving the data transmitted by the interface module 2 and transmitting data to the interface module. Buffers 24 are provided for both transmission and reception. Received signals are processed in the CPU On-board RAM 26 stores data and parameters for the signal processing. The processed image is displayed on the liquid crystal display 7. Control signals input on line 27 are processed by the CPU 25 and transmitted to the interface module 2.

The portable monitor device can be any device capable of wireless network connection that has a graphic display and a general purpose processor. PDAs are preferred due to their availability but many mobile phones, such as WAP phones and I-NET phones are also suitable.

Naturally, purpose built devices will also be suitable. The inventor envisages that hybrid devices that interface a custom DSP with a standard PDA will be useful. In this form, the video decoding process may be performed in the custom DSP instead of the CPU of the PDA.

Fig 6 shows an embodiment in which a VHS video camera 28 is connected to an interface module 29 that provides a low power signal to an antenna 31 for a local area network. This embodiment may have a typical range of 150m for signals 30. The signals 30 are typically radio frequency signals although infrared signals may also be suitable in point to point applications.

The monitor device 33 is a personal digital assistant with a wireless network card 34 and antenna 32 to receive signals 30. The wireless network card 34 interfaces to the personal digital assistant through a PCMCIA slot, compact flash port, or other expansion port.

The embodiment of Fig 6 supports simplex video and full duplex AMENDED SHEET

PFEA/AU

WO 01/06791 PCTIAU00/00364 audio communication. It is most suitable for implementation in domestic applications, for child minding and security. For example, a parent may use the configuration to monitor a child sleeping in a separate room.

Another application is to monitor an entryway. The full duplex audio allows communication with a person seeking entrance while the simplex video channel allows the person to be viewed.

The primary advantage of the embodiment of Fig 6 is that the video monitor is portable and therefore the user can move freely around the home. It will be appreciated that this is a significant improvement over known prior art security and child-minding systems.

For longer range applications the embodiment of Fig 7 is preferable. A digital video camera 35 provides audio and video input to the interface module 36. The transmission module 37 is a standard telephone interface for transmitting signals 38 across a CDMA or GSM cellular phone network from antenna 39. The monitor device comprises a personal digital assistant 40 with a modem 41 connected to a phone 42. The PDA is programmed with algorithms to process the received data for display.

In a variation of this embodiment the PDA and phone are integrated into a single device. Suitable devices have recently become commercially available.

As mentioned above, the embodiments provide simplex video and duplex audio communication. The second embodiment provides duplex audio via the phone. Duplex audio communication for the first embodiment may be provided by built-in microphones provided in some PDA's or by an external microphone coupled to the network card.

These embodiments may be extended by providing local storage of audio and video footage which can be viewed on command from the monitor. Movement and sound detection functionality have also been provided in the interface module so that an alarm can be provided if movement or sound is detected in the viewed scene. Combined movement detection, sound detection and local video storage are Substitute Sheet (Rule 26) RO/AU WO 01/06791 PCT/AU00/00364 11 particularly useful for the second embodiment to facilitate dial-up operation.

In one method of operation, the interface module is configured to detect movement in the field of view. When movement is detected the video footage is stored locally and a call is placed to the portable monitor device. The phone touch tones or the duplex audio channel may then be used to send tone commands to trigger replay of the stored video footage. The great advantage of this systems is that security is provided without the need of a fixed base station with permanent monitoring.

The external storage may be used to store a fixed period of data in one or more FIFO buffers. The size of the FIFO buffers (and hence the length of recorded video) can be a user set variable. When movement is detected the user can choose to view a block of video from a fixed period before the alarm.

It will be appreciated that multiple cameras and interface modules can be programmed to a single or multiple video monitors. It would therefore be possible for a number of security guards to carry personal digital assistants providing mobile monitoring of multiple camera installations.

Although the above embodiments only offer simplex video communication, it will be appreciated that the invention is not limited to this implementation. The provision of simplex rather than duplex video is due to the processing power available in commercial PDA's.

Improved processing power will allow the PDA's to run software that permits compression and transmission of video images. A video recording means is added to the portable monitor device for duplex video transmission.

It will be appreciated that suitable network protocols must be used to ensure that video data is reliably transmitted across the wireless link to the remote monitor. These may be connection oriented, such as TCP, or connectionless, such as UDP. The nature of the Substitute Sheet (Rule 26) RO/AU r ,L V /V V J Received 20 February 2001 12 protocol will change depending on the nature of the wireless network being used, the bandwidth, and the channel characteristics. The protocol must perform the following functions: error control; flow control; packetisation; connection establishment; and link management.

There are many existing protocols for these purposes that have been designed for use with data networks. However in the case of video, special attention may be required to handle errors since retransmission of corrupted data is inappropriate due to the real-time constraints imposed by the nature of video on the reception and processing of transmitted data.

To handle this situation the following error control scheme is provided.

Frames of video data are individually sent to the receiver, each with a check sum or cyclic redundancy check appended to enable the receiver to assess if the frame has been received in error; (2a) If there was no error then the frame is processed normally; (2b) If the frame is in error then the frame is discarded and a status message is sent to the transmitter indicating the number of the video frame that was in error; The video transmitter upon receiving such an error status message stops sending all predicted frames and instead immediately sends the next available key frame to the receiver; After sending the key frame the transmitter resumes sending normal interframe coded video frames until another error status message is received.

A key frame is a video frame that has only been intraframe coded but not interframe coded. Interframe coding is where the prediction processes is performed and makes these frames dependent on all the preceding video frames after and including the last key frame. Key frames are only sent as the first frame and whenever an error occurs. The first frame needs to be a key frame because there is no previous frame to predict from to perform the interframe coding AMENDED SHEET

'PEAJU

WO 01/06791 PCT/AU00/00364 13 process.

Although the primary application is envisaged as being point-topoint, a broadcast implementation is also possible. In this case, there may be multiple monitors for a single base station. This implementation may be particularly useful in a facility security application where a number of guards are monitoring the security of a facility from a variety of locations.

Authentication security may also be incorporated into the system to minimise the risk of unauthorised use if the portable monitor device is lost or stolen. Suitable security protocols will be known to persons skilled in this area.

Throughout the specification the aim has been to describe the invention without limiting the invention to any specific combination of features.

Substitute Sheet (Rule 26) RO/AU

Claims (33)

1. A wireless video surveillance system comprising: a) one or more video recording means for recording video images of a scene; b) at least one interface module for converting said video images to transmittable data; and c) a hand-held portable monitor device comprising conversion and display means for respective conversion of said transmittable data to said video images, and display of said video images, said portable monitor device having firstwireless communication means associated therewith for receiving said transmittable data from said interface module; wherein said interface module comprises second wireless communication means for transmission of said transmittable data from said interface module to said portable monitor device.

2. The wireless video surveillance system of claim 1 wherein the portable monitor device is a personal digital assistant or similar hand-held processing unit incorporating processor means, memory means and display means.

3. The wireless video surveillance system of claim 1 wherein the interface module includes input means for receiving video signals from said video recording means.

4. The wireless video surveillance system of claim 1 wherein the video recording means is a digital camera

5. The wireless video surveillance system of claim I wherein the video recording means is an analogue video camera.

6. The wireless video surveillance system of claim 5 wherein the interface module includes a video input means and analog to digital conversion means.

7. The wireless video surveillance system of claim 1 wherein the interface module includes processing means for converting said video images to transmittable data and one or more transmission buffers. AMENDEr SHEET 1PEA/AU Received 20 February 2001

8. The wireless video surveillance system of claim 7 wherein the processing means is programmed with video and audio compression algorithms.

9. The wireless video surveillance system of claim 1 wherein the interface module includes processing means programmed with video and audio compression algorithms and wherein the portable monitor device incorporates a processor means programmed with corresponding audio and video decompression algorithms.

The wireless video surveillance system of claim 1 wherein the first wireless transmission means is a signal receiving means.

11. The wireless video surveillance system of claim 1 wherein the first wireless transmission means is a signal receiving and transmitting means.

12. The wireless video surveillance system of claim 1 wherein the second wireless communication means is a signal transmitting means.

13. The wireless video surveillance system of claim 1 wherein the second wireless communication means is a signal receiving and transmitting means.

14. The wireless video surveillance system of claim 1 wherein the second wireless communication means is integral with said interface module.

The wireless video surveillance system of claim 1 wherein the portable monitor device includes input means for inputting signals for transmitting to said interface module.

16. The wireless video surveillance system of claim 15 wherein said first wireless communication means includes means for transmitting said signals and said second wireless communication means includes means for receiving said signals.

17. The wireless video surveillance system of any one of claims 1-16 further comprising motion detecting means for detecting motion occurring within said scene.

18. The wireless video surveillance system of claim 17 wherein the motion detecting means activates a warning signal when a predetermined AMENDED SHEET IPEAIAU 1% /AUVU/VV_3V-t Received 11 May 2001 16 threshold amount of movement is detected by the motion detecting means.

19. The wireless video surveillance system of claim 18 wherein said warning signal is transmitted to the portable monitor device.

20. The wireless video surveillance system of any one of claims 17-19 further comprising selection means for selection of a video camera which is closest to the detected motion.

21. The wireless video surveillance system as claimed in any one of claims 1-20 further comprising video footage storage means which can be viewed on command from the portable monitor device.

22. The wireless video surveillance system as claimed in any one of claims 1-21 further comprising audio footage storage means which can be viewed on command from the portable monitor device.

23. The wireless video surveillance system as claimed in any one of claims 17-22 wherein the interface module further comprises alarm activation means for activation of an alarm if motion is detected by the motion detecting means.

24. The video surveillance system as claimed in any one of claims 1-23 further comprising audio detecting means for detecting sound which originates from within said scene.

The video surveillance system as claimed in claim 24 wherein the interface module further comprises alarm activation means for activation of an alarm if sound is detected by said audio detecting means.

26. A method of providing wireless video surveillance including the steps of: recording a video image of a scene; processing the recorded image to form data for wireless transmission; transmitting the data to a hand-held portable monitor device; processing the data to display the image on the portable monitor device, wherein the step of processing the recorded image includes the further steps of compressing the image prior to transmission of the image to the portable monitor device and ,MT- JOFD SHEET :P'A/AU PCT/AU00/00364 Received 11 May 2001 16a decompressing the image at the portable monitor device.

27. The method of claim 26 wherein the step of compressing the image comprises the following sequential steps: Temporal decorrelation to determine which portions of an image frame have changed to enable information to be selectively updated to reproduce the changes; Interframe coding comprising decomposition of interframe images resulting from step into basic coding units and substantial quantisation of the basic coding units to eliminate unwanted information; and; Reordering of symbols or values to be coded to create a compact fit stream for each frame.

28. The method of claim 27 wherein step comprises the following sequential steps: Optional block based motion compensation; Temporal prediction; and Replenishment.

29. The method of claim 27 wherein the reordering of symbols or values of step comprises variable length based coding.

The method of claim 27 wherein the reordering of symbols or values of step comprises statistical based coding.

31. The method of claim 27 further comprising the step of activating an alarm if the temporal decorrelation of step indicates that an image frame has changed.

32. The method of claim 26 further including the step of storing said data for later transmission.

33. The method of claim 26 wherein the step of transmitting the data further indicates the steps of: transmitting the data as individual frames of video data, said individual frames including key frames and predicted frames, each frame including a check sum or cyclic redundancy check appended to enable a receiver at the portable monitor device to assess if the frame has been received in error; AMENDED SHEET IPEAIAU PCT/AU00/00364 Received 11 May 2001 16b if the frame is in error, discarding the frame and transmitting a status message indicating an identifier of the frame that is in error; stopping the transmitting of predicted frames and transmitting the next available keyframe; and after transmitting the keyframe resuming transmitting predicted frames. AMENDED SHEET IPEAAU