WO2003023716A2

WO2003023716A2 - Dtermining occupancy of a region monitored by a video apparatus

Info

Publication number: WO2003023716A2
Application number: PCT/GB2002/004128
Authority: WO
Inventors: Brian David Vincent Smith; Trevor Michael Ellis
Original assignee: Pips Technology Limited
Priority date: 2001-09-11
Filing date: 2002-09-11
Publication date: 2003-03-20
Also published as: AU2002326025A1; WO2003023716A3; GB2379502A; GB2379502B; GB0121966D0

Abstract

The method comprises the steps of marking a background surface (10) of the region (11) with a well-defined pattern of markings (14); recording an image of the region using the video apparatus (12, 13) and producing a video signal embodying the recorded image; and using the video apparatus (13) to analyze the video signal embodying the recorded image in order to determine occupation of the monitored region (11) by an object. The video apparatus (13) determines occupation of the monitored region by determining from the video signal embodying the recorded image an obscuring of the marked well-defined pattern (14) by a silhouette of the object. The video apparatus (13) determines an obscuring of markings of the pattern (14) by transforming the video signal to obtain a signal indicative of at least a part of a frequency spectrum of the video signal.

Description

A Method of Using Video Apparatus to Determine Occupancy of a Region Monitored by the Video Apparatus and Video Apparatus for use in the Method

The present invention relates to a method of using video apparatus to determine occupancy of a region monitored by the video apparatus and to video apparatus for use in the method.

Using video apparatus to automatically detect an object, the shape of which is known beforehand, e.g. a gear wheel on a conveyor belt, is an established technique in image processing. Indeed having found the object, further image processing may be applied to determine if there are defects within the object, e.g. a missing tooth on the gearwheel.

The present invention relates to a method of using video apparatus to detect objects that fall within a broad class, e.g. the vehicles on a road, where the problem is to determine visually whether there is an object within the class at a particular location. In this example it is desirable to know whether a vehicle is present, the approximate size of the vehicle and the direction and speed of the vehicle. The task of visually recognising vehicles reliably and robustly is difficult.

The present invention provides in a first aspect a method of using video apparatus to determine occupancy of a monitored region, the method comprising the steps of: marking a background surface of the region with a well-defined pattern of markings; recording an image of the region using the video apparatus and producing a video signal embodying the recorded image; and using the video apparatus to analyze the video signal embodying the recorded image in order to determine occupation of the monitored region by an object; wherein: the video apparatus determines occupation of the monitored region by determining from the video signal embodying the recorded image an obscuring of the marked well-defined pattern by a silhouette of the object; and the video apparatus determines an obscuring of the pattern by transforming the video signal to obtain a signal indicative of at least a part of a frequency spectrum of the video signal.

In a second aspect the present invention provides video apparatus comprising: a video camera which records video images and produces video signals; and an electronic image processor which filters each video signal using a bandpass filter to produce a bandpass filtered signal, which rectifies the bandpass filtered video signal to produce a rectified signal and which filters the rectified signal using a lowpass filter to produce a lowpass filtered signal; wherein: when the video apparatus is used to monitor a region having a background surface on which there are a plurality of markings in a well-defined pattern then the video camera can be aligned and the bandpass filter set in such a way that the filtering of each video signal is matched to a fundamental spatial frequency corresponding to the separation of the markings of the pattern and therefore the level of the lowpass filtered signal gives an indication of obscuring of the markings by an object passing over them.

In a third aspect the present invention provides video apparatus comprising: a video camera which records video images and produces video signals; and an electronic image processor which filters each video signal using a bandpass filter to produce a bandpass filtered signal, which rectifies the bandpass filtered video signal to produce a rectified signal and which integrates the rectified signal using an integrator to produce an integrated filtered signal; wherein: when the video apparatus is used to monitor a region having a background surface on which there are a plurality of markings in a well-defined pattern then the video camera can be aligned and the bandpass filter set in such a way that the filtering of each video signal is matched to a fundamental spatial frequency corresponding to the separation of the markings of the pattern and therefore the level of the integrated signal gives an indication of obscuring of the markings by an object passing over them.

In a fourth aspect the present invention provides video apparatus comprising: a video camera which records video images and produces video signals; and an electronic image processor which transforms each video signal to obtain at least one component of the frequency spectrum of the video signal; wherein: when the video apparatus is used to monitor a region having a background surface on which there are a plurality of markings in a well-defined pattern then the video camera can be aligned in such a way that at least one component of the frequency spectrum is matched to a fundamental spatial frequency corresponding to the separation of the markings of the pattern and therefore the level of the transformed video signal gives an indication of obscuring of the markings by an object passing over them.

As an aid to understanding the invention, preferred embodiments thereof will now be described by way of example and with reference the accompanying drawings, in which:

Figure 1 is a schematic representation of apparatus used in a method according to the present invention;

Figure 2 is a representation of a first image recorded by video apparatus used in a method according to the present invention;

Figure 3 is a representation of a second image recorded by video apparatus used in a method according to the present invention;

Figure 4 is a representation of a third image recorded by video apparatus used in a method according to the present invention; Figure 5 is a graphical representation of a video signal;

Figure 6 is a graphical illustration of a frequency spectrum of a video signal;

Figure 7 is a block diagram showing a part of a first embodiment of the video apparatus used in a method according to the present invention;

Figure 8 is a block diagram showing a part of a second embodiment of video apparatus used in a method according to the present invention; and Figure 9 is a circuit diagram showing a circuit which can be incorporated in the apparatus in addition to the parts of the apparatus shown either in figure 7 or figure 8

In Figure 1 there can be seen a roadway 10, a region 11 of which is monitored by video apparatus which comprises a video camera 12 connected to an electronic image processor 13. The video apparatus is used to determine the presence of vehicles in the region 11. To facilitate this a well defined pattern

14 of markings is marked on the surface of the roadway in the region 11. The pattern comprises nine markings affixed (e.g. painted) to the roadway surface in predetermined positions, with pre-determined spacings between the markings, in the region 11, within the field of view of the video camera 12. For simplicity the video camera 12 is shown vertically above the monitored region 11, but it does not necessarily need to be (providing that this is taken into account in subsequent image processing) . A first recorded image is shown in Figure 2, this being an image of the region when a vehicle is not present.

Figure 3 shows a second recorded image of the monitored region 11. In the image a vehicle obscures two markings. The fact that two markings (references BQ and BR) are obscured gives the approximate size of the vehicle. The obscuring of the two markings in the recorded image enables the electronic processor 13 of the video apparatus to determine occupancy of the region 11 in a manner that will be described later.

Figure 4 shows a third recorded image taken subsequent to the recorded image of figure 2 with the vehicle having moved. Two different markings (references BP and BQ) are obscured. Again the obscuring of the two markings in the recorded image enables the electronic processor to determine occupancy of the region 11 by the vehicle. Also if the time delay between capturing the images in figure 2 and figure 3 is recorded by the electronic image processor 13 then the speed of the vehicle may be calculated.

Only a few simple markings are used in the above example; the principle may be extended to many more markings such that the size and position of the obscured area of the monitored region can be determined with greater precision. This also allows the speed of a vehicle to be accurately measured.

The markings are arranged such that the spatial spectrum of the markings in figure 2 can be analyzed by a matched filter looking for the fundamental spatial frequency corresponding to the separation of the marks.

The camera apparatus 12 will continuously record frames each embodying one recorded image of the region 11. A video signal for each frame will in the absence of a vehicle in the region 11 comprise a ^Λsquare wave' signal as shown in figure 5. The video signal for each frame will be a continuous signal representing the image as scanned across each video line in turn down the complete image. A typical frequency spectrum for a ^Λsquare-wave' video signal representing a recorded frame of a pattern equal marked and non-marked areas is shown in figure 6.

The frequency spectrum of a scanned frame can be obtained e.g. by carrying out a Fourier transform method (e.g. a Fast Fourier transform) on the video signal embodying the recorded frame. It is shown in figure 7 that the electronic processor 13 has a circuit in which at 20 there is obtained a frequency spectrum of a video signal representing a single frame. The derived frequency spectrum can then be analyzed to determine the presence of or otherwise of a vehicle in the region 11. This is done by comparing in comparator 21 the frequency spectrum of the single recorded frame video signal with a reference frequency spectrum stored by the apparatus at 22. The reference frequency spectrum will be the frequency spectrum of a video signal embodying a frame recoding an image of the region 11 when unobscured. The reference frequency spectrum stored at 22 could be stored in an initialization process for the apparatus or could be preprogrammed. A difference between the frequency spectrums is output at 23 and gives a reliable indication of the presence of a vehicle in the region 11 since the obscuring of markings in the region 11 by the vehicle would change the frequency spectrum of a recorded frame of region 11 from the frequency spectrum of an image of an unobscured pattern of markings in the region 11.

Rather than comparing complete frequency spectrums, only one component of a frequency spectrum (e.g. f in figure 6) of a recorded image could be compared with a reference threshold value for that component. If the magnitude of the f component of the frequency spectrum fell below the threshold value then presence of a vehicle in the region 11 could be concluded. The magnitude of the f component could be compared with several threshold values of different magnitudes and from this an indication of the size of the obscured part of the region 11 (i.e. the number of obscured markings) obtained and thereby an indication of the size of the vehicle passing over the region 11.

Rather than taking a Fourier transform of a video signal to obtain a frequency spectrum, which would typically be done digitally using suitable computer apparatus, an simpler method could be used implemented by the apparatus illustrated in figure 8. In this figure an alternative circuit in the electronic processor 13 is shown in which a video signal 31 embodying a single recorded frame is fed into a bandpass filter 32. The bandpass filter 32 is tuned (in bandwidth and centre frequency) to the fundamental spatial frequency of the markings in the pattern in the region 11 (f in figure 6) . The filtered signal output by the filter 32 is rectified by a rectifier 33 and the rectified signal is then passed through a low pass filter/integrator 34, which smooths/integrates the signal. The smoothed/integrated signal gives an approximation to the energy component at the chosen frequency. The smoothed/integrated signal is then compared in a comparator 35 with a threshold value stored at 36. If the smoothed/integrated signal is above the threshold then the apparatus will conclude that there is no vehicle present in the region 11. If the smoothed/integrated signal falls below the threshold then the apparatus will conclude that there is a vehicle present in the region 11. The smoothed/integrated signal will drop in value due to the obscuring of markings in the region 11.

The apparatus can be modified to compare the smoothed/integrated signal with a plurality of thresholds of different values to give an indication of how much of the region 11 is obscured by a vehicle (i.e. how many markings are obscured) and thus the size of the vehicle.

Carrying out frequency analysis of the video signals embodying the recorded images is very important since the signal representing the frequency spectrum of the signal (or a particular component of the spectrum) will , in comparison with the unprocessed video signal, have an improvement in the signal to noise ratio in the detection of the marks in region 11 that approximates to:

Improvement in SNR = (Sample frequency / 2 * Bandpass filter bandwidth)

This means that the methods described above are much more robust to changes in scene and contrast ratio than methods involving point to point comparison of images using video signals straight from the camera apparatus. This is important since e.g. road surfaces get dirty over time and so the contrast between a marking and the surrounding road surface can diminish.

The signal to noise ratio between the background scene and the markings may be increased by a number of additional means. The markings may have a distinctive colour that is optically filtered to give a high contrast image. Alternatively the markings may be made with a retro-reflective material (reflecting light back in the direction of illumination) . This is especially beneficial when the obscured area has a similar reflection amplitude to the markings. If a means of illumination (14 in Figure 1) of the markings^' is located close to the video camera, then the markings will have a much greater reflection amplitude than the obscured area. Reflection from extraneous lighting sources, e.g. the sun, may be further reduced by filtering the optical image with a narrow-bandpass optical filter the centre wavelength of which is the same as the wavelength of the illumination source.

The electronic processor 13 can be supplemented by further signal processing apparatus such as shown in Figure 9, which is a block diagram of an analogue comparator circuit which can be included in the electronic image processor 13. Video signals from the video camera 12 are fed at 40 to an analogue comparator 41 of the processor 13 where the signals are each compared with a pre-set threshold provided on a line 42. The threshold is set so that only those video signals from the areas of the video image that contain the markings exceed the threshold. Horizontal (HD) and Vertical (VD) synchronisation signals are derived from the incoming video signals by means of a sync stripper circuit 43. A video line counter 44 is reset with the VD signal and counts lines with the HD signal. The video line counter 44 output feeds three comparators 45,46,47 where the line numbers are compared with ^λA' , B' , and ^ΛC line numbers of the centre of the markings illustrated in figure 2. An output of a high-speed pixel clock 48 is also counted using a video pixel counter 49, and reset on each line with the HD signal. The video pixel counter output feeds three comparators 50,51,52 where the pixel

(horizontal) count is compared with the horizontal positions ^ΛP' , ^ΛQ' , ^XR' of the centre of the markings in figure 2.

The outputs of the six comparators are logically combined through an AND gate matrix (53 to 61) to produce the signals AP, BP, CP, AQ, BQ, CQ, AR, BR, CR which are true when the video signal is in the centre of the respective marked regions. These signals are used to sample the thresholded video signal using a set of D-type latches (62 to 70) to form the digital word DO through D8 (i.e. a binary string).

The result held in DO through D8 will have the respective bits within it set to a ^Λ0' if a marking is obscured and a l' if the marking is not obscured.

Analysis of a digital word generated at the end of each video field by a digital computer of the electronic processor 13 enables determination of the size and position of the obscured area of the region 11. Comparison by the digital computer of the digital words attributable to two successive video fields enables the speed of motion of the vehicle to be determined.

Alternatively the video signals output by the video camera 12 may be acquired by an electronic video processor 13 that comprises a digital computer and the foregoing analysis of the figure 9 circuit carried out in software run by the computer.

Claims

1. A^' method of using video apparatus to determine occupancy of a monitored region, the method comprising the steps of: marking a background surface of the region with a well-defined pattern of markings; recording an image of the region using the video apparatus and producing a video signal embodying the recorded image; and using the video apparatus to analyze the video signal embodying the recorded image in order to determine occupation of the monitored region by an object; wherein: the video apparatus determines occupation of the monitored region by determining from the video signal embodying the recorded image an obscuring of the marked well-defined pattern by a silhouette of the object; and the video apparatus determines an obscuring of markings of the pattern by transforming the video signal to obtain a signal indicative of at least a part of a frequency spectrum of the video signal.

2. A method as claimed in claim 1 wherein: the video signal is transformed by filtering the video signal using a bandpass filter to produce a bandpass filtered signal; the bandpass filtered video signal is rectified to produce a rectified signal; the rectified signal is filtered using a lowpass filter to produce a lowpass filtered signal; and the video camera is aligned with the markings of the well-defined pattern and the bandpass filter set in such a way that the filtering of the video signal is matched to a fundamental spatial frequency corresponding to the separation of the markings of the pattern and therefore the level of the lowpass filtered signal gives an indication of obscuring of the markings by an object passing over them.

3. A method as claimed in claim 1 wherein: the video signal is transformed by filtering the video signal using a bandpass filter to produce a bandpass filtered signal; the bandpass filtered video signal is rectified to produce a rectified signal; the rectified signal is integrated using an integrator to produce an integrated filtered signal; and the video camera is aligned with the markings of the well-defined pattern and the bandpass filter set in such a way that the filtering of the video signal is matched to a fundamental spatial frequency corresponding to the separation of the markings of the pattern and therefore the level of the integrated signal gives an indication of obscuring of the markings by an object passing over them.

4. A method as claimed in claim 1 wherein the video signal is transformed by producing a frequency spectrum of the video signal using a Fourier transform.

5. A method as claimed wherein at least a component of the frequency spectrum is compared with a reference signal and any difference between them used to determine an obscuring of markings of the pattern.

6. A method as claimed in any one of the preceding claims, wherein the pattern of markings is chosen such that and the video apparatus can by analysis of the recorded image distinguish between occupation of the monitored region by different sizes of object. 5

7. A method as claimed in any one of the preceding Claims, comprising the steps of: making the markings with a material which is chosen to reflect light with a particular spectrum; 10 and optically filtering the recorded image to enhance contrast between the markings and the remainder of the monitored region and/or an area obscured by an object in the monitored region. 15

8. A method as claimed in any one of the preceding Claims, comprising the steps of: making the markings with a material which is retro-reflective; and 20 illuminating the markings using illumination means aligned with a camera of the video apparatus .

9. A method as claimed in Claim 8, wherein the illumination means illuminates the markings with light

25 having a chosen spectrum, and the method additionally comprises the step of filtering the recorded image with a filter matched to the illuminating light spectrum.

30 10. A method as claimed in Claim 9, wherein the illumination means illuminates the markings with infrared energy of a particular wavelength, and the image is filtered with a narrow bandpass filter matched to the wavelength of illumination.

^'35

11. Video apparatus comprising a video camera and an electronic image processor for processing recorded images provided by the video camera, the electronic image processor being configured to operate by a method according to any one of the preceding claims .

12. Video apparatus comprising: a video camera which records video images and produces video signals; and an electronic image processor which filters each video signal using a bandpass filter to produce a bandpass filtered signal, which rectifies the bandpass filtered video signal to produce a rectified signal and which filters the rectified signal using a lowpass filter to produce a lowpass filtered signal; wherein: when the video apparatus is used to monitor a region having a background surface on which there are a plurality of markings in a well-defined pattern then the video camera can be aligned and the bandpass filter set in such a way that the filtering of each video signal is matched to a fundamental spatial frequency corresponding to the separation of the markings of the pattern and therefore the level of the lowpass filtered signal gives an indication of obscuring of the markings by an object passing over them.

13. Video apparatus comprising: a video camera which records video images and produces video signals; and an electronic image processor which filters each video signal using a bandpass filter to produce a bandpass filtered signal, which rectifies the bandpass filtered video signal to produce a rectified signal and which integrates the rectified" signal using an integrator to produce an integrated filtered signal; wherein: when the video apparatus is used to monitor a region having a background surface on which there are a plurality of markings in a well-defined pattern then the video camera can be aligned and the bandpass filter set in such a way that the filtering of each video signal is matched to a fundamental spatial frequency corresponding to the separation of the markings of the pattern and therefore the level of the integrated signal gives an indication of obscuring of the markings by an object passing over them.

14. Video apparatus as claimed in claim 12 wherein the lowpass filtered signal is compared with a threshold value to determine obscuring of the markings by an object passing over them.

15. Video apparatus as claimed in claim 13 wherein the integrated signal is compared with a threshold value to determine obscuring of the markings by an object passing over them.

16. Video apparatus comprising: a video camera which records video images and produces video signals; and an electronic image processor which transforms each video signal to obtain at least one component of the frequency spectrum; wherein: when the video apparatus is used to monitor a region having a background surface on which there are a plurality of markings in a well-defined pattern then the video camera can be aligned in such a way that at least one component of the frequency spectrum is matched to a fundamental spatial frequency corresponding to the separation of the markings of the pattern and therefore the level of the transformed video signal gives an indication of obscuring of the markings by an object passing over them.

17. Video apparatus as claimed in claim 16 wherein at least one component of the transformed video signal is compared in magnitude with a threshold value to determine obscuring of the markings by an object passing over them.

18. Video apparatus as claimed in any one of claims 11 to 17 wherein additionally: the electronic image processor for each complete recorded video frame resolves the video signals produced by the video camera into a plurality of resolved signals each relating to a unique part of the complete recorded video frame; the electronic image processor has thresholding means for comparing the video signals produced by the video camera with a threshold value to produce difference signals indicative of any difference between the threshold value and the video signals; and the electronic image processor uses the resolved signals to control sampling means which samples the difference signals to produce a binary output string having a plurality of elements each associated with a unique part of the complete recorded video frame; whereby when the video apparatus is used to monitor a region having a background surface on which there are a plurality of markings in a well-defined pattern then the video camera can be aligned, the video signals can be resolved and threshold value can be set so that each component of the binary output string indicates whether a particular marking is obscured.

19. Video apparatus as claimed in claim 18 which determines speed of motion of an object across the monitored region by comparing the markings obscured in successive recorded images and by recording the time elapsing between the successive images.

20. Video apparatus as claimed in claim 18 or claim 19 which determines a direction of motion of an object across the monitored region by comparing the markings obscured in successive recorded images.