GB2585863A

GB2585863A - Object detecting and monitoring

Info

Publication number: GB2585863A
Application number: GB1910272.2A
Authority: GB
Inventors: Basil Harrold William; Jackson Timothy; David Lewis Jonathan; James Nickalls John
Original assignee: Telensa Holdings Ltd
Current assignee: Telensa Holdings Ltd
Priority date: 2019-07-18
Filing date: 2019-07-18
Publication date: 2021-01-27
Also published as: GB201910272D0

Abstract

A first sensing system identifies a travelling object at a first location by detecting an identifier thereon. A second sensing system detects an identified travelling object at the first location and tracks a detected travelling object to monitor it as it moves from the first location. The travelling object may be a vehicle or person. The identifier may be a licence plate. The first system may comprise one or multiple Automatic Number Plate Recognition cameras 60; RFID detectors; or facial recognition cameras. The second system may be multiple video cameras 62, radar detectors or wireless transceivers and may be used to determine when a vehicle occupies a parking space while the identifier indicates whether it is entitled to park there. Thus, a small number of ANPR cameras are provided at key locations and then each passing vehicle is tracked over a wider range 74 using less sophisticated cameras - mitigating the high cost and short effective range of ANPR cameras.

Description

OBJECT DETECTING AND MONITORING

This relates to monitoring and detecting objects, and in particular, but not exclusively, to monitoring vehicular traffic through a network of roads.

It is known to mount cameras in locations from which they can monitor traffic. For example, it is known to mount cameras at traffic light-controlled road junctions, in order to check that vehicles are not entering the junction when a stop signal is showing. Similarly, it is known to mount cameras adjacent to areas where traffic flow is restricted, such as bus lanes or high-occupancy vehicle lanes, in order to check that only permitted vehicles are using those areas. As another example, it is known to use cameras to identify when a vehicle is parked in a location where parking is not permitted.

Automatic number plate recognition (ANPR) systems, also known as Automatic license plate recognition (ALPR) or Automatic vehicle identification (AVI), are also known. These include at least one camera, positioned to obtain an image of a vehicle number plate or licence plate, plus software for obtaining the vehicle identifier from the image. ANPR systems are often used in situations where a camera is used to check for traffic violations, so that any infringers can be identified and contacted by the relevant authority.

However, ANPR cameras are relatively expensive for widespread deployment. Moreover, due to the high resolution required in the camera, they typically have a short effective range. In addition, of course, an ANPR camera can only work as intended if one of the number plates of the vehicle is facing the camera. Thus, ANPR cameras are usually of very restricted usefulness in monitoring for parking violations in areas where parking is permitted with restrictions. Although in principle a vehicle identifier can be used to determine whether a particular vehicle is entitled to park in a particular location where parking is restricted, it is often impractical or impossible to read the number plate of every vehicle in a car park or in a street.

According to an aspect of the present invention, there is provided a method of detection, comprising: identifying a travelling object at a first location by using a first sensing system to detect an identifier present on the travelling object; detecting an identified travelling object at the first location using a second sensing system; and tracking a detected travelling object using the second sensing system to monitor the travelling object as it moves from the first location.

The first sensing system may comprise a first camera, or may comprise a plurality of first cameras.

The identifier may be a vehicle licence plate, and the first sensing system may then comprise an Automatic Number Plate Recognition camera.

The second sensing system may comprise a plurality of second cameras, and the 15 second cameras may be video cameras.

The second sensing system may comprise a plurality of radar detectors.

The travelling object may be a vehicle, and the method may further comprise: determining using the second sensing system when the vehicle occupies a parking space; and determining using the vehicle identifier whether the vehicle is entitled to occupy the parking space.

According to another aspect, there is provided a detection system, comprising: a first sensing system, configured for identifying a travelling object at a first location by detecting an identifier present on the travelling object; a second sensing system, configured for detecting an identified travelling object at the first location, and for tracking a detected travelling object to monitor the travelling object as it moves from the first location.

The identifier may be a vehicle licence plate, and the first sensing system may comprise an Automatic Number Plate Recognition camera.

The second sensing system may comprise a plurality of second cameras, and the second cameras may be video cameras.

The second sensing system may comprise a plurality of radar detectors.

The travelling object may be a vehicle, and the second sensing system may be positioned to detect when the vehicle occupies a parking space, and the system may comprise a memory for storing information relating to vehicle identifiers of vehicles that are entitled to occupy the parking space.

This has the advantage that a relatively small number of sensors can be deployed in the first sensing system.

For example, in the case where the first sensing system comprises an Automatic Number Plate Recognition sensing system, a relatively small number of ANPR cameras can be deployed, while any vehicles that have been identified using the ANPR system can be tracked over a wider range using the second sensing system.

For a better understanding of the invention, and to show how it may be put into effect, reference will now be made, by way of example, to the accompanying drawings, in which:-Figure 1 shows a part of a road network; Figure 2 shows in more detail one section of the road network shown in Figure 1; Figure 3 is a flow chart illustrating a method in accordance with the invention; Figure 4 illustrates a system in accordance with the invention; and Figure 5 illustrates one use of the method and system in accordance with the invention.

DETAILED DESCRIPTION

The system and method described in detail herein relate to identifying and then tracking a moving vehicle, using a first camera system and a second sensing system. However, this embodiment is provided for the purposes of illustration only, and the system and method of the disclosure can be used for identifying and tracking any moving object, using suitable sensor systems.

Figure 1 illustrates a part of a road network, in a town or city. Specifically, Figure 1 shows various intersecting roads 10, 12, 14, 16, 18, and these define a large number of routes, along which vehicles can travel. Thus, the vehicle 100 on the road 10 can move onto the road 12, as shown at 100a; the road 14, as shown at 100b; or the road 16, as shown at 100c. If the vehicle 100 moves onto the road 14, as shown at 100b, it can then continue along the road 14, as shown at 100d and 100f, or it can move onto the road 18, as shown at 100e.

Figure 1 also shows various monitoring sites, which are provided with sensor apparatus 20, 22, 24, 26, 28, 30, as described in more detail below.

The road network is covered by a first sensing system. and by a second sensing system, as described in more detail below, and the sensor apparatus 20, 22, 24, 26, 28, 30 may be provided with components of the first sensing system and of the second sensing system, or with components of just one of the two sensing systems.

The first sensing system is able to detect an identifier that is present on a travelling object, while the second sensing system is able to detect the presence of the travelling object, without necessarily being able to detect the identifier.

Figure 2 shows a monitoring site 40 in more detail. Specifically, Figure 2 shows a road junction 42, at which four road segments 44, 46, 48, 50 meet.

In the example illustrated in Figure 2, the monitoring site 40 includes an Automatic number plate recognition (ANPR) system, which includes an ANPR camera 60, positioned to obtain an image of vehicle number plates or licence plates, plus software for obtaining the vehicle identifier from the image. In addition, the monitoring site 40 includes a second camera 62.

The identification sensors, for example the ANPR camera 60, are advantageously installed at locations where they have good visibility of vehicle registration plates, for example on a street light at the entry location or the exit location of a road segment (i.e. a section of road without junctions). In the example shown in Figure 2, the camera 60 is positioned so that it can detect the registration plates of vehicles at the position 70, at the end of the road segment 44.

ANPR cameras typically have a short effective range, and this is illustrated in Figure 2 by the segment 72, which shows the effective range of the ANPR camera 60.

By contrast, the second camera 62 has a longer effective range, and this is illustrated in Figure 2 by the segment 74. Thus, the second camera 62 is able to detect vehicles at all of the entry or exit locations of the road segments 44, 46, 48, 50 that meet at the junction 42, i.e. the locations 80, 82, 84, 86, 88, 90, 92, as well as the location 70.

Figure 2 shows the ANPR camera 60 and the second camera 62 located very close to each other. In practice, the ANPR camera 60 and the second camera 62 may be provided effectively as a single piece of equipment in a single housing. Alternatively, they may be provided as separate pieces of equipment mounted adjacent to each other, for example, mounted to the same street light or other piece of street furniture.

As a further alternative, they may be provided completely separately, with the second camera 62 located such that it can detect vehicles that are being identified by the ANPR camera 60.

Figure 3 illustrates a part of the sensor network 100 that is used in the methods described herein. Specifically, Figure 3 shows a first remote unit 102 and a second remote unit 104, both connected to a central site 106.

The first remote unit 102 has a sensor 110, which is connected to a processor 112, which in turn is connected to a memory 114. The memory 114 may contain stored data and/or program instructions for causing the processor 112 to perform methods as described herein on the stored data and/or on data received from the sensor 110. The first remote unit 102 also includes a transceiver 116, for communicating with the central site 106.

The second remote unit 104 has sensors 120, 122, which are connected to a processor 124, which in turn is connected to a memory 126. The memory 126 may contain stored data and/or program instructions for causing the processor 124 to perform methods as described herein on the stored data and/or on data received from the sensors 120, 122. The second remote unit 104 also includes a transceiver 128, for communicating with the central site 106.

In practice, the sensor network 100 will typically include many more than two remote units.

The central site 106 includes a transceiver 130, for communicating with the remote units 102, 104, etc. The transceiver 130 of the central site 106 is connected to a processor 132, and to a memory 134. The memory 134 may contain stored data and/or program instructions for causing the processor 132 to perform methods as described herein on the stored data and/or on data received from the remote units 102, 104, etc. As described above, the sensor network 100 includes a first sensor system and a second sensor system, where the first sensor system comprises ANPR cameras, and the second sensor system comprises other cameras. In addition, the second sensor system may include radar units. A combination of a camera and a radar unit is particularly useful for tracking a moving object as described in more detail below.

In other examples, the first sensor system may use any technology to identify an object, for example the first sensor system may use a Radio Frequency Identification, RFID, detector to detect an RFID tag on an object and thereby identify the object.

As another example, the second sensor system may use any technology to determine that an object is the same object that has previously been identified at a different location. For example, the second sensor system may include a plurality of Bluetooth (or other short-range wireless communication, such as WiFi) transceivers, which may again be mounted on street lights or the like.

As a vehicle or other object that is Bluetooth-enabled passes one of these transceivers, 35 the transceiver is able to detect the MAC address of the device. The movement of a device having that MAC address can be used as the second sensor system to track the object, after it has first been identified using the first sensing system.

As a further example, the second sensor system may include a plurality of RFID detectors at different locations, for example mounted on street lights or the like. When REID detectors are used in the second sensor system, each REID detector detects the RFID tag on an object as it passes the detector, but the intention in this case is not to identify the object, but to track the object, by determining that the object has the same identifier as an object previously detected by another detector.

The remote unit 102 includes a single sensor 110, which may be a part of the first sensor system or the second sensor system. The remote unit 104 includes two sensors 120, 122, which may be parts of the first sensor system and the second sensor system respectively. in general, the second sensor system includes more sensors than the first sensor system, and so, of the remote units that include a single sensor, more of these include a sensor that is part of the second sensor system than a sensor that is part of the first sensor system.

Where reference is made to the processing of information from multiple remote units, this may for example be carried out in a processor at the central unit. Where reference is made to the processing of information from a single remote unit, this may for example be carried out in a processor of that remote unit, or in a processor at the central unit.

Figure 4 is a flow chart, illustrating a method in accordance with the invention.

The method starts at step 150, by identifying a travelling object at a first location by using a first sensing system to detect an identifier present on the travelling object.

In the illustrated embodiment, in which the travelling object is a motor vehicle, the first sensing system may include a network of ANPR cameras, and the vehicle may be identified by using one of the ANPR cameras as a component of the first sensing system.

For example, in the network illustrated in Figure 1, the sensor apparatus 20 may include an ANPR camera, positioned such that it can detect vehicles such as the vehicle 100 as it moves along the road 100.

In the situation illustrated in Figure 2, the ANPR camera 60 may be used to detect a licence plate on a vehicle within the ANPR camera range 72. The licence plate information obtained from the specific vehicle can then be compared with a database of stored licence plate information relating to a large number of vehicles.

In other embodiments, each travelling object may be provided with an identifying tag, such as a Radio Frequency Identification, RFID, tag. In that case, the first sensing system may for example include one or more RFID detectors for detecting the identifying RFID tags when the travelling objects pass the detectors. The information stored on the RFID tag can then be compared with a database of information that associates the information stored on multiple RFID tags with respective objects, allowing an object that passes a detector to be identified.

In other embodiments, the travelling objects may be people, and the first sensing system may for example include one or more facial recognition cameras, which are able to identify people as they pass one specific location.

At step 152 of the method shown in Figure 4, an identified travelling object is detected at the first location using a second sensing system.

In the illustrated embodiment, in which the travelling object is a motor vehicle, the second sensing system may include a network of other cameras and/or radar units, and the vehicle may be detected by using one of the cameras or radar units as a component of the second sensing system.

For example, in the network illustrated in Figure 1, the sensor apparatus 20 may include a second camera, positioned such that it can detect vehicles such as the vehicle 100 as it moves along the road 100, within the effective field of view of the ANPR camera that is provided at the same location.

In the situation illustrated in Figure 2, the camera 62 may be used to detect the vehicle at substantially the same time as it is identified by the ANPR camera 60.

Thus, an image of the vehicle is formed by the camera 62, and this can be associated with the vehicle identifier that is detected by the ANPR camera 60. Specifically, if it is known that the ANPR camera 60 is able to read licence plate information from vehicles in a relatively small area, a camera 62 can be positioned such that it is also able to detect vehicles over that same area, and an image of a vehicle that is within that small area can be associated with the licence plate information.

Multiple such images may be associated with the detected vehicle identifier, and optionally data obtained from such images may be combined to form a model of the vehicle, for example containing information about such features as the colour, size, 3D shape etc of the vehicle.

As mentioned above, the second sensing system may include a radar unit instead of, or as well as, a camera, and the radar unit can similarly detect a moving object at the time that it is known to be in the relatively small coverage area of the ANPR camera.

Then, at step 154 of the method shown in Figure 4, the detected travelling object is tracked using the second sensing system, in order to monitor the travelling object as it moves from the first location.

As described above, the second sensing system includes multiple sensors such as cameras and/or radar units. In certain embodiments, these sensors are positioned such that their effective fields of view overlap.

Thus, in the situation illustrated in Figure 2, when the ANPR camera 60 reads the licence plate of a vehicle at the location 70, the camera 62 can detect the vehicle at the location 70 and form an image thereof. Then, assuming that vehicles drive on the left hand side of the road in this illustrated location, the camera 62 can also detect whether the vehicle passes through the location 80, the location 84, or the location 88 when it leaves the junction 42. This can be done by tracking the location of the originally detected vehicle over successive frames of video captured by the camera 62.

If the field of view of one or more other camera overlaps with the field of view of the camera 62, then that other camera can be used to track the path of the vehicle through

its field of view.

If there is no other camera whose field of view overlaps with the field of view of the camera 62, then the image of the vehicle that was obtained by the camera 62 can still be used to recognise that vehicle as it enters the field of view of another camera. In particular, if the camera 62 has been able to determine which of the locations 80, 84, or 88 the vehicle passed through, the images generated by the next camera long the relevant road segment can be examined for further images of the vehicle.

Thus, even when there is a relatively small number of ANPR cameras, the licence plate information can be associated with a vehicle image, and the larger number of standard cameras can be used to track the vehicle. The licence plate information can then still be associated with the vehicle image, even when the vehicle is some considerable distance from the ANPR camera that captured its licence plate information.

As mentioned above, even though the tracking of vehicles using cameras is discussed, radar units can be used in a similar way to track vehicles.

Thus, when the second sensor system includes cameras, each video camera provides a sequence of frames of the scene. On each frame, a detector is applied to detect which pixels have moved compared to the previous frame. Since the camera is typically static, this implies that an object is moving in the scene. Moving objects, so identified, are then passed to a classifier (typically a convolutional neural network) which identifies the object type and its location in the frame. Given a sequence of such frames, it is possible to track the movement of objects in the frame, using for example a Kalman filter.

Close to the camera, it is possible to infer the distance of the object from the camera, based on the location of the object in the image. Further away, the range can be inferred from knowledge of the type of object being seen, and its apparent size in the camera image. For example, a bus occupying a small part of the image can be inferred to be in the distance. The output of this processing will be a classification of each object, and a track through the scene.

Some additional processing may be required to improve these tracks. For example, an 35 object may be hidden in some frames as it passes behind another object. A system track is generated by applying such insights and the prior knowledge that we have a set of physical objects moving in the scene.

When the second sensor system includes radar detectors, these may for example be frequency-modulated continuous-wave (FMCW), multiple input, multiple output (MIMO) radar units periodically recording frames of radar data. In one example, a sequence of 3 Fast Fourier Transforms (FFTs) are applied to the received data.

The first is a range FFT on the combined data from all the receivers in the MIMO radar system. Then, a Doppler FFT is applied between successive radar chirps (and again using combined data from all MIMO receivers). The output of the first two FFTs is known as the radar video which is a set of 2D range-velocity FFT bins updated in time every radar frame. A detection process, for example using a constant false alarm rate (CFAR) detector, is applied to the radar video to detect moving objects. These are radar returns that have non-zero Doppler and received power well above the

background level.

For each CFAR detection, an Angle FFT is performed across the separated data from each receiver, in order to calculate the angle of arrival of the signal. Typically, the receivers are spaced half a wavelength apart in order to facilitate this.

The output of this is a set of radar points, with each point having a range, velocity, angle and signal-noise ratio (SNR). The radar points are updated in every frame. A physical object in the scene, such as a vehicle, will produce a cluster of radar points and a clustering algorithm is applied to associate points with an object. Each cluster is then tracked using a tracking algorithm such as a Kalman filter. As with the video system described above, the knowledge that we are tracking real objects in the scene can be used to improve the tracks, for example where one object passes behind another. It is possible to perform some limited classification of objects using the radar system, based on the size of the cluster and the knowledge of the range.

When the second sensor system includes both cameras and radar detectors, these independently produce system tracks for each object in the scene. It is typically necessary to perform some co-ordinate conversion to map these tracks into the same co-ordinate system. If we first consider a simple scene with one moving vehicle, we should have one system track from each of the radar and the camera. If the tracks are sufficiently similar, then they can be fused. It is important to look at the whole available tracks rather than just the data from a single frame. The same association process can be applied to more complex scenes.

Moreover, similar systems can be used to track other moving objects, having a first sensor system (such as a RFID tag reader) that is able to obtain identifier information, and a second sensor system (such as a camera system) that is able to track an identified object.

One use of such a system, to be described in more detail below, is in detecting parking violations.

Figure 5 shows a typical situation, in which a number of vehicles 170, 172, 174 are parked at a roadside. A street light 176 is nearby, and a camera 178 or other sensor can be mounted to the street light 176.

In some situations, parking is prohibited, and so a conventional camera can be used to determine that a vehicle is parked, and it is known that any parked vehicle must be parked illegally. In those situations, the conventional camera provides enough information to allow an enforcement officer to be sent to the vehicle, which can then be ticketed.

In many other situations, however, parking is permitted, but with conditions. For example, identification of a vehicle is essential for many parking management use cases. For example, a residents' parking scheme may be in operation, whereby the parking of certain registered vehicles is permitted. Similarly, the location may be one where the holders of disabled parking permits are allowed to park. As another example, parking may be permitted on payment of a fee, and the fee payment may be carried out by the driver supplying licence plate information to a parking authority. As a further example, it may be permitted to park certain categories of vehicles, such as emergency vehicles, or electric vehicles, in some areas.

Traditionally, the legality of the parking in such areas is checked by having the enforcement officer visiting each parked vehicle, and checking its entitlement to park, but this is labour intensive and inefficient One possibility that could be considered would be to provide ANPR cameras at locations where they can see every parking space where parking is permitted with conditions. The identity of each parked vehicle could then be determined, and this identity could be compared with suitable databases, in order to determine whether a parked vehicle falls into one of the relevant categories, such as, in the examples given above: a registered resident's vehicle; a vehicle owned by a holder of a disabled parking permit; a vehicle whose driver has paid the relevant parking fee; or an emergency or electric vehicle.

However, this would require a very large number of ANPR cameras, because an ANPR camera needs to be able to view the licence plate in an unobstructed way, and at a suitable angle, and with appropriate illumination, in order for the associated software to be able to detect the licence plate information. For example, in the situation illustrated in Figure 5, if the camera 178 were an ANPR camera, it would probably be unable to read the licence plates 180, 182, 184 of any of the vehicles 170, 172, 174.

Therefore, an alternative is to use the method of Figure 4.

Specifically, the first sensor system is used to identify the vehicle, and this may be an ANPR camera, which is positioned at a location where it can work reliably, that is, it has an unobstructed, well-lit, field of view that covers a specific point. The second sensor system is used to detect the vehicle at, or close to, that same specific point. The licence plate information detected by the ANPR camera system can then be associated with the vehicle detected at the same point by the second sensor system.

The second sensor system can then be used to track the vehicle from the point at which it was identified. In particular, the second sensor system can be used to track the vehicle until it reaches a parking location, and so the overall system is able to identify the parked vehicle.

When the vehicle is tracked to a parking location, the licence plate information detected by the ANPR camera is still associated with the vehicle that has been tracked.

At that point, the licence plate information can be used to reference one or more database, that is used to store information about vehicles that are permitted to park at that location.

For example, if the parking location is a residents' parking area, a database will be able to determine whether the parked vehicle is suitably registered in the residents' parking scheme. As another example, if parking is permitted on payment of a fee, a database will store the licence plate information of each vehicle for which a parking payment has been made. As a further example, a database will store the licence plate information of each vehicle that is associated with a disabled person, and has the right to park in certain locations for that reason. As a further example, a database will store the licence plate information of other vehicles, such as emergency vehicles, that are permitted to park in locations where parking is generally not permitted.

Thus, tracking an identified vehicle to a parking location allows the system to determine whether the vehicle is entitled to park at that location at that time. This means that an enforcement officer only need to visit a parked vehicle when there are good grounds for thinking that the vehicle is not entitled to be parked there, and so the human intervention can be more targeted, and therefore faster and more efficient. In other cases, no human intervention is necessary, and a notice of violation can be mailed directly to the owner of a vehicle that is determined to be parked illegally.

Although the use of the method for parking enforcement has been described as one specific example, it will be appreciated that the same method can be used on other situations, for example to identify vehicles that are travelling in bus lanes, high-occupancy vehicle lanes, or the like, without requiring an ANPR camera to be positioned at each location of interest. Rather, a relatively small number of ANPR cameras can be provided at a number of key locations, and then each vehicle passing one of the ANPR cameras can be tracked using less sophisticated cameras.

Thus, a vehicle or other moving object can be identified at one location using a first sensor system, and a second sensor system can be used to track the vehicle or other moving object from that first location to at least one other location.

Claims

CLAIMS1. A method of detection, comprising: identifying a travelling object at a first location by using a first sensing system to detect an identifier present on the travelling object; detecting an identified travelling object at the first location using a second sensing system; and tracking a detected travelling object using the second sensing system to monitor the travelling object as it moves from the first location.
2. A method according to claim 1, wherein the first sensing system comprises a first camera.
3. A method according to claim 1, wherein the first sensing system comprises a plurality of first cameras.
4. A method according to claim 2 or 3, wherein the identifier is a vehicle licence plate, and the first sensing system comprises an Automatic Number Plate Recognition camera.
5. A method according to any preceding claim, wherein the second sensing system comprises a plurality of second cameras.
6. A method according to claim 5, wherein the second cameras are video cameras.
7. A method according to any preceding claim, wherein the second sensing system comprises a plurality of radar detectors.
8. A method according to any preceding claim, wherein the travelling object is a vehicle, the method further comprising: determining using the second sensing system when the vehicle occupies a parking space; and determining using the vehicle identifier whether the vehicle is entitled to occupy the parking space.
9. A detection system, comprising: a first sensing system, configured for identifying a travelling object at a first location by detecting an identifier present on the travelling object: a second sensing system, configured for detecting an identified travelling object at the first location, and for tracking a detected travelling object to monitor the travelling object as it moves from the first location.
10. A system according to claim 9, wherein the first sensing system comprises a first camera.
11. A system according to claim 10, wherein the first sensing system comprises a plurality of first cameras.
12 A system according to claim 10 or 11, wherein the identifier is a vehicle licence plate, and the first sensing system comprises an Automatic Number Plate Recognition 15 camera.
13. A system according to any of claims 9 to 12, wherein the second sensing system comprises a plurality of second cameras.
14. A system according to claim 13, wherein the second cameras are video cameras.
15. A system according to any of claims 9 to 14, wherein the second sensing system comprises a plurality of radar detectors.
16. A system according to any of claims 9 to 15, wherein the travelling object is a vehicle, wherein the second sensing system is positioned to detect when the vehicle occupies a parking space; and comprising a memory for storing information relating to vehicle identifiers of vehicles that are entitled to occupy the parking space.