GB2586198A

GB2586198A - A monitoring and recording system

Info

Publication number: GB2586198A
Application number: GB1903136.8A
Authority: GB
Inventors: Colin Haddon Thomas; Paul Clive
Original assignee: TESCAP Ltd
Current assignee: TESCAP Ltd
Priority date: 2019-03-08
Filing date: 2019-03-08
Publication date: 2021-02-17
Also published as: GB201903136D0; WO2020183345A1

Abstract

A monitoring and recording system for compiling an asset register of street furniture items 22,26,28 for accounting and maintenance purposes. The street furniture may be lamp posts, traffic lights, street markings, road signs, etc. A vehicle mounted camera derives image frames from images, preferably panoramic images, and a global positioning system (GPS) is used to tag each frame with location data. The vehicle may be an autonomous vehicle such as a UAV or drone. An image identifier identifies street furniture items and generates digital identity data which uniquely identifies each item. The image frames, GPS data and digital identity data are transmitted to a remote database for storage. The distance of street furniture items from the vehicle may be determined using a pulsed radiation technique such as LiDAR. An operator may label items of street furniture at a user interface. Preferably, the identity of items of street furniture and their status or condition is determined either by the operator or by an automated image recognition process, which may use a convolutional neural network.

Description

A Monitoring and Recording System

Field

The present invention relates to a monitoring and recording system. More particularly the invention relates to a monitoring and recording system for monitoring items such as road signs, street lamps, traffic lights and street markings, all of which items are collectively known as, and hereinafter referred to as, street furniture or assets.

Background

There is a continuous need, especially by town councils, town planners, traffic departments, maintenance and repair companies and local authorities, to monitor, count and catalogue items of street furniture and their state of repair for accounting and maintenance purposes. This is particularly important for planning and when managing budgets for maintenance, repair and replacement.

Prior Art

A number of automatic surveillance systems exist in order to monitor traffic, especially vehicles that have been illegally parked. One example is that described in UK Patent Application GB-A-2 527 903 (TESCAP) which relates to a vehicle parking enforcement system.

The system described determines whether a parked vehicle is permitted to park in a particular location and automatically monitors the whereabouts of traffic enforcement officers carrying hand held communication devices; then allocates a suspected parking violation to the nearest traffic enforcement officer to investigate and issue a parking violation ticket as appropriate.

An aim of the present invention is to provide a system that is able to be used in real time in order to monitor the state of, record the location of and verify the status of items of street furniture.

The invention arose in order to provide an improved system that is capable of being readily implemented and used alongside existing monitoring and recording systems.

Summary of the Invention

According to a first aspect of the present invention there is provided a monitoring and recording system for compiling an asset register of items of street furniture comprising: a vehicle mounted camera which derives a plurality of image frames from images; a global positioning system (GPS) which provides location data which is associated with each image frame; an image identifier which identifies items of street furniture and generates street furniture digital identity data; and a transmitter which transmits digitised image frames, their associated GPS location data and the street furniture digital identity data to a remote database whereat the digitised image frames, GPS location data and the street furniture digital identity data are stored.

Optionally a convolutional neural network (CNN) is employed to process image data so that frames of data are compared with a series of data derived from verified images.

In a preferred embodiment the imager performs automatic image recognition of items of street furniture or other assets, by using the convolutional neural network (CNN), and either labels the item of street furniture with a code, thereby providing it with a name, or enables an operator to designate the item of street furniture with a name.

During an initial acquisition phase of each item of street furniture, an operator is optionally provided with a preselected name and an option to verify the preselected name. This enables the operator to verify quickly a preselected name; or to choose an alternative name from a predefined menu; or to define a custom name for the item of street furniture.

In some embodiments there may also be provided a means to recognise a unique identifier or code that is mounted on or marked on or transmitted from an item fitted to the item of street furniture. So that for example, the imager is able to obtain and record an image of the item of street furniture.

Preferably a scanner or image recognition device or optical character reader (OCR) or some other automated sensor, is operable to interpret the unique identifier or code and associate it with the particular item of street furniture, thereby enabling the generation of street furniture digital identity data and/or immediate verification of the type or nature of an item of street furniture.

The code may be alpha numeric and/or a bar code or a 0-code or a code received form a transponder or transmitter associated with the item of street furniture. Certain types of bar code may be arranged in a vertical array so that images of the bar code may be captured quickly by a moving imaging system.

In alternative arrangement the code is transmitted as a pulsed radio frequency (RF) signal from a radio frequency identity (RFID) device associated with the item of street furniture and which RFID device may be, for example, powered by a dedicated independent power supply which may be a solar panel.

Optionally a data field is also provided to enable the status or condition of items of street furniture to be captured and recorded, either by way of an operator input terminal or by an automated system such as an intelligent image assessment means which determines a specific qualitative parameter or the type or nature of the item of street furniture.

For example the data field may require a response to questions such as: Is the item of street furniture present? Is the item of street furniture damaged? Does the item of street furniture require maintenance? Alternatively an intelligent image assessment may be performed automatically by the intelligent image assessment means which determines if a lamp is working or a post is bent or if there is a break in a continuous barrier or a street sign is missing or has been damaged.

Ideally digitised image frames, their associated GPS location data and the street furniture digital identity data are stored as digital data in a database or other automatically accessible data store which may be contained in the vehicle or which is located remotely.

Digital data however is preferably transmitted as a continuous data stream or in packets or frames, via wireless data network to a remote database in order to create an asset register. Subsequently the database may be accessed and used as an asset register. Access to the register permits retrieval of location data, street furniture digital identity data and data indicating status so that once accessed data may be used by and/or updated remotely by other users who may be office based.

Authorised users may access and update the aforesaid location data, street furniture digital identity data and data indicating status via hardwire connections or for example via 3G, 4G or 5G networks or other wireless Internet systems, via an Internet connection or via a bespoke dedicated local area network (LAN) or website which may be, for example, in an office, such as a council maintenance department or in a local government planning department or in a law enforcement department.

Preferably the aforementioned plurality of digital image frames is manipulated by an authorised user using a data processor operating under control of software. In one user configuration data may be accessed and manipulated in order to present a continuous digital data image of surroundings which data are stored with the location data of items of street furniture. Additionally, any related data, such as a unique identifier, digital identity data and any operator defined name or status criteria of items of street furniture, may also be stored for the purposes of maintenance, inspection and automated comparison with previously obtained data.

It is appreciated that the present invention is therefore able to be incorporated in or used with the mobile imaging systems as provided in the aforementioned vehicle parking enforcement system, and that data obtained therefrom is able to be scanned and checked, independently of any assessment of parking violations.

It is further appreciated that all forms of image and other data obtained may be compressed in order to speed up transmission and in order to occupy less bandwidth as well as require less storage space. Furthermore data may be encrypted in order to ensure security.

Ideally the monitoring and recording system for compiling an asset register of items of street furniture includes: an imaging means or camera system that is adapted to obtain a 360 degree panoramic image of surroundings. In this embodiment ideally software is operative with a data processor and is configured to receive data inputs from an operator in the vehicle so as to stitch together digital images to produce a continuous non-distorted view of a scene such as a street. An advantage of this is that the images can be made available to third parties, such as local authorities or planning departments, for inspection or planning applications. As such this set of image data may be supplied to third parties under a contract. It is appreciated that post processing may also be carried out on data for example in order to improve machine learning algorithms.

Making available such image data, under contract, may entail modifying data which may require certain types of image data to be included or removed. Software is ideally provided in order to achieve bespoke requirements, such as for example obscuring individuals' faces, vehicle registration numbers or names of shop fronts or hoardings in order to comply with local privacy laws. Likewise data may be encrypted in order to encode the modified data for security purposes or in order to confirm to local privacy laws.

In other embodiments may have facial image recognition software incorporated within imaging systems in order to monitor or track the whereabouts of suspect individuals. Facial recognition may be performed automatically and without input from, or even the knowledge of, an operator or vehicle driver.

Automatic data updates may be transmitted to a vehicle in order to suggest a route for a driver so as to incorporate as many possible points for investigation as possible. In a similar manner various databases within different departments may be polled in order to instruct the system to harvest relevant data of a particular type, for example the state and condition of bus shelter, which advertisements are being displayed on which hoardings or billboards and the number of cars parked in large carparks so as to indicate an approximately number of available parking spaces. The data harvested in this manner may be required quickly and so may be handled in a different manner to the street furniture digital identity data.

In some embodiments there is also provided a means for determining a distance of an object of street furniture from a point on the vehicle in order to pinpoint its absolute location. The means for determining a distance of an object of street furniture from a point on the vehicle ideally includes a source of radiation and a detector to detect reflected radiation from an object of interest.

Preferably the means for determining a distance of the object of street furniture from the vehicle includes: a source of pulsed radiation, a detector arranged to receive a reflected signal and a counter for determining from the source and reflected signal a 'time of flight' of a radiation pulse and therefore the distance of the object of street furniture from the vehicle.

In some embodiments a recording device, which might include a voice recorder such as a digital microphone, enables the operator to provide a definition or a name by way of an oral description which is translated and included in the street furniture digital identity data by a voice recognition system.

Alternatively the voice recognition system may be used to control a camera for example to respond to voice commands of orient in a specific direction or to zoom a camera, for example where a view is obscured, so that a voice command can be used to control or to actuate an imaging device, such as a digital zoom.

In a particularly preferred embodiment the monitoring and recording system enables the operator to associate with a digital image, a descriptor of the street furniture so that this data may be used to generate the digital identity data. The descriptor may be generated for example by way of a touch sensitive display or a stylus on screen.

In some embodiments the system includes a display with a menu that operates using predictive text or short codes for items of street furniture, such as LP for lamp post or TP for telegraph pole, thereby saving time when typing the descriptor, which once verified is used to generate the street furniture digital identity data.

Optionally the monitoring and recording system includes a menu operable by the operator to provide a label for an item of street furniture.

So as to speed up and simplify operation of the system, a menu or a selection of options may be presented on a touch sensitive display thereby enabling an operator for example to assess and measure a distance of an object of street furniture from the vehicle in real time. This information may be used by the operator in order to label the precise whereabouts of a specific item, for example for subsequent investigation or maintenance.

The monitoring and recording system may be incorporated in an existing road inspection system or parking monitoring system or mapping system or it may be deployed in an autonomous vehicle, such as a driverless car or a drone or police helicopter.

A particularly preferred embodiment of the monitoring and recording system includes: a comparator which compares images or portions of an image; and voice recognition and response system that responds to a voice command; and a display which enables an operator to manipulate a scene in order to view an image showing amongst other things items of street furniture.

In a particularly preferred embodiment a scanner scans a scene and obtains stored data and presents image data in a continuous format with the aforementioned digital identity data and operator defined name and status of items of street furniture. Once images are scanned and combined with data and stored on the database they may be retrieved for example for maintenance planning or in order to determine if an object is missing, damaged or faulty. In order for a CNN to be able to locate an object it must be trained with images of the object in question so as to build a weighting function which act as an area of object association on a graph. The rule of training is that the more images obtained the greater is the accuracy of identification.

Ideally the system receives a frame of data from a camera or imager and a frame of data is fed into the CNN via a CNN input. Processing of the data then takes place in the CNN after which a dictionary of details is output from the CNN. The dictionary of details is fed back into an image processing system which determines the class of objects to which the imaged object belongs. For example the image processing system determines if the imaged object is a lamppost or telephone box. The probability of the object being in one class of objects is determined by a number of factors including the position and size of a boundary box that surrounds the object.

Ideally the CNN is therefore able to determine if an object that cannot be readily identified is detected by matching the object with similar size objects in the tracking history using two techniques: Firstly, the system refers to a series of histograms of previous detection events and the one in question is compared with them and if a high enough match is derived, typically around 90% or higher, then the object is assumed to be the same as that from the 'best match'. A verification is then performed to check the location of the match.

Finally, locations of objects in a final camera frame are checked against currently available detected objects. If any are similar enough then they are assumed to be the same object provided histograms are a sufficiently close match. It is appreciated that this technique does not check objects that have not been imaged within the most recent few seconds, as such objects may have moved or been moved while out of view of the imager. For example during some imaging events a bus or lorry may obscure a field of view of the imager.

Image data may be manipulated (encrypted or compressed) or processed (scanned) or digitised locally in the vehicle or remotely at a data centre.

According to another aspect of the invention there is provided method of compiling an asset register of items of street furniture comprising the steps of: operating a vehicle mounted camera in order to derive a plurality of image frames; generating a continuous digital image of surroundings; associating the assets in the image with a location derived from a global position system (GPS); providing location data which is associated with each image frame; deriving from the images identity of items of street furniture and associating with each item of street furniture a unique identifier; and/or the option of transmitting image frames, GPS data and the unique identifier as digital data to a remote database.

It is understood that aspects of the system may be incorporated into the method as appropriate.

Preferred examples of the invention will now be described, by way of example only, and with reference to the Figures in which:

Brief Description of the Figures

Figure 1 shows a monitoring and recording system according to the first aspect of the invention in use on a first earlier date; Figure 2 shows an example of a typical screen output which is a schedule of recorded items of street furniture, with their unique identity codes and their GPS locations and their locations corresponding to the instant of Figure 1; Figure 3 shows the monitoring and recording system of Figure 1 in use at the same location on a second later date; Figure 4 shows typical screen output which is a schedule of recorded items of interest and their locations corresponding to the instant of Figure 3; Figure 5 shows a print out of a verification report corresponding to the typical screen output which is a schedule of recorded items of interest and their locations; Figure 6 shows an overview of one example of a monitoring and recording system with an asset register and a mobile vehicle mounted imaging system; and Figure 7 shows a second monitoring and recording system in use with a plurality of examples of street furniture.

Detailed Description of Preferred Embodiments of the Invention Referring to the Figures generally, and in particular to Figure 6, there is shown an example of a monitoring and recording system 10 according to the first aspect of the present invention which in use obtains image data, processes the image data, and transmits it in a digitised format via a data network 66 to a remote location where it is compiled to produce an asset register of items of street furniture 17, 19, 21, 22, 23, 25, 26, 27, 28, and 29. The asset register 44 is stored on a database 50 and is accessible remotely by one or more users (not shown).

The system 10 comprises: a vehicle 12 mounted 3600 camera system 30 that includes a plurality of cameras and which obtains frames of image data; two, three or four additional pulsed radiation transponders 31, 32, 33 and 34 which acquire reflected radiation from imaged objects; and a data processor 38 which calculates distances of objects from the vehicle and sends this information to an on-board data store 29. Image data is processed by the data processor 38 and transmitted in real time with other data files to a receiver 43 from where data is relayed, via a data network 66, to a database 50. Alternatively, data may be transmitted as an entire file when the vehicle 12 is able to be connected to the database 50 or a related system via a hardwire cable (not shown). The image data and files are used to compile the asset register 44.

As described in detail below, and with reference to Figures 1 and 6, images of items of street furniture 17, 19, 21, 22, 23, 25, 26, 27, 28 and 29 are initially obtained during a first acquisition phase, for example on the first occasion when the vehicle 12 drives along a road and when an operator first identifies and labels each item of street furniture 17, 19, 21, 22, 23, 25, 26, 27, 28 and 29.

During an initial acquisition phase, the operator is optionally provided with a preselected name of each item of street furniture 17, 19, 21, 22, 23, 25, 26, 27, 28, and 29 as it is imaged and labels each of the items with a name.

A menu, such as that depicted in Table 1 below, is presented on a touch sensitive display 36 of a portable device, laptop or palm pilot 14. The menu provides the operator with options for selection and naming each item of street furniture 17, 19, 21, 22, 23, 25, 26, 27, 28, and 29 detected by the system or an option to verify a preselected or preassigned name or choice of names or short hand codes as previously described. This may be done using a touch sensitive display or with a mouse and microphone.

For example, the menu might offer the choices for the items of street 17, 19, 21, 22, 23, 25, 26, 27, 28, and 29 which may be presented as a table or menu on the screen 16 as depicted in Table 1 below.

Item of Street Furniture Short Hand Code CSV digital ID Unique identifier GPS location Lamp post LP 111 #A1263537 X1, Y1 Telegraph pole TP 112 #A1963568 X2, Y2 Sign post SP 113 #A1863527 X3, Y3 Bus Lane BL 211 #B2601453 X4, Y4 Railway Bridge RB 214 #C5375485 X5, Y5

Table 1

Subsequently the name of the street furniture 17, 19, 21, 22, 23, 25, 26, 27, 28, and 29 and their unique identifier codes are compiled and transmitted as digital data to a data network 66 which once relayed to a database 50 may be accessed by users from remote terminals accessing databases in a control room or secure data centre.

Further choices may be generated and provided by using a neural network and artificial intelligence systems which in combination with computer imaging, pattern recognition and machine learning systems are adapted to interpret and label automatically images obtained by the cameras of the camera system 30 so that an operator merely has to affirm the identity or name of an item.

A unique identifier or code 24 is optionally also marked on an adhesive label or integrated in some other way with some or all items of street furniture so that for example pulsed infra-red (IR) radiation from transponders 31, 32, 33 and 34 are able to record the unique identifier or code 24 and the distance of an item of street furniture and image recognition device or optical character reader (OCR) or some other automated sensor, is operable to interpret the unique identifier and associate it with the particular item of street furniture and thereby generate the street furniture digital identity data which unambiguously identifies each item of street furniture rather like a finger print.

Use of the invention ensures unambiguous identification of each item of street furniture 17, 19, 21, 22, 23, 25, 26, 27, 28, and 29 and its precise location. In use image data is provided to an operator who is enabled to verify quickly the name of each item of street furniture from a preselected list of names or types of street furniture as well as its state of repair. The image recognition software having advantageously already limited the number of choices of names for an item of street furniture from a longer menu. Alternatively, names of street furniture are selected from a predefined menu or in the event that image recognition is unable to identify a particular item of street furniture a custom name for the item of street furniture may be input by the operator.

Frames of captured image data are digitised, time and date stamped and labelled with a GPS marker. Digitised images together with other data are then transmitted by transmitter 42 to a database via a base station 56 and network 66 or directly to a remote receiver 43 where they are stored and uploaded to a database 50. One stored on the database data relating to digital images of surroundings may be obtained and manipulated. Thus, a database 50 may be interrogated and viewable by SQL or similar imaging processing software that obtains stored data and presents the image data in a continuous format on a monitor 46.

Referring to Figures 1 and 3 a global position system (GPS) 40 provides location data of the absolute whereabouts of the pulsed radiation transponders 31, 32, 33 and 34 so that location data is associated with each digital image frame.

The pulsed radiation transponders 31, 32, 33 and 34 thereby derive distance of items of street furniture at locations along a route taken by the vehicle 12. The GPS time, date and place stamps each image. Image data is stored on an on-board data store 29 and linked than item of street furniture from a unique identifier; and a transmitter transmits image frames, GPS data and the unique identifier as digital data to a remote database which stores the location, GPS data, and unique identifier and an operator defined name for the item of street furniture.

Outputs from digital cameras 30a, 30b 30c and 30d provide a continuous digital image data of surroundings. Global position system (GPS) 40 provides location data which is associated with each image frame and transmitted therewith. Thus the data showing the type and nature and location of street furniture, when combined with the unique identifier 24, can be obtained, digitised and stored continually (day and night) and prided as incremental updates to a database.

Image recognition software, running on computer 48a, is operable to derive an identity of each item of street furniture from its unique identifier 24. If no unique identifier 24 is present or detected, a suggestion of the nature, type, name, condition or type of street furniture is provide as a menu for the operator to select or verify, as described above. In the event that no identity or name can be provided, a digital flag is associated with a file for investigation at a later time. This may involve sounding an alert to the vehicle driver to halt at a particular location and investigate an item for naming or inspecting.

The transmitter 42 transmits image frames, GPS data and the unique identifier 24 as digital data to a remote database 50 which stores the GPS location data and the street furniture digital identity data.

Referring to Figure 1 there is shown a diagrammatic overview of one embodiment of a monitoring and recording system 10 which was obtained on a first earlier date. A unique identifier 24 is associated with the lamp post 28 as the operator is provided with a short hand code 'LP' which may be presented as 111, as shown in the Table. However, the system 10 recognises that this particular lamp post 28 is at a unique position from its GPS location coordinates and so assigns the unique identifier #A1263537 as a CSV. In use an operator defined name may also be input by an operator, for example using a touch sensitive display or panel 36 on a portable device, laptop or palm pilot 14 which provides the operator with selection options in order to verify a preselected name and thereby generate the unique identifier.

Referring to Figure 2 there is shown a print out of a screen shot of a schedule of recorded items of interest (assets) and their locations. The list has been derived from images obtained from the cameras 30a, 30b 30c and 30d and from pulsed radiation transponders 31 to 34, and occasionally augmented by an operator command. This data is transmitted and stored on the database 50 from where it can be retrieved by a user (not shown) and printed.

By way of example some time after acquisition of the data shown in Figure 1, the image of Figure 3 illustrates a diagrammatic overview of the scene shown in Figure 1 and obtained on a second later date.

In the scene shown in the example depicted in Figures 1 and 3, street lamp 28 has an RFID 18 transmitter mounted on its upper surface which is powered by a solar array (not shown). The RFID tag 18 transmits a pulsed coded signal which includes the GPS location and identity and status of each item of street furniture. The coded signal is detected and decoded and associated with the street lamp and a record of its status is associated with its digital identity data thereby providing a record of its location and an indication of its status/condition.

Figure 4 shows typical screen output which is a schedule of imaged and recorded items of and their locations. In particular the absence of pothole 20 which was present in a previously acquired image (Figure 1). On comparison of the two images in Figures 1 and 3 it can be deduced that the pothole has been repaired. This may be important from a perspective of providing an audit of road works/road repairs Figure 5 shows a print out of a verification report corresponding to the typical screen output which is a schedule of recorded items of interest and their locations. There is shown a table or print out which has been obtained by comparing either two images or a catalogue of items represented as digital data or by both techniques in order to arrive at a discrepancy or difference.

The image in Figure 1 is of a scene that was obtained on 20 November 2017 as can be seen from the data stamp at the top left hand corner of the image. The image in Figure 3 is shown as having been obtained on 2 February 2018, again from the date stamp at the top left hand corner of the image.

Data from digital scanner 48b and image recognition system 60 are compared in a comparator 49 which compares scanned image data and data provided by the image recognition system 60, from images obtained at different times (Figures 1 and 3) in order to assess the status of an asset and determine for example if the asset (item of street furniture) is missing or damaged or faulty. The example shown in Figures 1 and 3 shows a pothole 20 (in the bottom left hand corner) as being present on 2 November 2017. In the view shown in Figure 3 the pothole is shown as having been repaired and therefore a completion report may be raised and transmitted to a local authority or customer or to an auditing authority.

Figure 6 shows an overview of one example of a monitoring, and recording system 10 which is optionally used in surveillance. Remote updating of a digitised street map, stored as an asset register 44 on the database 50, may be made via input terminals (not shown) for example in local planning departments or council maintenance offices in order to provide an update obtained from other sources enables an image obtained from data from the digital scanner 48b and image recognition system 60 are compared in a comparator 49 which compares scanned image data and data provided by the image recognition system 60, in order to indicate a difference, updated event or feature for confirming, for example whether a piece of maintenance or repair work has been carried out. Optionally an automated procedure may be provided for this purpose with an option of a human audit or 'sign off' being provided.

Figure 7 shows an alternative embodiment of a system that operates using a LiDAR system 35 to detect and measure distances to solid objects. LiDAR is a surveying technique in which is used to measure distances from a source to a target object. This carried out typically by illuminating the target with a pulsed laser beam and measuring reflected pulses with a suitable sensor. Differences in return times of pulses and data derived from interference of waves is used to determine the distance to the target. LiDAR may thereby be used to create a digital three-dimensional (3-D) representation of the target and/or an environment monitored by the system.

By employing a LiDAR system 35 with a GPS location system, it is now possible to identify an object (asset) and its location and record this data on a digital map. Previously this was a multi stage operation involving a surveyor (individual or an automated system) locating an asset, determining what type of asset it is, optionally determining the asset's condition, and then recording this data in a database. Needless to say, this entire process was labour intensive and therefore expensive. There was also a risk that errors would be introduced into the data that was captured and recorded.

Ideally image and location data which is derived using the system is stored on a database or optionally on a bespoke database with machine learning capability, so that an existing object can be associated or correlated with image and location data derived from the system. This data enables a real time graphics feature to be provided to a user so that the user is able to "snap to map", for example by clicking a mouse, and so identify (or verify) an existing object and confirm its exact location with respect to a closest GPS location in order to verify data on the database or initiate a query for an operator to verify later.

An advantage with the invention is that because it operates automatically multiple detections may be performed simultaneously using two or more, and preferably six, imaging systems which operate in parallel. Preferably software is operative to knit together images and/or location data, thereby providing a virtual image environment, for example for a remote inspection team and/or to compress data in order to enhance the data for storage or transmission.

The system may use artificial intelligence, which may include image recognition software, so that certain objects, such as potholes, generate an action warning which in turn may result in a report being generated with a series of check lists or activities, as well as an optional automatic transmission of a short message service (SMS) alert to a maintenance or inspection team; for example, when multiple detections of such certain objects have occurred at the same or at nearby GPS locations or when particular damage or faults are identified.

As in the previous embodiment, the LiDAR system associates a unique asset ID with all items detected, for example, so that a register of assets can be compiled.

Ideally image recognition software enables non-detected items, which have been marked as financial assets, to be flagged and optionally an alert to be issued so that personnel can be despatched to investigate and survey the absence of the marked non-identified item. Alternatively, or additionally, the system may flag detected items which have not been marked as financial assets and may optionally issue an alert so that personnel can be dispatched to investigate survey and authenticate the previously unknown or un-registered asset. As data includes a GPS location, it is also envisaged that software will be employed to generate a preferred route along which investigating personnel may walk, or drive, so as to visits as many locations in a shift as possible.

As images are acquired using a 360-degree camera zooming into and around objects is achievable in real time or later so as to enable detailed inspection of an asset. This inspection is important for repair and maintenance purposes.

Images obtained using the system are ideally watermarked with data and timestamps which include at least the following: date and time of acquisition, local ambient weather and light conditions and GPS location. Watermarking may be imprinted on image as text or another distinguishable mark and/or may be imprinted as encrypted data, such as in bar code or QR-code format.

By using both LiDAR and high definition (HD) 360-degree imaging the user has the ability to serialise all assets that possess a monetary value, such as bicycle racks, street railings; as well as to provide a community service by detecting potholes, litter bins placed on sidewalks which may present as a hazard; or by detecting items of street furniture requiring maintenance and repair. This is achieved in a short time period at relatively low cost.

Most companies that provide asset registers tend to manually count street furniture over a period of months and this tends to cost to be in the hundreds of thousands of pounds. This also means the register will never be accurate as these counts only take place every 10 or so years.

In addition to detecting items of street furniture, the system may also detect, image and locate stationary or mobile other vehicles such as cars 70 or bicycles 72.

It will be appreciated that variation may be made to the above-described embodiments without departing from the scope of the invention as defined by the claims.

For example, the system may monitor fly tipping, unauthorised skips, dumped items of furniture, or abandoned cars which when imaged attract an automatic flag which can be relayed to an inspection team for subsequent inspection, such as during quieter traffic times or at weekends, and/or when time permits for the removal of such items.

Other variations include obtaining images from one or more unmanned aerial vehicles (UAVs) or drones in order to image large car parks and/or places which may have temporarily restricted access or are inaccessible by vehicles, such as a construction site, crowded areas, or a pedestrian zone which might impede traffic movements during busy periods. List

system 12 vehicle 14 on board personal digital assistant 16 on-board microphone 17 tree 18 RFID tag 19 traffic light pothole 21 bus stop 22 manhole cover 23 bin 24 unique identifier bench 26 guard rail or barrier 27 Sign 28 lamp post 29 traffic cones a vehicle mounted 360° camera 31 first pulsed radiation transponder 32 second pulsed radiation transponder 33 third pulsed radiation transponder 34 fourth pulsed radiation transponder LiDAR system 36 touch sensitive display 38 data processor global position system (GPS) 42 radio frequency (RF) transmitter 43 radio frequency (RF) receiver 44 asset register 46 monitor 48a computer 48b digital scanner 49 image comparator or data comparator remote database 56 base station image recognition system 66 data network car 72 cyclist

Claims

Claims 1 A monitoring and recording system for compiling an asset register of items of street furniture comprising: a vehicle mounted camera which derives a plurality of image frames from images; a global positioning system (GPS) which provides location data which is associated with each image frame; an image identifier which identifies items of street furniture and generates street furniture digital identity data; and a transmitter which transmits digitised image frames, their associated GPS location data and the street furniture digital identity data to a remote database whereat the digitised image frames, GPS location data and the street furniture digital identity data are stored.
2 A monitoring and recording system according to claim 1 wherein an automated means generates street furniture digital identity data from an operator defined name.
3. A system according to claim 1 or 2 wherein the imager is adapted to obtain a 360° panoramic image of surroundings.
4. A system according to any preceding claim includes means for determining a distance of an object of street furniture from a point on the vehicle.
A system according to claim 4 includes at least one source of pulsed radiation, a detector arranged to receive a reflected radiation signal and a counter for determining from the source and the reflected signal the distance and bearing of the object of street furniture from the vehicle.
6 A system according to any preceding claim wherein a digital scanner scans image data and a comparator compares scanned image data in order to assess the status of an asset or if an asset is missing or damaged or faulty and to record the status or condition of the asset.
7. A system according to any preceding claim includes a menu operable by an operator to provide a label for an item of street furniture.
8 A system according to claim 7 wherein an operator defined name includes a descriptor or word or code derived from an oral description provided by an automatic voice recognition system.
9 A system according to claim 8 wherein the operator defined name includes a digital image or other descriptor provided by the image identifier.
10.A system according to any of claims 7 to 9 wherein the menu is operable with a predictive text generator.
11.A system according to any preceding claim wherein the vehicle is an autonomous vehicle, such as a driverless vehicle or unmanned aerial vehicle (UAV) or a drone.
12.A system according to any of claims 4 to 11 wherein the means for determining the distance of the object from the vehicle includes a source of pulsed infra-red (IR) radiation, a detector arranged to receive a reflected signal and a counter for determining from the source and reflected signal a 'time of flight' of a radiation.
13.A system to any preceding claim includes a facial recognition system.
14.A system to any preceding claim includes a means for post processing data and a neural network
15.A system to any preceding claim includes a secure communication link that transmits data in an encrypted form to a remote location.
16.A system according to any preceding claim includes a reporting tool for generating a maintenance report or report for inspection of items of street furniture.
17.A method of compiling an asset register of items of street furniture comprising the steps of: operating a vehicle mounted camera in order to derive a plurality of image frames; generating a continuous digital image of surroundings; associating the assets in the image with a location derived from a global position system (GPS); providing location data which is associated with each image frame; deriving from the images identity of items of street furniture and associating with each item of street furniture a unique identifier; and transmitting image frames, GPS data and the unique identifier as digital data to a remote database.
18.A method according claim 17 includes the steps of: enabling a user to access the data.
19.A method according to claim 17 or 18 comprising the steps of: operating the camera in order to obtain a 3600 degree panoramic image of surroundings.
20.A method according to any of claims 17 to 19 comprising the steps of: determining a distance of an object of street furniture from a point on the vehicle.
21.A method according to any of claims 17 to 20 comprising the steps of: employing a source of pulsed radiation and at least one receiver in order to determine the distance of the object of street furniture from the vehicle.
22.A method according to any of claims 17 to 21 comprising the steps of: scanning images and comparing one image with an earlier obtained image in order to determine if an asset is missing or damaged or faulty.
23.A method according to any of claims 17 to 22 comprising the steps of: enabling an operator to define a name by employing a voice recognition system.
24.A method according to any of claims 17 to 23 comprising the steps of: employing a neural network (CNN) to process image data so that frames of data are compared with a series of data derived from verified images.
25.A method according to claim 24 wherein the CNN determines if an object that cannot be readily identified is detected by matching the object with similar size objects in the tracking history using a series of histograms of previous detection events.
26.A method according to claim 25 wherein the CNN wherein if a matching criteria exceed 90% the object is assigned the identification of a best match.
27.A method according to claim 26 wherein a verification is performed to check the location of the best match.