GB2624627A - A system and method of detecting curved mirrors within an image - Google Patents

Abstract

A system (100) and computer implemented method of detecting curved mirrors 814 within an image 802 are provided, characterised in that the method comprises the steps of: applying a machine learned algorithm on the image to identify a curved mirror candidate 800 within the image; and applying an anomaly detection algorithm on the identified curved mirror candidate to verify that the candidate is a curved mirror. The anomaly detection algorithm could be based on a Deep Autoencoder Gaussian Mixture Model (DAGMM). The mirror candidate could be represented by a point 808,810 on a transformation map 812 wherein the verification step considers the distance of such points from a cluster 816,818 that represents traffic signs. The method could find application in an Advanced Driver Assistance System (ADAS) and be useful at road intersections or junctions or at sharp bends or curves in the road where curved (convex) mirrors are sometimes deployed.

Description

A SYSTEM AND METHOD OF DETECTING CURVED MIRRORS WITHIN

AN IMAGE

TECHNICAL FIELD

The present disclosure relates broadly to a system and computer implemented method of detecting curved mirrors within an image.

BACKGROUND

Road traffic accidents tend to occur at or around areas of road intersections, and at road positions such as those having sharp curves, due to poor visibility of traffic from different directions to a vehicle. In particular, the number of accidents at road intersections without traffic lights is generally higher than those with traffic lights.

To reduce the number of accidents at road intersections especially those without traffic lights, road safety mirrors, e.g., curved mirrors are typically installed at road intersections and at road positions such as those having sharp curves. Such curved mirrors may provide the vehicle with a view of blind spots and the like that exist at these road intersections and positions, thereby allowing road users to confirm safety and avoid or reduce the occurrence of accidents. As would be appreciated, the curved mirrors are external to the vehicles.

It has been proposed for image capturing devices, such as video cameras, to be provided on vehicles to detect curved mirrors on the road, e.g., at road intersections and positions such as those having sharp curves. Curved mirror detection serves an important function of blind spot detection.

It has been recognised that improvements to existing Advanced Driver Assistance Systems (ADAS) are desired. In particular, existing objection recognition techniques can be enhanced, in order to meet real-time curved mirror recognition for use in ADAS.

For example, existing ADAS are unable to quickly detect curved mirrors in real time. Moreover, the accuracy of existing systems is also questionable, as many detected curved mirrors are false positive detections. Accurate detection of a curved mirror is difficult because a curved mirror does not have any fixed pattern, as it merely reflects an image of the surroundings where it is situated.

Furthermore, the inventors have recognised that curved mirror detection calculation speed may also influence risk prediction to collision. As calculation load increases with increasing complexity of detection systems, the curved mirror detection calculation speed tends to be slower. One problem that may arise is that current curved mirror detection may take a substantial amount of time before curved mirror(s) are detected.

Thus, there is a need for a system and computer implemented method of detecting curved mirrors within an image, which seek to address or at least ameliorate one of the above problems.

SUMMARY

In accordance with an aspect of the present disclosure, there is provided a computer implemented method of detecting curved mirrors within an image, characterised in that the method comprises the steps of: applying a machine learned algorithm on the image to identify a curved mirror candidate within the image; and applying an anomaly detection algorithm on the identified curved mirror candidate to verify if the curved mirror candidate is a curved mirror.

The anomaly detection algorithm of the method as disclosed herein may be based on a Deep Autoencoder Gaussian Mixture Model (DAGMM).

The curved mirror candidate of the method as disclosed herein may be verified as a curved mirror if it is unlikely to be a traffic sign.

The anomaly detection algorithm of the method as disclosed herein may comprise applying an autoencoder function on image data associated with the curved mirror candidate to obtain a point on a transformation map, said point representative of the curved mirror candidate on the transformation map, verifying the curved mirror candidate as a curved mirror if the point is beyond a threshold distance away from a cluster representative of a traffic sign on the transformation map.

There may be multiple clusters representative of traffic signs, the curved mirror candidate is verified as a curved mirror if the point is beyond a threshold distance away from each of the clusters.

The machine learned algorithm may be trained using bounding boxes with parameters that are optimised for curved mirrors.

The parameters that are optimised for curved mirrors may comprise height of a bounding box, width of the bounding box, horizontal coordinate of the centre point of the bounding box, vertical coordinate of the centre point of the bounding box, and height to weight ratio of the bounding box.

The method may further comprise a step of acquiring the image using an image capturing device.

The step of obtaining the image may comprise obtaining a plurality of images over different time instances.

The image may be a real time image.

In accordance with another aspect of the present disclosure, there is provided a system for detecting curved mirrors within an image, the system comprising, an image capturing device; and an electronic control unit coupled to the image capturing device; characterised in that the image capturing device is configured to acquire the image; and the electronic control unit is configured to apply a machine learned algorithm on the image to identify a curved mirror candidate within the image; and to apply an anomaly detection algorithm on the identified curved mirror candidate to verify if the curved mirror candidate is a curved mirror.

The anomaly detection algorithm of the system as disclosed herein may be based on a Deep Autoencoder Gaussian Mixture Model (DAGMM).

The curved mirror candidate of the system as disclosed herein may be verified as a curved mirror if it is unlikely to be a traffic sign.

The anomaly detection algorithm of the system as disclosed herein may comprise applying an autoencoder function on image data associated with the curved mirror candidate to obtain a point on a transformation map, said point representative of the curved mirror candidate on the transformation map, verifying the curved mirror candidate as a curved mirror if the point is beyond a threshold distance away from a cluster representative of a traffic sign on the transformation map.

In accordance with another aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon instructions for instructing a processing unit of a system to execute a computer implemented method of detecting curved mirrors within an image, characterised in that the method comprises the steps of: applying a machine learned algorithm on the image to identify a curved mirror candidate within the image; applying an anomaly detection algorithm on the identified curved mirror candidate to verify if the curved mirror candidate is a curved mirror.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments of the invention will be better understood and readily apparent to one of ordinary skill in the art from the following written description, by way of example only, and in conjunction with the drawings, in which: FIG. 1 is a schematic block diagram for illustrating a system for detecting curved mirrors within an image in an example embodiment.

FIG. 2 is a schematic flow chart for illustrating a computer implemented method of detecting curved mirrors within an image in an example embodiment.

FIG. 3A to FIG. 3C show a series of images for illustrating data set preparation for road safety mirror, e.g., curved mirror detection using a machine learned algorithm in an example embodiment.

FIG. 4 is a one frame shot of verification result of road safety mirror detection in an example embodiment.

FIG. 5A and FIG. 5B show a series of images representing examples of true positive images in the form of road safety mirrors detected by an example embodiment.

FIG. 6A to FIG. 6D show a series of images representing examples of false positive images in the form of road traffic signs detected by an example embodiment.

FIG. 7 is a transformation map comprising data points of curved mirror candidates identified within an image in an example embodiment.

FIG. 8 is a schematic diagram illustrating an improved anomaly detection method in an example implementation.

FIG. 9A and FIG. 9B are photographs showing the effectiveness of the improved anomaly detection method in an example implementation.

FIG. 10 is a schematic diagram and a photograph showing road safety mirror detection at a T crossing in an example embodiment.

FIG. 11 is a chart showing dimensions of curved mirrors relative to dimensions of default boxes for model learning in an example embodiment.

FIG. 12 is a schematic drawing of a computer system suitable for implementing an example embodiment.

DETAILED DESCRIPTION

Example, non limiting embodiments may provide a system and computer implemented method of detecting road safety mirrors, e.g., curved mirrors within an image.

In various embodiments, the term "curved mirror" as used herein broadly refers to a mirror with a curved reflecting surface. In various embodiments, the curved mirror comprises a curved mirror area. In various embodiments, the surface is a convex (i.e., bulging outwards) surface. In various embodiments, the curved mirror is a convex mirror.

In various embodiments, the term "image" or "image data" as used herein broadly refers to any content or data that can be rendered for viewing by a user. In various embodiments, the image or image data comprises computer readable data.

In various embodiments, the image or image data may be converted into computer readable data. In various embodiments, the image or image data may be converted into an appropriate or compatible format to be used by components of the system and method as disclosed herein. For example, an image or image data may be a frame from a video, a portion of the frame from the video, a still image, a portion of the still image, or the like.

FIG. 1 is a schematic block diagram for illustrating a system 100 for detecting curved mirrors within an image in an example embodiment. The system 100 is disposed onboard a vehicle. The system 100 comprises an image capturing device 102 and a processing unit 104 coupled to the image capturing device 102.

In the example embodiment, the image capturing device 102 is configured to acquire the image. The image capturing device 102 of the vehicle may be a camera or a video camera that is disposed in a vicinity of a rear view mirror at a front of the vehicle body or on a front grille of the vehicle. The image capturing device 102 is disposed so as to have an image capture area at a predetermined angle toward the front of the vehicle.

In the example embodiment, the processing unit 104 may be configured to control the functions of the components of the system 100. In the example embodiment, the processing unit 104, e.g., an electronic control unit (ECU) of the vehicle, is configured to apply a machine learned algorithm on the image to identify a curved mirror candidate within the image and to apply an anomaly detection algorithm on the identified curved mirror candidate to verify if the curved mirror candidate is a curved mirror.

In the example embodiment, the anomaly detection algorithm may be based on a Deep Autoencoder Gaussian Mixture Model (DAGMM). The anomaly detection algorithm may be configured to verify the curved mirror candidate as a curved mirror if it is unlikely to be a traffic sign. For example, the anomaly detection algorithm may comprise applying an autoencoder function on image data associated with the curved mirror candidate to obtain a point, e.g., data point, on a transformation map, said point representative of the curved mirror candidate on the transformation map, and verifying the curved mirror candidate as a curved mirror if the point is beyond a threshold distance away from a cluster representative of a traffic sign on the transformation map. In the example embodiment, there may be multiple clusters of points that are representative of traffic signs. The curved mirror candidate is verified as a curved mirror if the point is beyond a threshold distance away from each of the clusters.

In the example embodiment, the processing unit 104 may be further coupled to an object detection unit 106 and the objection detection unit 106 may be further coupled to an action unit 108. The objection detection unit 106 may be configured to perform object detection on curved mirror(s) detected by the processing unit 104. The action unit 108 may be configured to receive one or more instructions from the object detection unit 106 and may be configured to, for example but not limited to, activate a braking function of the vehicle, and/or activate a warning system to a user of the vehicle, and/or taking over steering control, or take no action.

During operation, a cross road or a traffic intersection may be identified by the image capturing device 102. In some example embodiments, map information may be provided to the processing unit 104 such that the processing unit 104 may use the map information, e.g., matching a map against Global Positioning System (GPS) coordinates, to identify a cross road or a traffic intersection. At such a cross road or traffic intersection, the processing unit 104 applies a machine learned algorithm on the image captured by the image capturing device 102 to identify a curved mirror candidate within the image and thereafter applies an anomaly detection algorithm on the identified curved mirror candidate to verify if the curved mirror candidate is a curved mirror.

FIG. 2 is a schematic flow chart 200 for illustrating a computer implemented method of detecting curved mirrors within an image in an example embodiment. At step 202, a machine learned algorithm is applied on the image to identify a curved mirror candidate within the image. At step 204, an anomaly detection algorithm is applied on the identified curved mirror candidate to verify if the curved mirror candidate is a curved mirror. The method of detecting curved mirrors within an image may be implemented using the system 100 of FIG. 1. For example, the identification and verification of a curved mirror candidate within the image may be performed by the processing unit 104.

In the example embodiment, the method may further comprise a step of obtaining/acquiring the image using an image capturing device. The image capturing device may be a camera or a video camera. The image capturing device may be disposed onboard a vehicle and is disposed in a vicinity of a rear view mirror at a front of the vehicle body or on a front grille of the vehicle. The step of obtaining the image may comprise obtaining a front view image towards the front of the vehicle. The step of obtaining the image may comprise obtaining a plurality of images over different time instances. For example, the plurality of images over different time instances may be a series of frames from a video in a time sequential manner, each frame representing an image taken at a particular time instance. The image may be a real time image. The image may be a frame taken from a real time video.

In the example embodiment, there may be no curved mirror candidate identified within the image. In the example embodiment, there may be one or more curved mirror candidates identified within the image. In the example embodiment, there may be no curved mirror verified from the one or more curved mirror candidates identified within the image. In the example embodiment, there may be one or more curved mirrors verified from the one or more curved mirror candidates identified within the image.

In the example embodiment, the machine learned algorithm may be a real time object detection system. The real time object detection system may be a one pass object detection system where the image is only parsed once. The real time object detection system may be a You Only Look Once (YOLO) v3 system. In the example embodiment, the machine learned algorithm may be trained using bounding boxes with parameters that are optimised for curved mirrors. The parameters that are optimised for curved mirrors may comprise height of a bounding box, width of a bounding box, horizontal coordinate of the centre point of the bounding box, vertical coordinate of the centre point of the bounding box, and height to weight ratio of the bounding box.

In the example embodiment, the anomaly detection algorithm may be based on the likelihood that the curved mirror candidate is not a traffic sign. The anomaly detection algorithm may be based on a Deep Autoencoder Gaussian Mixture Model (DAGMM). In the example embodiment, the anomaly detection algorithm may comprise a step of applying an autoencoder function on image data associated with the curved mirror candidate to transform the candidate into a different domain, and obtain a point on a transformation map, said point representative of the curved mirror candidate on the transformation map; and a step of verifying the curved mirror candidate as a curved mirror if the point is beyond a threshold distance away from a cluster representative of a traffic sign on the transformation map. In the example embodiment, there may be multiple clusters of points that are representative of traffic signs. The curved mirror candidate is verified as a curved mirror if the point is beyond a threshold distance away from each of the clusters.

FIG. 3A to FIG. 3C show a series of images for illustrating data set preparation for road safety mirror, e.g., curved mirror detection using a machine learned algorithm in an example embodiment. In the example embodiment, the machine learned algorithm used for road safety mirror detection is based on a real time object detection system known as You Only Look Once (YOLO) v3. The latest version as of 2019 has been selected in the example embodiment.

FIG. 3A shows an original image taken by an image capturing device in its original pixel size. For learning data, high resolution pixel image (approximately about 12 million pixels) may be used. As shown in FIG. 3A, Region of Interest (ROI) information of road safety mirror is included in the original image. ROI means a region of interest where the area of a target object, i.e., road safety mirror, is enclosed in a square 302 in FIG. 3A. FIG. 3B shows a defined shrink pixel size image which is scaled down from the original image of FIG. 3A. FIG. 3C shows an image in the Va*b* colour space to be used for the data set for learning. FIG. 3C is converted from the defined shrink pixel size image of FIG. 3B.

In the example embodiment, the original image of FIG. 3A is scaled to the defined shrink pixel size of FIG. 3B and converted to the Va*b* image of FIG. 30. As would be appreciated by a person skilled in the art, an RGB colour space is based on red, green, blue colours. In the Va*b* colour space, the capital letter V means luminosity and the small letters a*, b* means complementary colours. The inventors have recognised that the Va*b* colour space seems to be closer to human visual. In the RGB colour space, it is sometimes hard to distinguish colour area depending on brightness situation. On the other hand, it may be possible to distinguish the colour area like human eyes in the Va*b* colour space. Thus, in the example embodiment, the original image is converted to the Va*b* colour space for use as the input of YOL0v3 deep learning method.

FIG. 4 is a one frame shot of verification result of road safety mirror detection in an example embodiment. A public road running test was executed using example embodiments of the system and method disclosed herein. In the example implementation, a low resolution camera (approximately about 0.32 million pixels) was used as the image capturing device, i.e., not the high resolution camera used for data collection. In the example implementation, a NVIDIA Jetson Xavier module was used as the processing unit, e.g., controller. Road safety mirrors detected by the example implementation are enclosed by squares 402 and 404 as shown in FIG. 4.

FIG. 5A and FIG. 5B show a series of images representing examples of true positive images in the form of road safety mirrors detected by an example embodiment. FIG. 6A to FIG. 6D show a series of images representing examples of false positive images in the form of road traffic signs detected by an example embodiment.

Notwithstanding the use of the low resolution camera (approximately about 0.32 million pixels), the results showed good precision of more than 95% as described in Table 1 and Equation (1).

TP 696 = 95.2% Equation (1) Precision = TP+FP 696+35 Table 1: Verification result of road safety mirror detection True positive (TP) False Positive (FP) 696 detections 35 detections (See FIG. 5A and FIG. 5B for examples of true positive images) (See FIG. 6A to FIG. 6D for examples of false positive images) As would be appreciated from the results in Table 1, there are some FP in the results of road safety mirror detection.

Example embodiments of the system and method as disclosed herein further provide a countermeasure to the issue of false positive results and these will be described with reference to FIG. 7 to FIG. 10 below.

FIG. 7 is a transformation map 700 comprising data points, e.g., 702a and 704a, of curved mirror candidates identified within an image in an example embodiment.

As shown in FIG. 7, the transformation map 700 comprises a first data point 702a representative of a first traffic sign 702b, a second data point 704a representative of a second traffic sign 704b, and a cluster 706 of data points representative of e.g., a first curved mirror 708 presented as a circle and a second curved mirror 710 presented as an ellipse on the image. Each data point on the transformation map 700 is obtained by applying an anomaly detection method on image data associated with a curved mirror candidate. The first data point 702a and second data point 704a are examples of false positive results that do not represent a curved mirror.

The anomaly detection method may be used as a countermeasure to overcome the issue of false positive results. An example of an anomaly detection method is the Deep Autoencoder Gaussian Mixture Model (DAGMM). By using the Gaussian Mixture Model, a data point may be obtained for each object and a cluster of data points may be obtained for a plurality of objects on the transformation map 700.

However, the inventors have recognised that even with existing anomaly detection method such as DAGMM, it is difficult to define a distribution of road safety mirror in comparison with traffic sign due to the dynamic and inconsistent image reflected from a road safety mirror. Unlike a traffic sign which typically displays a fixed pattern, a curved mirror does not have any fixed pattern because it reflects an image of the surroundings where it is situated. Accordingly, there is no consistent pattern displayed by a curved mirror.

To overcome the difficulty of differentiating between true positive results from false positive results, the inventors have devised an improved anomaly detection algorithm.

FIG. 8 is a schematic diagram illustrating an improved anomaly detection method in an example implementation. In the example implementation, the improved anomaly detection method is based on the DAGMM.

In the example implementation, a curved mirror candidate 800 is identified from an image 802. The curved mirror candidate 800 may be detected by applying a machine learned algorithm, e.g., a deep learning method YOL0v3, on the image 802. The curved mirror candidate 800 may either be a road traffic sign (i.e., false positive) or a curved mirror (i.e., true positive). There may be one or more curved mirror candidates 800 identified from the image 802. For example, there are two curved mirror candidates identified in the image 802, as represented by two boxes 804 and 806.

Thereafter, the improved anomaly detection method is implemented by a step of applying an autoencoder function on image data associated with the curved mirror candidate 800 to obtain a data point (e.g., 808, 810) on a transformation map 812, said data point (e.g., 808, 810) representative of the curved mirror candidate 800 on the transformation map 812; and a step of verifying the curved mirror candidate 800 as a curved mirror 814 if the data point (e.g., 808) is beyond a threshold distance away from a cluster that is representative of a traffic sign on the transformation map 812.

As shown in FIG. 8, there are multiple clusters (e.g., 816, 818) that are representative of traffic signs 820. If the data points within the clusters (e.g., 816, 818) represent traffic signs, data points that are close to the clusters are likely to be traffic signs, while data points that are far from the clusters are likely to be road safety mirrors, e.g., curved mirrors. Accordingly, in the example implementation, the distance of a data point to the clusters representing traffic signs is calculated. If the distance of a data point to the clusters representing traffic signs is over a defined threshold, the curved mirror candidate is regarded as a road safety mirror. If the distance of the data point to the clusters representing traffic signs is within the defined threshold, the curved mirror candidate is regarded as not a road safety mirror. In other words, the curved mirror candidate 800 is verified as a curved mirror 814 if the data point e.g., 808 is beyond a threshold distance away from each of the clusters (e.g., 816, 818). In general, the threshold distance used to verify the curved mirror candidate may depend on factors such as the resolution of the camera used.

Accordingly, in various example implementations, the threshold distance for verifying the curved mirror candidate may be tuned and optimised to maximise true positive results of detection and to minimise false positive results of detection of curved mirrors.

FIG. 9A and FIG. 9B are photographs showing the effectiveness of the improved anomaly detection method in an example implementation. FIG. 9A shows an image taken from part of a movie (i.e., sequence of frames/images) result without the improved anomaly detection method e.g., DAGMM applied. As anomaly detection was not applied, there were 31 counts of false positive detected in this movie. On the other hand, FIG. 9B shows an image taken from part of the same movie result with the improved anomaly detection method e.g., DAGMM applied. As shown in FIG. 9B, there was no false positive detected in the same movie.

Accordingly, the effectiveness of the improved anomaly detection method as disclosed herein was proven by this movie.

FIG. 10 is a schematic diagram and a photograph showing road safety mirror detection at a T crossing in an example embodiment. In the example embodiment, road safety mirror detection by a method applying the improved anomaly detection algorithm DAGMM has been confirmed after a subject vehicle 1000 approaches a T crossing. Two road safety mirrors were identified as shown by boxes 1002 and 1004. Accordingly, the system and method of detecting curved mirrors as disclosed herein have been demonstrated to be effective in reducing false positive results in road safety mirror detection and that road safety mirror can be successfully detected.

FIG. 11 is a chart showing dimensions of curved mirrors relative to dimensions of default boxes for model learning in an example embodiment. In the example embodiment, a machine learned algorithm may be trained using bounding boxes with parameters that are optimised for curved mirrors. The parameters that are optimised for curved mirrors may comprise height of a bounding box, width of a bounding box, horizontal coordinate of the centre point of the bounding box, vertical coordinate of the centre point of the bounding box, and height to weight ratio of the bounding box.

In the example embodiment, a deep learning approach is utilised to recognise curved mirrors. Instead of using a normal default box for model learning, the machine learning model may use an optimised curved mirror size from the normal default box. The optimal mirror size is determined based on testing results, in which an image capturing device e.g., camera detects a curved mirror on a public road. The optimal mirror size may depend on factors such as the resolution of the camera used. In various example embodiments therefore, the mirror size may be tuned and optimised accordingly. In the example embodiment as shown in FIG. 11, the optimal mirror size is about 0.2 relative to the whole image size. In comparison to conventional normal default box usage, curved mirror detection from a greater distance is achievable by adopting optimised curved mirror size. In the example embodiment, the use of an optimised curved mirror size achieves a faster detection time of 21.5 milliseconds (ms) while maintain an equivalent level of accuracy. An example of the machine learning model may be a Single Shot Multibox detector (S SD).

Table 2: Accuracy and detection time of different methods of detection Method F value Detection time (ms) Normal 0.433 29.6 Minimised 0.447 21.5 In an example implementation using the optimised curved mirror size, a crossing road is first detected by an image capturing device camera. Optionally, map information may be useful. For example, map information may be provided such that the map information is matched against Global Positioning System (GPS) coordinates, to identify a cross road or a traffic intersection. Thereafter, at the crossing road, the machine learned algorithm is applied to identify/detect curved mirror area.

In an example embodiment, it is described that YOL0v3 is used as a real time detection system for detection of curved mirrors candidates. YOL0v3 is a real time object detection system that applies a single neural network to a full image. This network divides the image into regions and predicts bounding boxes and probabilities for each region. These bounding boxes are weighted by the predicted probabilities. The YOL0v3 system has several advantages over classifier-based systems. It looks at the whole image at test time so its predictions are informed by global context in the image. It also makes predictions with a single network evaluation unlike systems like R-CNN which require thousands for a single image.

This makes it extremely fast, more than 1000 times faster than R-CNN and 100 times faster than Fast R-CNN. Further information about YOL0v3 may be found in the publication "Redmon, Joseph & Farhadi, Ali. (2018). YOL0v3: An Incremental Improvement.", which is incorporated herein by reference. An example of YOL0v3 may be found in "Ammar A, Koubaa A, Ahmed M, Saad A, Benjdira B. Vehicle Detection from Aerial Images Using Deep Learning: A Comparative Study. Electronics. 2021; 10(7): 820" which is incorporated herein by reference, wherein an example of the architecture of YOL0v3 may be found at least in Section 3.2.1 and Figure 2, and an example of the training of YOL0v3 may be found at least in Section 10 4.2.

A first example of a dataset that may be used to train the YOL0v3 machine learning model is the COCO dataset available at https://cocodataset.orgMhome which is a large-scale object detection, segmentation, and captioning dataset and a second example of a dataset that may be used to train YOL0v3 machine learning model is the CIFAR-10 or CIFAR-100 dataset available at http://www.cs.toronto.eduFkrizicifar.html which are labelled subsets of the 80 million tiny images dataset. It is contemplated that any other suitable dataset may be used.

In an example embodiment, it is described that DAGMM is used as an anomaly detection algorithm to verify if the curved mirror candidate is a curved mirror. DAGMM utilises a deep autoencoder to generate a low-dimensional representation and reconstruction error for each input data point, which is further fed into a Gaussian Mixture Model (GMM). Instead of using decoupled two-stage training and the standard Expectation-Maximization (EM) algorithm, DAGMM jointly optimises the parameters of the deep autoencoder and the mixture model simultaneously in an end-to-end fashion, leveraging a separate estimation network to facilitate the parameter learning of the mixture model. The joint optimization, which well balances autoencoding reconstruction, density estimation of latent representation, and regularization, helps the autoencoder escape from less attractive local optima and further reduce reconstruction errors, avoiding the need of pre-training. Further information about DAGMM may be found in the publication "Long, B., Song, Q., Min, M., Cheng, W., Lumezanu, C., Cho, D., & Chen, H. (2018). Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection. ICLR.", which is incorporated herein by reference.

One example of a dataset that may be used to train the DAGMM machine learning model is the KDDCUP99 10 percent dataset from the UCI repository available at https://archive.ics.uci.edutml/datasets/KDD+Cup+1999+Data. It is contemplated that any other suitable dataset may be used.

The described example embodiments have been verified using a front camera mounted on a vehicle in feasibility studies and the effectiveness of the method has been demonstrated by experimental results on public road.

The described example embodiments can usefully detect curved mirrors that are presented as a true circle, as well as curved mirrors that are presented as an ellipse from the view angle of the image capturing device. This is in contrast to existing ADAS which typically fails to detect a curved mirror in cases where the curved mirror looks like an ellipse from the view angle of the image capturing device.

The described exemplary embodiments, in comparison to an approach using only known deep learning models, can usefully reduce occurrence of false positive results and improve precision rate of detecting curved mirrors. This is in contrast to existing detection systems which mistakenly identify road traffic signs as curved mirrors.

The described exemplary embodiments, in comparison to an approach using only known deep learning models, can usefully provide faster calculation and can usefully provide time savings.

The described example embodiments may be used for collision prediction and collision avoidance, for example at road positions with blind spots and traffic intersections without traffic lights.

The described example embodiments may advantageously reduce the amount of investment needed for building new facility at road intersections and positions with poor visibility, by utilising road safety mirrors which are one of the legacy infrastructures that are already present on the road.

The described example embodiments may advantageously be used to support Autonomous Driving (AD) systems of levels 1 and 2 such as Advanced Driver Assistance System (ADAS) and may be further expanded to support AD fiD systems of more than level 3. The described example embodiments may also be usefully applied in other forms of vehicles such as unmanned ground vehicles (uGVs), automated guided vehicles (AGVs), autonomous vehicles, drones etc. The described example embodiments may further be advantageously used in the field of robotics.

The terms "coupled" or "connected" as used in this description are intended to cover both directly connected or connected through one or more intermediate means, unless otherwise stated.

The terms "configured to (perform a task/action)", "configured for (performing a task/action)" and the like as used in this description include being programmable, programmed, connectable, wired or otherwise constructed to have the ability to perform the task/action when arranged or installed as described herein. The terms "configured to (perform a task/action)", "configured for (performing a task/action)" and the like are intended to cover "when in use, the task/action is performed", e.g., specifically to and/or specifically configured to and/or specifically arranged to and/or specifically adapted to do or perform a task/action.

The term "and/or", e.g., "X and/or Y" is understood to mean either "X and Y" or "X or Y" and should be taken to provide explicit support for both meanings or for either meaning.

The terms "associated with", "related to" and the like used herein when referring to two elements refers to a broad relationship between the two elements. The relationship includes, but is not limited to, a physical, a chemical or a biological relationship. For example, when element A is associated with element B, elements A and B may be directly or indirectly attached to each other or element A may contain element B or vice versa.

The terms "exemplary embodiment", "example embodiment", "exemplary implementation", "exemplarily" and the like used herein are intended to indicate an example of matters described in the present disclosure. Such an example may relate to one or more features defined in the claims and is not necessarily intended to emphasise a best example or any essentialness of any features.

The description herein may be, in certain portions, explicitly or implicitly described as algorithms and/or functional operations that operate on data within a computer memory or an electronic circuit. These algorithmic descriptions and/or functional operations are usually used by those skilled in the information/data processing arts for efficient description. An algorithm is generally relating to a self consistent sequence of steps leading to a desired result. The algorithmic steps can include physical manipulations of physical quantities, such as electrical, magnetic or optical signals capable of being stored, transmitted, transferred, combined, compared, and otherwise manipulated.

Further, unless specifically stated otherwise, and would ordinarily be apparent from the following, a person skilled in the art will appreciate that throughout the present specification, discussions utilizing terms such as "scanning", "calculating", "determining", "replacing", "generating", "initializing", "outputting", and the like, refer to action and processes of an instructing processor/computer system, or similar electronic circuit/device/component, that manipulates/processes and transforms data represented as physical quantities within the described system into other data similarly represented as physical quantities within the system or other information storage, transmission or display devices etc. The description also discloses relevant device/apparatus for performing the steps of the described methods. Such apparatus may be specifically constructed for the purposes of the methods, or may comprise a general purpose computer/processor or other device selectively activated or reconfigured by a computer program stored in a storage member. The algorithms and displays described herein are not inherently related to any particular computer or other apparatus. It is understood that general purpose devices/machines may be used in accordance with the teachings herein. Alternatively, the construction of a specialized device/apparatus to perform the method steps may be desired.

In addition, it is submitted that the description also implicitly covers a computer program, in that it would be clear that the steps of the methods described herein may be put into effect by computer code. It will be appreciated that a large variety of programming languages and coding can be used to implement the teachings of the description herein. Moreover, the computer program if applicable is not limited to any particular control flow and can use different control flows without departing from the scope of the invention.

Furthermore, one or more of the steps of the computer program if applicable may be performed in parallel and/or sequentially. Such a computer program if applicable may be stored on any computer readable medium. The computer readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a suitable reader/general purpose computer. In such instances, the computer readable storage medium is non transitory. Such storage medium also covers all computer readable media e.g., medium that stores data only for short periods of time and/or only in the presence of power, such as register memory, processor cache and Random Access Memory (RAM) and the like. The computer readable medium may even include a wired medium such as exemplified in the Internet system, or wireless medium such as exemplified in Bluetooth technology. The computer readable medium may be, for example, cloud storage on the Internet or within an intranet. The computer program when loaded and executed on a suitable reader effectively results in an apparatus that can implement the steps of the described methods, e.g., in a physical embodiment. The computer readable medium is intended to be transferable and is reproducible in that the computer program if applicable is reproducible.

The example embodiments may also be implemented as hardware modules. A module is a functional hardware unit designed for use with other components or modules. For example, a module may be implemented using digital or discrete electronic components, or it can form a portion of an entire electronic circuit such as an Application Specific Integrated Circuit (AS IC). A person skilled in the art will understand that the example embodiments can also be implemented as a combination of hardware and software modules.

Additionally, when describing some embodiments, the disclosure may have disclosed a method and/or process as a particular sequence of steps. However, unless otherwise required, it will be appreciated the method or process should not be limited to the particular sequence of steps disclosed. Other sequences of steps may be possible. The particular order of the steps disclosed herein should not be construed as undue limitations. Unless otherwise required, a method and/or process disclosed herein should not be limited to the steps being carried out in the order written. The sequence of steps may be varied and still remain within the scope of the disclosure.

Further, in the description herein, the word "substantially" whenever used is understood to include, but not restricted to, "entirely" or "completely" and the like. In addition, terms such as "comprising", "comprise", and the like whenever used, are intended to be non restricting descriptive language in that they broadly include elements/components recited after such terms, in addition to other components not explicitly recited. For an example, when "comprising" is used, reference to a "one" feature is also intended to be a reference to "at least one" of that feature. Terms such as "consisting", "consist", and the like, may, in the appropriate context, be considered as a subset of terms such as "comprising", "comprise", and the like.

Therefore, in embodiments disclosed herein using the terms such as "comprising", "comprise", and the like, it will be appreciated that these embodiments provide teaching for corresponding embodiments using terms such as "consisting", "consist", and the like. Further, terms such as "about", "approximately" and the like whenever used, typically means a reasonable variation, for example a variation of +/-5% of the disclosed value, or a variance of 4% of the disclosed value, or a variance of 3% of the disclosed value, a variance of 2% of the disclosed value or a variance of 1% of the disclosed value.

Furthermore, in the description herein, certain values may be disclosed in a range. The values showing the end points of a range are intended to illustrate a preferred range. Whenever a range has been described, it is intended that the range covers and teaches all possible sub ranges as well as individual numerical values within that range. That is, the end points of a range should not be interpreted as inflexible limitations. For example, a description of a range of 1% to 5% is intended to have specifically disclosed sub ranges 1% to 2%, 1% to 3%, 1% to 4%, 2% to 3% etc., as well as individually, values within that range such as 1%, 2%, 3%, 4% and 5%. The intention of the above specific disclosure is applicable to any depth/breadth of a range.

Different example embodiments can be implemented in the context of data structure, program modules, program and computer instructions executed in a computer implemented environment. A specially configured general purpose computing environment is briefly disclosed herein. One or more example embodiments may be embodied in one or more computer systems, such as is schematically illustrated in FIG. 12.

One or more example embodiments may be implemented as software, such as a computer program being executed within a computer system 1200, and instructing the computer system 1200 to conduct a method of an example embodiment.

The computer system 1200 comprises a computer unit 1202, input modules such as a keyboard 1204 and a pointing device 1206 and a plurality of output devices such as a display 1208, and printer 1210. A user can interact with the computer unit 1202 using the above devices. The pointing device can be implemented with a mouse, track ball, pen device or any similar device. One or more other input devices (not shown) such as a joystick, game pad, satellite dish, scanner, touch sensitive screen or the like can also be connected to the computer unit 1202. The display 1208 may include a cathode ray tube (CRT), liquid crystal display (LCD), field emission display (FED), plasma display or any other device that produces an image that is viewable by the user.

The computer unit 1202 can be connected to a computer network 1212 via a suitable transceiver device 1214, to enable access to e.g. the Internet or other network systems such as Local Area Network (LAN) or Wide Area Network (WAN) or a personal network. The network 1212 can comprise a server, a router, a network personal computer, a peer device or other common network node, a wireless telephone or wireless personal digital assistant. Networking environments may be found in offices, enterprise wide computer networks and home computer systems etc. The transceiver device 1214 can be a modem/router unit located within or external to the computer unit 1202, and may be any type of modem/router such as a cable modem or a satellite modem.

It will be appreciated that network connections shown are exemplary and other ways of establishing a communications link between computers can be used.

The existence of any of various protocols, such as TCP/IP, Frame Relay, Ethernet, FTP, HTTP and the like, is presumed, and the computer unit 1202 can be operated in a client server configuration to permit a user to retrieve web pages from a web based server. Furthermore, any of various web browsers can be used to display and manipulate data on web pages.

The computer unit 1202 in the example comprises a processor 1218, a Random Access Memory (RAM) 1220 and a Read Only Memory (ROM) 1222 The ROM 1222 can be a system memory storing basic input/ output system (BIOS) information. The RAM 1220 can store one or more program modules such as operating systems, application programs and program data.

The computer unit 1202 further comprises a number of Input/Output (I/O) interface units, for example I/O interface unit 1224 to the display 1208, and I/O interface unit 1226 to the keyboard 1204. The components of the computer unit 1202 typically communicate and interface/couple connectedly via an interconnected system bus 1228 and in a manner known to the person skilled in the relevant art.

The bus 1228 can be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.

It will be appreciated that other devices can also be connected to the system bus 1228. For example, a universal serial bus (USB) interface can be used for coupling a video or digital camera to the system bus 1228. An IEEE 1394 interface may be used to couple additional devices to the computer unit 1202. Other manufacturer interfaces are also possible such as FireWire developed by Apple Computer and i.Link developed by Sony. Coupling of devices to the system bus 1228 can also be via a parallel port, a game port, a PCI board or any other interface used to couple an input device to a computer. It will also be appreciated that, while the components are not shown in the figure, sound/audio can be recorded and reproduced with a microphone and a speaker. A sound card may be used to couple a microphone and a speaker to the system bus 1228. It will be appreciated that several peripheral devices can be coupled to the system bus 1228 via alternative interfaces simultaneously.

An application program can be supplied to the user of the computer system 1200 being encoded/stored on a data storage medium such as a CD ROM or flash memory carrier. The application program can be read using a corresponding data storage medium drive of a data storage device 1230. The data storage medium is not limited to being portable and can include instances of being embedded in the computer unit 1202. The data storage device 1230 can comprise a hard disk interface unit and/or a removable memory interface unit (both not shown in detail) respectively coupling a hard disk drive and/or a removable memory drive to the system bus 1228. This can enable reading/writing of data. Examples of removable memory drives include magnetic disk drives and optical disk drives. The drives and their associated computer readable media, such as a floppy disk provide nonvolatile storage of computer readable instructions, data structures, program modules and other data for the computer unit 1202. It will be appreciated that the computer unit 1202 may include several of such drives. Furthermore, the computer unit 1202 may include drives for interfacing with other types of computer readable media.

The application program is read and controlled in its execution by the processor 1218. Intermediate storage of program data may be accomplished using RAM 1220. The method(s) of the example embodiments can be implemented as computer readable instructions, computer executable components, or software modules. One or more software modules may alternatively be used. These can include an executable program, a data link library, a configuration file, a database, a graphical image, a binary data file, a text data file, an object file, a source code file, or the like. When one or more computer processors execute one or more of the software modules, the software modules interact to cause one or more computer systems to perform according to the teachings herein.

The operation of the computer unit 1202 can be controlled by a variety of different program modules. Examples of program modules are routines, programs, objects, components, data structures, libraries, etc. that perform particular tasks or implement particular abstract data types. The example embodiments may also be practiced with other computer system configurations, including handheld devices, multiprocessor systems, microprocessor based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, personal digital assistants, mobile telephones and the like. Furthermore, the example embodiments may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a wireless or wired communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

The example embodiments may also be practiced with other computer system configurations, including handheld devices, multiprocessor systems/servers, microprocessor based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, personal digital assistants, mobile telephones and the like. Furthermore, the example embodiments may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a wireless or wired communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

In the described example embodiments, a vehicle may have been described as a user driven car. It will be appreciated that the exemplary embodiments are not limited as such. For example, the vehicle may comprise any movable object that can detect a curved mirror.

It will be appreciated by a person skilled in the art that other variations and/or modifications may be made to the specific embodiments without departing from the scope of the invention as broadly described. For example, in the description herein, features of different example embodiments may be mixed, combined, interchanged, incorporated, adopted, modified, included etc. or the like across different example embodiments. For example, exemplary embodiments are not necessarily mutually exclusive as some may be combined with one or more embodiments to form new exemplary embodiments. Furthermore, it will be appreciated that while the present disclosure provides embodiments having one or more of the features/characteristics discussed herein, one or more of these features/characteristics may also be disclaimed in other alternative embodiments and the present disclosure provides support for such disclaimers and these associated alternative embodiments. The present embodiments are, therefore, to be considered in all respects to be illustrative and not restrictive.

REFERENCE SIGNS LIST

system for detecting curved mirrors within an image 102 image capturing device 104 processing unit 106 object detection unit 108 action unit 302 square 402 square 404 square 700 transformation map 702a first data point 702b first traffic sign 704a second data point 704b second traffic sign 706 cluster of data points 706 first curved mirror 708 second curved mirror 800 curved mirror candidate 802 image 804 box 806 box 808 data point 810 data point 812 transformation map 814 curved mirror 816 cluster of data points 818 cluster of data points 820 traffic sign 1000 subject vehicle 1002 box 1004 box 1200 computer system 1202 computer unit 1204 keyboard 1206 pointing device 1208 display 1210 printer 1212 computer network 1214 transceiver device lo 1218 processor 1220 random access memory 1222 read only memory 1224 input/output interface unit 1226 interface unit 1228 system bus 1230 data storage device

Claims (15)

1. CLAIMS1. A computer implemented method of detecting curved mirrors (814) within an image (802), characterised in that the method comprises the steps of applying a machine learned algorithm on the image (802) to identify a curved mirror candidate (800) within the image (802); and applying an anomaly detection algorithm on the identified curved mirror candidate (800) to verify if the curved mirror candidate (800) is a curved mirror (814).
2. 2. The method according to claim 1, wherein the anomaly detection algorithm is based on a Deep Autoencoder Gaussian Mixture Model (DAGMM).
3. 3. The method according to claim 1 or 2, wherein the curved mirror candidate (800) is verified as a curved mirror (814) if it is unlikely to be a traffic sign (820).
4. 4. The method according to any one of claims 1 to 3, wherein the anomaly detection algorithm comprises applying an autoencoder function on image data associated with the curved mirror candidate (800) to obtain a point (808, 810) on a transformation map (812), said point (808, 810) representative of the curved mirror candidate (800) on the transformation map (812), verifying the curved mirror candidate (800) as a curved mirror (814) if the point (808, 810) is beyond a threshold distance away from a cluster (816, 818) representative of a traffic sign (820) on the transformation map (812).
5. 5. The method according to claim 4, wherein there are multiple clusters (816, 818) representative of traffic signs (820), the curved mirror candidate (800) is verified as a curved mirror (814) if the point (808, 810) is beyond a threshold distance away from each of the clusters (816, 818).
6. 6. The method according to any one of claims 1 to 5, wherein the machine learned algorithm is trained using bounding boxes with parameters that are optimised for curved mirrors.
7. 7. The method according to claim 6, wherein the parameters that are optimised for curved mirrors comprise height of a bounding box, width of the bounding box, horizontal coordinate of the centre point of the bounding box, vertical coordinate of the centre point of the bounding box, and height to weight ratio of the bounding box.
8. 8. The method according to any one of claims 1 to 7, further comprising a step of acquiring the image (802) using an image capturing device (102).
9. 9. The method according to claim 8, wherein the step of obtaining the image (802) comprises obtaining a plurality of images over different time instances.
10. 10. The method according to any one of claims 1 to 9, wherein the image is a real time image.
11. 11. A system (100) for detecting curved mirrors (814) within an image (802), the system (100) comprising, an image capturing device (102); and an electronic control unit (104) coupled to the image capturing device (102); characterised in that the image capturing device (102) is configured to acquire the image (802); and the electronic control unit (104) is configured to apply a machine learned algorithm on the image (802) to identify a curved mirror candidate (800) within the image (802); and to apply an anomaly detection algorithm on the identified curved mirror candidate (800) to verify if the curved mirror candidate (800) is a curved mirror (814).
12. 12. The system according to claim 11, wherein the anomaly detection algorithm is based on a Deep Autoencoder Gaussian Mixture Model (DAGMM).
13. 13. The system according to claim 11 or 12, wherein the curved mirror candidate (800) is verified as a curved mirror (814) if it is unlikely to be a traffic sign (820).
14. 14. The system according to any one of claims 11 to 13, wherein the anomaly detection algorithm comprises applying an autoencoder function on image data associated with the curved mirror candidate (800) to obtain a point (808, 810) on a transformation map (812), said point (808, 810) representative of the curved mirror candidate (800) on the transformation map (812), verifying the curved mirror candidate (800) as a curved mirror (814) if the point (808, 810) is beyond a threshold distance away from a cluster (816, 818) representative of a traffic sign (820) on the transformation map (812).
15. 15. A computer readable storage medium having stored thereon instructions for instructing a processing unit of a system to execute a computer implemented method of detecting curved mirrors (814) within an image (802), characterised in that the method comprises the steps of: applying a machine learned algorithm on the image (802) to identify a curved mirror candidate (800) within the image (802); applying an anomaly detection algorithm on the identified curved mirror candidate (800) to verify if the curved mirror candidate (800) is a curved mirror (814).