GB2609483A - Method and system for recognizing and classifying restricted road signs for a vehicle automation system - Google Patents

Abstract

A method of recognizing restriction road signs (601-607,Fig.6), comprising: determining, based on shape, the presence of a road sign in an image; segmenting an image of only the road sign 300 into a plurality of predefined image regions 301, 303; identifying a presence or absence of a predefined pattern in the predefined image regions; if the pattern is present, and depending on position of the predefined pattern, the traffic sign is determined to be a height or width restriction sign; if the pattern is not present, potential characters are determined and the sign is classified as weight or speed sign based on the number of valid character candidates. The predefined sign shape may be circular. The predefined pattern to be detected may be a notch (arrow) 301. The validity of the character candidate is determined from the aspect ratio of each character. The sign is determined to be a speed sign when the number of valid characters is equal or less than two. The sign is determined to be a weight sign when the number of valid characters is greater than three and a predefined character (t) is present. Machine learning models may be used to detect the traffic sign shape, predefined pattern and character candidates.

Description

METHOD AND SYSTEM FOR RECOGNIZING AND CLASSIFYING RESTRICTED ROAD SIGNS FOR A VEHICLE AUTOMATION SYSTEM

TECHNICAL FIELD

The present subject matter is related in general to advanced driver assistance system and computer vision, more particularly, but not exclusively to method and system for recognizing and classifying restricted road signs for a vehicle automation system.

BACKGROUND

Traffic signs are integral part of road safety in ADAS environment. It is a support system that can be useful in directing drivers thereby ensuring a safe drive. Several approaches have been proposed in identification of traffic signs i.e., speed limits, city signs, supplementary signs etc., such as Histogram of Gradients (HOG) with Support vector machine (SVM) based recognition system, Optical character recognition module (OCR), traditional computer vision, machine learning, deep learning approach based of Convolutional neural network (CNN), color, shape, motion information and wavelet-based features for detection. However, restrictions road signs such as, width, height and weight limit are unexplored and require improvement in classification because of its similarity in visual appearance within signs and combination of digit and text character. Hence, finding and correcting such errors is crucial for ensuring driving safety.

Currently techniques like OCR, HOG, color space models face challenges with high miss or false classification rate for different variations of traffic signs. This is mainly because vision images are affected with real world artifacts such as, bad weather, occlusion, noise (poor image quality), blur, over exposure and the like. Therefore, accurately recognizing each character of the traffic sign in case of signs with similar features becomes a challenging task. As a result, there is high confusion/misclassification rate of restriction signs. Specially, speed limit signs suffer a higher confusion rate from the restriction signs.

In existing systems, for prediction or correcting errors or misclassification, all character information available on sign board is utilised. Particularly, these systems require ideal conditions where all the characters can be segmented correctly and should have better quality. Traditional computer vision approaches work very well when segmented character does not have any size variation and orientation variations. Deep Learning models such as, CNN, R-NN etc., used for classification and detection are not effective for real-time performance due to complex computation. Also, as automotive systems generally provide only few megabytes of memory where all autonomous algorithms of the ADAS system need to be accommodated, it becomes complex to store and process such large amount of real-time road sign data. Moreover, there is no proper handling of height, weight, and width restriction signs in the existing methodologies.

The information disclosed in this background of the disclosure section is only for enhancement of understanding of the general background of the invention and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person skilled in the art.

SUMMARY

In an embodiment, the present disclosure may relate to a method for recognizing and classifying restricted road signs for a vehicle automation system. The method includes obtaining a plurality of images from predefined regions of a restricted road sign image. The road sign image is obtained on detecting the road sign image to be of a predefined shape. The method includes identifying one of presence or absence of a predefined pattern in the plurality of images at the predefined regions using a pretrained classification model. On identifying the presence of the predefined pattern, the detected road sign image is classified as one of height restricted sign or width restricted sign depending on position of the predefined pattern in the plurality of images. On identifying absence of the predefined pattern, extracting a plurality of segments from predefined areas of the road sign image, and classifying the detected road sign image to be one of a speed limit sign or a weight restricted sign based on number of valid segments from the plurality of segments.

In an embodiment, the present disclosure may relate to a road sign recognition system for recognizing and classifying restricted road signs for a vehicle automation system. The road sign recognition system may comprise a processor and a memory communicatively coupled to the processor, where the memory stores processor executable instructions, which, on execution, may cause the road sign recognition system to obtain a plurality of images from predefined regions of a restricted road sign image. The road sign image is obtained on detecting the road sign image to be of a predefined shape. The road sign recognition system identifies one of presence or absence of a predefined pattern in the plurality of images at the predefined regions using a pretrained classification model. On identifying the presence of the predefined pattern, the detected road sign image is classified as one of height restricted sign or width restricted sign depending on position of the predefined pattern in the plurality of images. On identifying absence of the predefined pattern, the road sign recognition system extracts a plurality of segments from predefined areas of the road sign image and classifies the detected road sign image to be one of a speed limit sign or a weight restricted sign based on number of valid segments from the plurality of segments.

The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system and/or methods in accordance with embodiments of the present subject matter are now described, by way of example only, and with reference to the accompanying figures, in which: Fig.1 illustrates an exemplary block diagram of an autonomous vehicle for recognizing and classifying restricted road signs in accordance with some embodiments of the present disclosure; Fig.2 shows a detailed block diagram of a road sign recognition system in accordance with some embodiments of the present disclosure; Fig.3A shows an exemplary image of extracted regions of a height restricted sign in accordance with some embodiments of the present disclosure; Fig.3B shows an exemplary block for performing training for height restricted sign in accordance with some embodiments of the present disclosure, Fig.4 shows an exemplary representation of an autonomous vehicle, i.e., a car for recognizing and classifying restricted road signs in accordance with some embodiments of present disclosure; and Fig.5A illustrates a flowchart showing a method for autonomous vehicle for recognizing and classifying restricted road signs in accordance with some embodiments of present disclosure; and Fig.6 illustrates an exemplary representation of different road signs with similarity in digits in accordance with some embodiments of present disclosure.

It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative systems embodying the principles of the present subject matter. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in computer readable medium and executed by a computer or processor, whether or not such computer or processor is explicitly shown.

S

DETAILED DESCRIPTION

In the present document, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or implementation of the present subject matter described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

While the disclosure is susceptible to various modifications and alternative forms, specific embodiment thereof has been shown by way of example in the drawings and will be described in detail below. It should be understood, however that it is not intended to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternative falling within the spirit and the scope of the disclosure.

The terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a setup, device, or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus proceeded by "comprises.., a" does not, without more constraints, preclude the existence of other elements or additional elements in the system or method.

In the following detailed description of the embodiments of the disclosure, reference is made to the accompanying drawings that form a part hereof, and in which are shown by way of illustration specific embodiments in which the disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense.

Embodiments of the present disclosure may relate to a method and road sign recognition system for recognizing and classifying restricted road signs for a vehicle automation system. Generally, in traffic signs, especially restricted signs, there is quite a similarity in digits representation In case, segmentation of digits fails because of blurriness in images or any real artifacts, it affects classification process. For instance, there is a high chance of misclassification between height and width restriction signs, as there is a similar pattern of segments or characters information and less variation in visual appearance. Existing systems predict or correct errors or misclassification by utilising complete character information available on sign board. Particularly, these systems require ideal conditions where all the characters on sign board can be segmented correctly and should have better quality. Traditional computer vision approaches work well when segmented character does not have any size variation and orientation variations. Deep Learning models such as, CNN, R-NN etc., used for classification and detection are not effective for real-time performance due to complex computation. Also, as automotive systems generally provide only few megabytes of memory where all autonomous algorithms of the ADAS system need to be accommodated, it becomes complex to store and process such large amount of real-time road sign data. Moreover, currently there is no appropriate handling of height, weight, and width restriction signs in the existing methodologies. Hence, classification of these restricted signs becomes very importance and relevant especially for supporting commercial vehicles.

Accordingly, the present disclosure resolves this problem by obtaining images from predefined regions of a restricted road sign image, which is obtained on detecting the road sign image to be of a predefined shape, such as, a circular shape. Typically, the restricted road sign image represents an image of the restricted road sign. The detected road sign image is classified as one of height restricted sign or width restricted sign upon identifying a predefined pattern, for instance a notch mark, in the plurality of images at the predefined regions. However, in case the predefined pattern is not identified, plurality of segments is extracted from predefined areas of the road sign image and processed to identify valid segments. Thereafter, based on the number of valid segments, the road sign image is classified as one of a speed limit sign or a weight restricted sign using pretrained machine leaning model. Thus, the present invention increases sign coverage, improves performance of road sign recognition, and enhance classification by decreasing false positives at different levels. Specifically, the present invention reduces confusion rate of weight restricted signs due to speed limit signs and put forward new way of classification approach for height/ width restriction signs.

Fig.1 illustrates an exemplary block diagram of an autonomous vehicle for recognizing and classifying restricted road signs in accordance with some embodiments of the present disclosure.

As shown in Fig.1, an exemplary block of an autonomous vehicle 100 is illustrated. The autonomous vehicle 100 includes a road sign recognition system 101. A person skilled in the art would understand that the autonomous vehicle 100 may also include any other units, not mentioned explicitly in the present disclosure. The road sign recognition system 101 is implemented in the autonomous vehicle 100 for recognising and classifying restricted road signs. The restricted road signs are traffic signs which indicates different type of restrictions on the autonomous vehicles on road. For instance, the restricted road signs may be a height restricted sign, a width restricted sign and a weight restricted sign. In an embodiment, the road sign recognition system 101 may be configured within an Electronic Control Unit (ECU) (not shown explicitly in Fig.1) of the autonomous vehicle 100. Further, the road sign recognition system 101 may include an I/0 interface 105, a memory 107 and a processor 109. The I/O interface 105 may be configured to receive data plurality of images of a road sign. The data from the I/0 interface 105 may be stored in the memory 107. The memory 107 may be communicatively coupled to the processor 109 of the road sign recognition system 101. The memory 107 may also store processor instructions which may cause the processor 109 to execute the instructions for recognizing and classifying restricted road signs.

At any instant, while the autonomous vehicle 100 is in motion and detects a road sign 103, the road sign recognition system 101 may process the detected road sign 103 by checking for a predefined shape of the road sign 103. The detected road sign 103 is captured and processed by using a pretrained shape classification model. In an embodiment, the shape classification model may be any machine learning model. In an embodiment, for the classification of the road sign 103, a bounding box of 32by-32 pixel is extracted along width and height. For instance, the predefined shape may be a circular shape. In an embodiment, the classification based on shape is performed in order to detect restricted road signs. On detecting the road sign 103 to be of the predefined shape, the road sign recognition system 101 may obtain a plurality of images from predefined regions of restricted road sign image. In an embodiment, the predefined regions may be extracted along longitudinal and latitudinal areas of the restricted road sign image. Following that, patterns from the plurality of images may be rotated to a predetermined angle for maintaining same symmetry and averaging with another image from the plurality of image.

Further, the road sign recognition system 101 may identify one of presence or absence of a predefined pattern in the plurality of images at the predefined regions using a pretrained classification model. In an embodiment, the classification model may be trained using any machine learning techniques. Particularly, the classification model is trained using a plurality of training image dataset extracted at predetermined pixel area with and without the predefined pattern. The training of the classification model is explained in detail in subsequent part of the description. The predefined pattern may include for instance a notch mark. Thus, the road sign recognition system 101 may identify the presence or absence of the notch mark in the plurality of images at the predefined regions. In case the predefined pattern is present, the road sign recognition system 101 may classify the road sign image as one of height restricted sign or width restricted sign based on position of the predefined pattern in the plurality of images. For instance, if the predefined pattern is detected latitudinally in the plurality of images, the road sign image is classified as the height restricted sign. While, if the predefined pattern is detected longitudinally in the plurality of images, the road sign image is classified as the width restricted sign.

Alternatively, in case the predefined pattern is not identified in the plurality of images, the road sign recognition system 101 may extract a plurality of segments from predefined areas of the road sign image. Further, the road sign recognition system 101 may calculate aspect ratio (width by height) for each of the plurality of segments. In an embodiment, the aspect ratio may be calculated in order to reject segments comprising small characters such as, I,' or false appearing segments and focus on segments including digits and characters (such as, 't') with predefined aspect ratio. Following that, the road sign recognition system 101 determines number of valid segments from the plurality of segments based on the aspect ratio. The valid segments are set of segments of road sign image which are within the predefined aspect ratio. Thereafter, the road sign recognition system 101 may classify the road sign image to be one of a speed limit sign or a weight restricted sign. The road sign image is classified as the speed limit sign on determining the number of valid segments to be lesser than or equal to a predefined number. For instance, the predefined number may be numerical two. Thus, when the number of valid segments is two or lesser than two, the road sign image is classified as the speed limit sign. For instance, when the speed limit is indicated as "35". While, the road sign image is classified as the weight restricted sign on determining the number of valid segments to be greater than the predefined number and lesser than or equal to a second predefined number. For instance, the second predefined number may be numerical three. Thus, when the number of valid segments is greater than two or lesser than equal to three, the road sign image is classified as the weight limit sign. For instance, when the weight restriction sign is indicated as 3, 5t. Particularly, the weight restricted sign is classified by using a pretrained supervised learning model. The supervised learning model may provide a confidence score for each character in the valid segments. This may specifically be performed when the number of valid segments is three. Specifically, the confidence score is determined to identify a predefined character such as, 't', in order to classify the road sign image as the weight restriction sign.

Fig.2 shows a detailed block diagram of a road sign recognition system in accordance with some embodiments of the present disclosure The road sign recognition system 101 may include data 200 and one or more modules 211 which are described herein in detail. In an embodiment, data 200 may be stored within the memory 107. The data 200 may include, for example, sign image data 201, models 203, segment data 205, training data 207 and other data 209.

The sign image data 201 may include the road sign image captured for the road sign 103. The sign image data 201 may include the plurality of images which is obtained from predefined regions of the road sign image. For instance, the plurality of images may be four rectangular areas around longitudinally and latitudinal areas of the road sign image. Fig.3A shows an exemplary image of extracted regions of a height restricted sign in accordance with some embodiments of the present disclosure. As shows, from the height restricted sign 300, two rectangular areas are each extracted longitudinally and latitudinally. The latitudinal rectangular areas are shown as 3011 and 3012, and the longitudinal rectangular areas are shown as 3031 and 3032.

Returning to Fig.2, the models 203 may include the pretrained shape classification model, the classification model, and the supervised learning model. The shape classification model may be trained using road signs with different shapes such as, circular shape, rectangular shape, triangular shapes, and the like, to detect the road sign image of the predefined shape. The classification model may be used for identifying the predefined pattern in the plurality of images at the predefined regions. The shape classification model and the classification model may be any machine learning models. The supervised learning model may be a Support Vector Machine (SVM) classifier for identifying the characters in the valid segments.

The segment data 205 may include information about the valid number of segments identified from the from predefined areas of the road sign image by calculating the aspect ratio. Particularly, the information may include the aspect ratio of each segment of the plurality of segments and details about the rejected segments. The valid segments are determined based on the predefined aspect ratio. For instance, symbols such as, ",", with small aspect ratio may be rejected from the segments.

The training data 207 may include the plurality of training image dataset which are extracted at predetermined pixel area with and without the predefined pattern. For instance, the training image dataset may include rectangular bounding box with thirty percentage of cut-out-size along width i.e., ten-pixel size and twenty-five percentage of cut out size along height i.e., eight-pixel size from the centre of the road sign.

The other data 209 may store data, including temporary data and temporary files, generated by modules 211 for performing the various functions of the road sign recognition system 101. In an embodiment, the data 200 in the memory 107 are processed by the one or more modules 211 present within the memory 107 of the road sign recognition system 101. In an embodiment, the one or more modules 211 may be implemented as dedicated units. As used herein, the term module refers to an application specific integrated circuit (ASIC), an electronic circuit, a field-programmable gate arrays (FPGA), Programmable System-on-Chip (PSoC), a combinational logic circuit, and/or other suitable components that provide the described functionality. In some implementations, the one or more modules 211 may be communicatively coupled to the processor 109 for performing one or more functions of the road sign recognition system 101. The said modules 211 when configured with the functionality defined in the present disclosure will result in a novel hardware.

In one implementation, the one or more modules 211 may include, but are not limited to an image capturing module 213, a training module 215, a pattern identification module 217, a segment extraction module 219, a determination module 221 and a road sign classification module 223. The one or more modules 211 may also include other modules 225 to perform various miscellaneous funcfionalities of the road sign recognition system 101. The other modules 225 may include a shape detection module which may detect the shape of the road sign 103 by using the pretrained shape classification model. For instance, the predefined shape may be the circular shape.

The image capturing module 213 may obtain the plurality of images from the predefined regions of the restricted road sign image. The predefined regions may be latitudinal and longitudinal areas in the road sign image. For instance, a bounding box of 32-by-32 pixel is extracted along width and height of the road sign image. As an example, the image capturing module 213 may obtain rectangular bounding box with 30 percentage of cut-out-size along width i.e., ten-pixel size and twenty-five percentage of cut out size along length i.e., eight-pixel size from the centre of the road sign 103. The image capturing module 213 may obtain the plurality of images on receiving confirmation about shape of the road sign 103 from the shape detection module.

The training module 215 is configured for training the classification model and the supervised learning model. The training module 215 trains the classification model for identifying the predefined pattern on the plurality of images at the predefined areas. The training module 215 performs the training using plurality of training image dataset extracted at predetermined pixel area with and without the predefined pattern. Particularly, the training module 215 may train the classification model with predefined pattern cut out inputs of three classes namely, first class of height, second class of width and third class which is a combination of speed and weight, which are mainly an average of intensity pixels of latitudinally and longitudinally extracted regions in combination. Fig.3B shows an exemplary block for performing training for height restricted sign in accordance with some embodiments of the present disclosure. As shown, the height restricted sign 300 from Fig.3A is shown for training. The two rectangular areas extracted each at latitudinally (3011 and 3012,) and longitudinal (3031 and 3032) are used for training the height restricted sign. As shown, the latitudinal area 3012 is rotated to 180 degrees for maintaining same symmetry and averaged with the latitudinal area 3011. Likewise, the longitudinal area 3031 is rotated to 90 degrees and the longitudinal area 3032 is rotated to -90 degrees for maintaining same symmetry and averaged together. Thereafter, a combined averaged image 305 from latitudinal and longitudinal areas is processed by reducing dimensionality using Principal Component Analysis (PCA) 307 and feature extraction 309 technique, where the features from the analysis is provided as an input to the classification model. Similarly, the training module 215 may train the classification model for the width restricted sign.

Returning to Fig.2, the pattern identification module 217 may identify the presence or absence of the predefined pattern, i.e., the notch mark in the plurality of images at the predefined regions using the trained classification model. In an embodiment, the classification model may be any machine learning model. In case of presence of the predefined pattern, the pattern identification module 217 may indicate to the road sign classification module 223. However, in case of absence of the predefined pattern, the pattern identification module 217 may indicate the same to the segment extraction module 219.

On receiving a notification from the pattern identification module, 217, the segment extraction module 219 may extract the plurality of segments from the predefined areas of the road sign image. Further, the segment extraction module 219 may calculate the aspect ratio for each of the plurality of segments. The aspect ratio may define the ratio between width and height of characters on the image. In an embodiment, the aspect ratio may be calculated using known techniques.

The determination module 221 may determine the number of valid segments based on the aspect ratio of each segment. Particularly, the determination module 221 may select the segments with predefined aspect ratio in order to remove the segment with small ratio, such as symbol "," in the restricted road sign image.

The road sign classification module 223 may classify the road sign image based on the inputs received either from the pattern identification module 217 or the determination module 221. In case of receiving the notification about presence of the predefined pattern in the plurality of images at the predefined regions, the road sign classification module 223 may classify the road sign image as either the height restricted sign or the width restricted sign based on the position of the predefined pattern in the plurality of images. For instance, if the predefined pattern is detected latitudinally in the plurality of images, the road sign image is classified as the height restricted sign. However, if the predefined pattern is detected longitudinally in the plurality of images, the road sign image is classified as the width restricted sign. Further, in case of receiving the input from the determination module 221, the road sign classification module 223 may classify the road sign image as either the speed limit sign or the weight restricted sign. The road sign classification module 223 may classify the road sign image as the speed limit sign on determining the number of valid segments to be lesser than or equal to the predefined number. That is, when the number of valid segments be two or less than two. However, when the number of valid segments is greater than the predefined number and lesser than or equal to a second predefined number, the road sign image is classified as the weight restricted sign. That is, when the number of valid segments is three, the road sign classification module may utilise the supervised learning model in order identify the confidence score for each character in the valid segments. Particularly, when the number of valid segments is three, the supervised learning model is used to recognise the predefined character "t" in order to classify the road sign image as the weight restricted sign.

Fig.4 shows an exemplary representation of an autonomous vehicle, i.e., a car for recognizing and classifying restricted road signs in accordance with some embodiments of present disclosure.

In Fig.4, an exemplary road environment 400 in which an autonomous vehicle, i.e., a car 401 is moving is disclosed. The car 401 include the road sign recognition system 101 (not shown explicitly in Fig.4).

As the car 401 is moving, it detects a road sign 403 which is identified to be of the circular shape. Accordingly, the road sign recognition system 101 of the car 401 captures an image of the road sign 403 and obtains the plurality of images from predefined regions of the road sign image. Further, the road sign recognition system 101 of the car 401 identifies one of presence or absence of the predefined pattern in the plurality of images at the predefined regions using the classification model. In the current scenario, the car 401 detects the notch mark in the predefined areas of the road sign image. That is, the notch mark is detected on the latitudinal areas of the road sign image. Thereafter, on determining the predefined pattern latitudinally, the road sign recognition system 101 of the car 401 classifies the road sign 403 as the height restricted sign and accordingly indicate necessary action to car 401 for navigation.

Fig.5A illustrate a flowchart showing a method for autonomous vehicle for recognizing and classifying restricted road signs in accordance with some embodiments of present disclosure.

As illustrated in Fig.5A, the method 500 includes one or more blocks for recognizing and classifying restricted road signs. The method 500 may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures and modules which perform particular functions or implement particular abstract data types.

The order in which the method 500 is described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the method. Additionally, individual blocks may be deleted from the methods without departing from the scope of the subject matter described herein. Furthermore, the method can be implemented in any suitable hardware, software, firmware, or combination thereof.

At block 501, the plurality of images is obtained by the image capturing module 213 from the predefined regions of the restricted road sign image. Particularly, the road sign image is obtained on detecting the road sign image to be of the predefined shape, such as the circular shape.

At block 503, presence or absence of the predefined pattern is identified by the pattern identification module 217 in the plurality of images at the predefined regions using the classification model. The classification model is trained using the training module 215. On identifying the presence of the predefined pattern, the detected road sign image is classified as one of the height restricted sign or width restricted sign depending on position of the predefined pattern in the plurality of images.

At block 505, the plurality of segments is extracted by the segment extraction module 219 from predefined areas of the road sign image. Further, the aspect ratio is calculated by the segment extraction module 219 for each of the plurality of segments.

At block 507, the detected road sign image is classified by the road sign classification module 223 as one of the speed limit signs or the weight restricted sign based on the number of valid segments from the plurality of segments. For instance, the road sign image is classified as the speed limit sign on determining the number of valid segments to be lesser than or equal to a predefined number, such as numeric value two. Alternatively, the road sign image is classified to be the weight restricted sign on determining the number of valid segments to be greater than the predefined number and lesser than or equal to a second predefined number. That is, when the number of valid segments is three.

Advantaqes of the present disclosure:

An embodiment of the present disclosure provides a framework for classification of height, width, and weight signs to existing traffic sign recognition module.

An embodiment of the present disclosure eliminates extensive computation required during road sign recognition in real time. The present disclosure provides a computationally efficient classification model which requires less utilization of runtime and memory.

An embodiment of the present disclosure solves confusion of road sign with speed signs by classifying height and width restriction signs based on first level classification layer architecture and weight signs based on a second level classification layer using simplified approach. Fig.6 illustrates an exemplary representation of different road signs with similarity in digits in accordance with some embodiments of present disclosure. Fig.6 shows exemplary speed limit sign 601, a height restricted sign 603, a width restricted sign 605 and a weight restricted sign 607. As shown, all the road signs include similarity of digits representation of "35". In case if segmentation of digits fails due to blurriness in images or any real artifacts, the classification of the road signs may be affected. For instance, consider if last text character t of the weight restricted sign 607 is failed to be detected. In such case, it be possible that the weight restricted sign 607 may be recognized as the speed limit sign 601. Similarly, for instance, there exist a high chance of misclassification between the height restricted sign 603 and the width restricted sign 605, due to a similar pattern of segment or characters information and less variation in visual appearance. Therefore, in such cases, the present disclosure solves the confusion and recognize the road signs correctly by using concatenated layers of classification to increase confidence of restrictions signs.

The present disclosure increases sign coverage compared to existing sign recognition (SR) module to support commercial vehicles.

An embodiment of the present disclosure utilizes multiple filters and concatenated layers to increase confidence of restrictions signs rather than a single classification layer. Thus, increasing the recognition accuracy and improving system performance by decreasing our false positives at different levels.

An embodiment of the present disclosure provides a framework which is independent and capable of presenting computationally efficient, less utilization of runtime, and memory-efficient approach compared to traditional approaches such as color space, binarization by using pre-trained model for classifying height restricted sign or width restricted sign and weight restricted signs.

The disclosed method and system overcome technical problem of differentiating and classifying prohibition signs i.e., width, height, and weight restriction traffic signs against speed limit signs by using concatenated layers to increase confidence of restrictions signs rather than a single classification layer. Particularly, the present disclosure resolves this problem by obtaining images from predefined regions of a restricted road sign image, which is obtained on detecting the road sign image to be of a predefined shape, such as, a circular shape. The detected road sign image is classified as one of height restricted sign or width restricted sign upon identifying a predefined pattern, for instance a notch mark, in the plurality of images at the predefined regions. However, in case the predefined pattern is not identified, plurality of segments is extracted from predefined areas of the road sign image and processed to identify valid segments. Thereafter, based on the number of valid segments, the road sign image is classified as one of a speed limit sign or a weight restricted sign using pretrained machine leaning model. Thus, the present invention increases sign coverage and improves performance of road sign recognition and classification by decreasing false positives at different levels. Specifically, the present invention reduces confusion rate of restricted signs due to speed limit signs.

Existing systems predict or correct errors or misclassification by utilising complete character information available on sign board. Particularly, these systems require ideal conditions where all the characters on sign board can be segmented correctly and should have better quality. Traditional computer vision approaches work well when segmented character does not have any size variation and orientation variations. Deep Learning models such as, CNN, R-NN etc., used for classification and detection are not effective for real-time performance due to complex computation. Also, as automotive systems generally provide only few megabytes of memory where all autonomous algorithms of the ADAS system need to be accommodated, it becomes complex to store and process such large amount of real-time road sign data Moreover, currently there is no appropriate handling of height, weight, and width restriction signs in the existing methodologies.

In light of the above-mentioned advantages and the technical advancements provided by the disclosed method and system, the claimed steps as discussed above are not routine, conventional, or well understood in the art, as the claimed steps enable the following solutions to the existing problems in conventional technologies. Further, the claimed steps clearly bring an improvement in the functioning of the system itself as the claimed steps provide a technical solution to a technical problem.

The described operations may be implemented as a method, system or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof. The described operations may be implemented as code maintained in a "non-transitory computer readable medium", where a processor may read and execute the code from the computer readable medium. The processor is at least one of a microprocessor, and a processor capable of processing and executing the queries. A non-transitory computer readable medium may include media such as magnetic storage medium (e.g., hard disk drives, floppy disks, tape, etc.), optical storage (CD-ROMs, DVDs, optical disks, etc.), volatile and non-volatile memory devices (e.g., EEPROMs, ROMs, PROMs, RAMs, DRAMs, SRAMs, Flash Memory, firmware, programmable logic, etc.), etc. Further, non-transitory computer-readable media include all computer-readable media except for a transitory. The code implementing the described operations may further be implemented in hardware logic (e.g., an integrated circuit chip, Programmable Gate Array (PGA), Application Specific Integrated Circuit (ASIC), etc.).

Still further, the code implementing the described operations may be implemented in "transmission signals", where transmission signals may propagate through space or through a transmission media, such as, an optical fiber, copper wire, etc. The transmission signals in which the code or logic is encoded may further include a wireless signal, satellite transmission, radio waves, infrared signals, Bluetooth, etc. The transmission signals in which the code or logic is encoded is capable of being transmitted by a transmitting station and received by a receiving station, where the code or logic encoded in the transmission signal may be decoded and stored in hardware or a non-transitory computer readable medium at the receiving and transmitting stations or devices. An "article of manufacture" includes non-transitory computer readable medium, hardware logic, and/or transmission signals in which code may be implemented. A device in which the code implementing the described embodiments of operations is encoded may include a computer readable medium or hardware logic. Of course, those skilled in the art will recognize that many modifications may be made to this configuration without departing from the scope of the invention, and that the article of manufacture may include suitable information bearing medium known in the art.

The terms "an embodiment", "embodiment", "embodiments", "the embodiment", "the embodiments", "one or more embodiments", "some embodiments", and "one embodiment" mean "one or more (but not all) embodiments of the invention(s)" unless expressly specified otherwise.

The terms "including", "comprising", "having" and variations thereof mean "including but not limited to", unless expressly specified otherwise.

The enumerated listing of items does not imply that any or all of the items are mutually exclusive, unless expressly specified otherwise.

The terms "a", "an" and "the" mean "one or more", unless expressly specified otherwise.

A description of an embodiment with several components in communication with each other does not imply that all such components are required. On the contrary, a variety of optional components are described to illustrate the wide variety of possible embodiments of the invention.

When a single device or article is described herein, it will be readily apparent that more than one device/article (whether or not they cooperate) may be used in place of a single device/article. Similarly, where more than one device or article is described herein (whether or not they cooperate), it will be readily apparent that a single device/article may be used in place of the more than one device or article or a different number of devices/articles may be used instead of the shown number of devices or programs. The functionality and/or the features of a device may be alternatively embodied by one or more other devices which are not explicitly described as having such functionality/features. Thus, other embodiments of the invention need not include the device itself The illustrated operations of Fig.5 show certain events occurring in a certain order. In alternative embodiments, certain operations may be performed in a different order, modified, or removed. Moreover, steps may be added to the above-described logic and still conform to the described embodiments. Further, operations described herein may occur sequentially or certain operations may be processed in parallel. Yet further, operations may be performed by a single processing unit or by distributed processing units.

Finally, the language used in the specification has been principally selected for readability and instructional purposes, and it may not have been selected to delineate or circumscribe the inventive subject matter. It is therefore intended that the scope of the invention be limited not by this detailed description, but rather by any claims that issue on an application based here on. Accordingly, the disclosure of the embodiments of the invention is intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims.

While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Referral numerals:

Reference Number Description

Autonomous vehicle 101 Road sign recognition system 103 Road sign I/O interface 107 Memory 109 Processor Data 201 Sign image data 203 Models 205 Segment data 207 Training data 209 Other data 211 Modules 213 Image obtaining module 215 Training module 217 Pattern identification module 219 Segment extraction module 221 Determination module 223 Road sign classification module 225 Other modules 401 Car 403 Road sign

Claims (26)

1. Claims: 1. A method of recognizing and classifying restricted road signs for a vehicle automation system, the method comprising: obtaining, by a road sign recognition system (101), a plurality of images from predefined regions of a restricted road sign image, wherein the road sign image is obtained on detecting the road sign image to be of a predefined shape; identifying, by the road sign recognition system (101), one of presence or absence of a predefined pattern in the plurality of images at the predefined regions using a pretrained classification model, wherein on identifying the presence of the predefined pattern, the detected road sign image is classified as one of height restricted sign or width restricted sign depending on position of the predefined pattern in the plurality of images; on identifying absence of the predefined pattern, extracting, by the road sign recognition system (101), a plurality of segments from predefined areas of the road sign image; and classifying, by the road sign recognition system (101), the detected road sign image to be one of a speed limit sign or a weight restricted sign based on number of valid segments from the plurality of segments.
2. 2. The method as claimed in claim 1, wherein the predefined shape is circular based shape.
3. 3. The method as claimed in claim 1, wherein the number of valid segments is determined from the plurality of segments based on aspect ratio calculated for each of the plurality of segments.
4. 4. The method as claimed in claim 1, wherein detecting the road sign image to be the predefined shape comprises performing a classification based on shapes using a pretrained shape classifying model by obtaining an image of a road sign (103).
5. 5. The method as claimed in claim 1 further comprising extracting patterns along longitudinally and latitudinally areas and further comprising rotating the patterns from the plurality of images to a predetermined angle for maintaining same symmetry and averaging with another image from the plurality of images.
6. 6. The method as claimed in claim 1, wherein the predefined pattern comprises a notch mark.
7. 7. The method as claimed in claim 1, wherein classifying the detected road sign image to be the speed limit sign on determining the number of valid segments to be lesser than or equal to a predefined number.
8. 8. The method as claimed in claim 7, wherein the predefined number comprises numeric value two.
9. 9. The method as claimed in claim 1, wherein classifying the detected road sign image to be the weight restricted sign on determining a predefined character and on determining the number of valid segments to be greater than a first predefined number and lesser than or equal to a second predefined number.
10. 10. The method as claimed in claim 9, wherein the second predefined number comprises numeric value three.
11. 11. The method as claimed in claim 1, wherein the pretrained classification model is trained using a plurality of training image dataset extracted at predetermined pixel area with and without the predefined pattern.
12. 12. The method as claimed in claim 1, wherein classifying the detected road sign image to be the weight restricted sign comprises using a pretrained supervised learning model.
13. 13. The method as claimed in claim 12 further comprises providing a confidence score for each character in the valid segments using the pretrained supervised machine learning model.
14. 14. A road sign recognition system (101) for recognizing and classifying restricted road signs for a vehicle automation system, comprising: a processor (109); and a memory (107) communicatively coupled to the processor (109), wherein the memory (107) stores processor instructions, which, on execution, causes the processor (109) to: obtain a plurality of images from predefined regions of a restricted road sign image, wherein the road sign image is obtained on detecting the road sign image to be of a predefined shape; identify one of presence or absence of a predefined pattern in the plurality of images at the predefined regions using a pretrained classification model, wherein on identifying the presence of the predefined pattern, the detected road sign image is classified as one of height restricted sign or width restricted sign depending on position of the predefined pattern in the plurality of images; on identifying absence of the predefined pattern, extract a plurality of segments from predefined areas of the road sign image; and classify the detected road sign image to be one of: a speed limit sign or a weight restricted sign based on number of valid segments from the plurality of segments.
15. 15. The road sign recognition system (101) as claimed in claim 14, wherein the predefined shape is circular based shape.
16. 16. The road sign recognition system (101) as claimed in claim 14, wherein the number of valid segments is determined from the plurality of segments based on aspect ratio calculated for each of the plurality of segments
17. 17. The road sign recognition system (101) as claimed in claim 14, wherein the processor (109) detects the road sign image to be the predefined shape by performing a classification based on shapes using a pretrained shape classifying model by obtaining an image of a road sign (103).
18. 18. The road sign recognition system (101) as claimed in claim 14, wherein the processor (109) extracts patterns along longitudinally and latitudinally areas and rotates the patterns from the plurality of images to a predetermined angle for maintaining same symmetry and averaging with another image from the plurality of images.
19. 19. The road sign recognition system (101) as claimed in claim 14, wherein the predefined pattern comprises a notch mark.
20. 20. The road sign recognition system (101) as claimed in claim 14, wherein the processor classifies the detected road sign image to be the speed limit sign on determining the number of valid segments to be lesser than or equal to a predefined number.
21. 21. The road sign recognition system (101) as claimed in claim 20, wherein the predefined number comprises numeric value two.
22. 22. The road sign recognition system (101) as claimed in claim 14, wherein the processor classifies the detected road sign image to be the weight restricted sign on determining a predefined character and on determining the number of valid segments to be greater than a first predefined number and lesser than or equal to a second predefined number.
23. 23. The road sign recognition system (101) as claimed in claim 22, wherein the second predefined number comprises numeric value three.
24. 24. The road sign recognition system (101) as claimed in claim 14, wherein the processor (109) trains the classification model using a plurality of training image dataset extracted at predetermined pixel area with and without the predefined pattern.
25. 25. The road sign recognition system (101) as claimed in claim 14, wherein the processor (109) classifies the detected road sign image to be the weight restricted sign by using a pretrained supervised learning model.
26. 26. The road sign recognition system (101) as claimed in claim 25, wherein the processor (109) provides a confidence score for each character in the valid segments using the pretrained supervised machine learning model.