WO2017116226A1

WO2017116226A1 - System and method to identify identification plate characters using background estimation binarization

Info

Publication number: WO2017116226A1
Application number: PCT/MY2016/050069
Authority: WO
Inventors: Yan Chai HUM; Kim Meng Liang; Hock Woon Hon; Hamam MOKAYED
Original assignee: Mimos Bhd
Current assignee: Mimos Bhd
Priority date: 2015-12-30
Filing date: 2016-10-21
Publication date: 2017-07-06
Anticipated expiration: 2018-06-30
Also published as: MY185612A

Abstract

The present disclosure relates to the field of automated identification plate recognition, and more particularly relates to automated identification plate recognition based on background estimation binarization. In an aspect, the present disclosure relates to a system for detecting and recognizing a vehicle identification plate, wherein system includes a plate detection module configured to detect location of the plate; a plate segmentation module that configured to segment characters in the detected plate into respective individual entities; a plate recognition module configured to recognize the individual entities of the detected plate as alphabets or numerals; and a plate post-analyzer module configured to determine full text of the detected plate, wherein the characters are segmented by the plate segmentation module by applying background estimation binarization in order to obtain the respective individual entities.

Description

SYSTEM AND METHOD TO IDENTIFY IDENTIFICATION PLATE CHARACTERS USING BACKGROUND ESTIMATION BINARIZATION

FIELD OF THE DISCLOSURE

The present disclosure relates to the field of identification plate recognition, and more particularly relates to system and method of automated identification plate recognition based on background estimation binarization. BACKGROUND

Automated applications such as management of car parking, management of toll roads, management of traffic, management of congestion in designated areas, management of over speeding vehicles, management of heavy vehicles in prohibited areas and management of vehicle crimes depend heavily on how accurately the identification plate of each and every vehicle is recognized in the pictures captured by surveillance video cameras (SVC). Accuracy of the automated speeding ticket system depends solely on accurate recognition of the identification plates. Designated areas are equipped with efficient SVCs that capture the front and back sides of each and every vehicle, including people inside the car, place, and the identification number plate as an absolute identity of the vehicle. The SVCs are highly sophisticated and are connected to a given network. Each and every captured video and/or image is passed back to the server end for further processing, if any, and storage.

One of the major objectives of the SVCs is to capture the number plates of each and every vehicle as accurately as possible. However, this is not the case most of the times as the accuracy of the video cameras depend on many factors like the height and distance of the video camera from the vehicle, angle of declination of the camera when taking pictures, speed of the zooming vehicles, ambiguous or deformed or damaged letters and/or numbers on the number plate, angle of the vehicle, dim number plate lights or unlit surroundings and glare around number plates. Further it also depends on natural weather conditions like rain or snow fall, inadequate light due to clouds and/or dust storms during the day (weather conditions), dark conditions from dusk till dawn. These, man-made or natural and adverse conditions significantly hamper the output of the SVCs in terms of accuracy.

A typical automated identification plate recognition (LPR) system aims at extracting the vehicle's registration number from the videos and/or images captured by the SVCs, as accurately as possible, at the backend. The LPR system (i) detects and crops the number plate from the given video and/or image, (ii) converts the image into gray scale, (iii) implements binarization, (iv) implements segmentation and (v) sends to classifier for character recognition. Success of LPR system depends on accurate identification plate recognition, which in turn depends on successful binarization. Post gray scaling, a pixel on the image have a value between 0 (absolute black) to 255 (absolute white), for 8 bit images, while post binarization the pixels have only two values 0 (black) or 1 (white), i.e. suppressing the noise and leaving only the foreground. Success of binarization is very critical since the segmentation and character recognition depends heavily on the quality of binarized image. A good binarization decreases the computational load, simplifies further processing and improves overall LPR system performance.

Binarization process depends on estimating the best gray scale value between 0 and 255 (for 8 bit images) for separating foreground from background noises for a given image. The estimation of the right gray scale value becomes very challenging for images that are captured in adverse conditions wherein a gray scale value in a character region (foreground) may represent the background region in other part of the image of the identification plate, i.e., the pixels intensity distribution of the identification plate characters is not mutually exclusive with the pixels intensity distribution of the identification plate background. An incorrect estimation, thus, would result in improper binarization leading to a loss of information.

The state-of-the-art binarization method of LPR system is mainly estimating a threshold (reference point) gray scale value by using statistical method such as Fisher's criterion for linear discriminant analysis used by Otsu thresholding method. Otsu method estimates the threshold reference value by selecting the value that maximizes the inter- class variance and minimizes the intra-class variance. Otsu method is logical and based on statistical foundation and is more generally categorized as Global thresholding since only one threshold reference value is used throughout image. Another category of thresholding method is regarded as Local thresholding where different reference value is chosen in different regions using windowing technique, but this technique is not suitable for car plate binarization for character extraction purposes since the size of the window is difficult to set adaptively to different type of car plate in different condition; and is not driven by resultant image. In LPR system, slight distortion from the plate affects the eventual outcome.

Existing LPR system, as disclosed in United States Patent US20120087587A1 , uses the binarization of Optical Character Recognition (OCR) technique wherein the binarization process includes several time consuming classifying steps such as establishing boundaries of the image objects and classifying them into suspect or non- suspect objects, followed by creating a binarization threshold map that store each threshold for the pixels of non-suspect objects. The system then identifies a global threshold to complete the binarization process. As such, the system relies heavily on the first classification process, wherein if this process fails, the entire binarization fails. Further, this method requires a substantial number of thresholds to adapt each pixel to the environment, which is computationally prohibitive for real-time car identification plate application. Finally, the prior art has no feedback mechanism in place, hence no feedback from the resultant binarized image is taken and therefore the system is not driven by how relevant the binarized image is processed by subsequent procedures for the final OCR classifier. There is, therefore, a need for a LPR system that incorporates binarization feedback into the system that is computationally feasible, that takes into account invariance of object edges to optimize binarized plate image in order to ease the character extraction process in subsequent steps, and hence increases overall accuracy of the LPR system.

SUMMARY

The present disclosure relates to the field of identification plate recognition, and more particularly relates to system and method of automated identification plate recognition based on background estimation binarization.

In an aspect, the present disclosure relates to a system for detecting and recognizing a vehicle identification plate, wherein system includes a plate detection module that is configured to detect location of the plate; a plate segmentation module that is configured to segment characters in the detected plate into respective individual entities; a plate recognition module that is configured to recognize the individual entities of the detected plate as alphabets and/or numeric; and a plate post-analyzer module that is configured to determine full text of the detected plate. This include the characters are segmented by the plate segmentation module by applying background estimation binarization in order to obtain the respective individual entities.

In an aspect, the plate segmentation module is configured to perform one or a combination of noise removal to remove non-character objects, transformation correction to correct orientation of the detected plate, and application of the background estimation binarization that is configured to isolate the characters from the detected plate by updating background values into values estimated from local statistics and followed by Otsu thresholding. In another aspect, the background estimation binarization is configured to estimate a background value that approximates gray scale values to generate an updated background value that, along with the gray scale values, is truncated into black background in order to isolate the characters into the individual entities.

In another aspect, the background estimation binarization is configured to include a fusion stage, a background estimation stage, and a plate quality evaluation stage, wherein at the fusion stage, gray scaled image of the detected plate is converted into a first binary image using Otsu method, and into a second binary image using local adaptive method, followed by multiplying the first binary image and the second binary image to obtain a fused background image. In an aspect, the first binary image is noise filtered before the multiplication.

In another aspect, the background estimation stage is configured to standardize gray scale intensity distribution of the gray scaled image of the detected plate, followed by computing mean and standard deviation of the gray scaled image, wherein based on the mean and standard deviation, background value of the gray scaled image is estimated. In yet another aspect, the standard deviation is scaled using a scaling factor, and wherein by changing the scaling factor, multiple estimated background values are generated. The gray scaled image is converted into corresponding candidate binary images based on the multiple estimated background values,

In an aspect, the plate quality evaluation stage is configured to process the candidate binary images and the fused background image based on computation of number of isolated blobs in each candidate binary image and fused background image, determination of size of each isolated blob, comparison of the size of each isolated blob with plate height, and assign a score to each blob based on a defined criteria, wherein scores for all blobs of each candidate binary image and fused background image are summed, and candidate binary image and fused background image with highest score is selected as final binary image and sent to the plate recognition module. The present disclosure further relates to a method for detecting and recognizing a vehicle identification plate, said method including the steps of detecting location of the plate; segmenting characters in the detected plate into respective individual entities; recognizing the individual entities of the detected plate as alphabets and/or numerals; and determining full text of the detected plate, wherein the characters are segmented by the plate segmentation module by applying background estimation binarization in order to obtain the respective individual entities.

Various objects, features, aspects and advantages of the inventive subject matter will become more apparent from the following detailed description of preferred embodiments, along with the accompanying drawing figures in which like numerals represent like components.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are included to provide a further understanding of the present disclosure, and are incorporated in and constitute a part of this specification. The drawings illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

FIGs. 1 A to 1 D illustrate Otsu thresholding technique.

FIG. 2 illustrates exemplary functional modules of the proposed LPR system in accordance with embodiments of the present disclosure.

FIG. 3 illustrates an exemplary process diagram of the LPR system in accordance with embodiments of the present disclosure.

FIG. 4 illustrates an exemplary process diagram of the plate segmentation in accordance with embodiments of the present disclosure. FIG. 5 illustrates an exemplary background estimation binarization process diagram in accordance with embodiments of the present disclosure. FIG. 6 illustrates an exemplary fusion process diagram in accordance with an embodiment of the present disclosure. FIG. 7 illustrates an exemplary background estimation process diagram in accordance with an embodiment of the present disclosure.

FIG. 8 illustrates an exemplary updated gray scaled image diagram in accordance with an embodiment of the present disclosure.

FIG. 9 illustrates an exemplary plate quality evaluation process diagram in accordance with an embodiment of the present disclosure.

FIG. 10 illustrates an exemplary blob metric evaluation diagram in accordance with an embodiment of the present disclosure.

FIG. 1 1 illustrates an exemplary pictorial diagram in accordance with an embodiment of the present disclosure.

DETAILED DESCRIPTION

In accordance with the present disclosure, there is provided a system and a method for an automated identification plate recognition system using background estimation binarization, which will now be described with reference to the embodiment shown in the accompanying drawings. The embodiment does not limit the scope and ambit of the disclosure. The description relates purely to the exemplary embodiment and its suggested applications.

The embodiment herein and the various features and advantageous details thereof are explained with reference to the non-limiting embodiment in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiment herein may be practiced and to further enable those of skill in the art to practice the embodiment herein. Accordingly, the description should not be construed as limiting the scope of the embodiment herein.

The description hereinafter, of the specific embodiment will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify or adapt or perform both for various applications such specific embodiment without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation.

Various terms as used herein are defined below. To the extent a term used in a claim is not defined below, it should be given the broadest definition persons in the pertinent art have given that term as reflected in printed publications and issued patents at the time of filing.

In an aspect, when a vehicle approaches or leaves, a SVC installed at designated areas including car parking, toll road, speed control check-post, prohibited area, etc., can capture video or pictures of the vehicle, ensuring to capture people inside the vehicle, place identity where the vehicle's video and/or pictures were captured and most importantly the identification plate bearing the registration number of the vehicle as an absolute identity of the vehicle. In an aspect, LPR system starts with detection of identification plate from the videos and/or images that are captured by the SVCs, wherein the detected plate undergoes a plate segmentation process whereby each and every character on the identification plate is extracted in turn. In an aspect, segmentation process starts with conversion of the detected identification plate image into gray scale image using gray processing technique whereby the R, G, B pixel values in the image are replaced by equivalent gray pixel values. Following the gray processing, the LPR system removes noise from the detected plate, which noise is occurred due to the fact that the videos and/or images were captured in adverse conditions, which causes a gray scale value in a character region (foreground) also representing the background region in other part of the image of the detected plate. Post noise removal, the detected plate is skewed whereby the detected plate is skewed and/or tilted to desired orientation in order to have the identification plate aligned horizontally resulting in a skewed gray scaled image.

In an aspect, the skewed gray scaled image is binarized using a fusion method and a background estimation method, wherein the fusion method uses two different techniques to produce two binarized images, which are fused to yield a binarized candidate image for further processing. In the first technique, the skewed gray scaled image is converted into binarized image using Otsu method followed by noise filtering, and in the second technique, the skewed gray scaled image is converted into another binarized image using local adaptive method. The Otsu method is more generally categorized as Global thresholding since only one threshold reference value is used throughout the image, whereas the local thresholding method opts for different reference values in different regions using windowing technique. The two binarized images are fused together, using multiplication method, to get a binarized candidate image. The multiplication method involves multiplying the pixel intensity of the two binarized images in order to get a binarized candidate image that is more resilient in uneven illumination condition in the input skewed gray scaled plate image.

In an aspect, the skewed gray scaled image is binarized using the background estimation method to produce several binarized candidate images for further processing. The background estimation method starts with computing a histogram of the skewed gray scaled image followed by histogram equalization in order to standardize the gray scale intensity distribution, followed by computing the mean, the standard deviation and the estimated background value. The estimated background value depends on a scaling factor. In an aspect, a number of different estimated background values (intensities) are generated using different scaling factors, and a binarized candidate image is obtained for each estimated background value. For each of the estimated background values, pixel intensity of each and every pixel of the input skewed gray scaled image is compared with the given estimated background intensity; replacing the image intensity with the estimated background intensity wherever image intensity is less than the given estimated background intensity, resulting in an updated gray scaled image, which is then subjected to Otsu thresholding method to produce a binarized candidate image that is more resilient in tackling inconsistencies and variations. This process is repeated to create a binarized candidate image for each of the generated estimated background values.

In an aspect, the binarized candidate images are then processed by the plate quality evaluation method, which aims at quantifying the quality of the input binarized candidate images. The evaluation is performed blob by blob, wherein each and every blob that is connected components of white pixels, for a given binarized candidate image, is evaluated on one or more criteria, and a score is assigned to the corresponding blob based on the successful criterion. This is followed by summing up of all metric scores that are assigned to each successful blob, for all the blobs of the given binarized candidate image, to get the plate blob metric evaluation score. The higher the plate blob metric evaluation score, the better is the binarization and segmentation. This process is repeated for all binarized candidate images so that each and every binarized candidate image has a corresponding plate blob metric evaluation score. In the feedback process, the binarized candidate images along with their corresponding plate blob metric evaluation score go through plate selection process that selects the binarized candidate image with the highest score for plate recognition, i.e., the binarized candidate image with best segmentation is selected for identification plate recognition. In an aspect, the assembly of segmented characters from the detected plate results in the identification plate recognition, whereby the individual entities on the identification plate are converted into text format of alphabets and/or numbers. The recognized plates, is then undergo plate post-analyzer process that analyzes all of the recognized identification plate in order to determine the final text content of the plate as a unique identity to individual vehicles. Once the identification plate text is revealed, the text content of the vehicle identification plate is displayed on the monitor before being stored permanently in the database storage at the backend.

FIG. 1A illustrates an existing diagram 100 showing a histogram 102 of a 6-level gray scale image 104 in order to understand the Otsu thresholding technique. For the sake of understanding the underlying principals and algorithms, a simple 6x6 pixel image 104 having only 6 levels of gray scale is chosen. The count of pixels, totaling to 36, for each of the 6 different gray scales is shown in the histogram 102.

Otsu's thresholding method involves iterating through all the possible threshold values and calculating a measure of spread for the pixel levels each side of the threshold, i.e., the pixels that either fall in foreground or background. The aim is to find the threshold value where the sum of foreground and background spreads is at its minimum.

FIG. 1 B illustrates an existing histogram 120 of the background pixels and calculations for finding the background weight, mean and variance (the measure of spread) for a given threshold, in this case the threshold value is 3. FIG. 1 C illustrates an existing histogram 140 of the foreground pixels and calculations for finding the foreground weight, mean and variance for a given threshold. Once the variance for the background and foreground are calculated, the 'Within-Class Variance' is calculated which is the sum of 'weighted variances' for the threshold value 3.

The algorithms used for various calculations are as follows:

∑ (Count of respective pixels)

Weight (W) =

Count of all pixels

∑ ((Gray scale value of the pixel)x(Count of the pixel))

Mean (μ) =

Count of respective pixels Count of respective pixels

Within Class Variance( σ _w ²) = W_b σ _b ² + W_f σ ²

= 0.4722 x 0.4637 + 0.5278 χ 0.5152

- 0.4909

FIG. 1 D illustrates an existing diagram 160 showing the final binarized image and histogram for each of the 6 values of the pixels from 0 to 5. The highlighted column 168 shows the values for the threshold calculated in FIG 1 B & 1 C. All pixels with a value less than 3 are background, and all pixels with a value equal to or greater than 3 are foreground. The binarization process includes replacing all background pixels to 0 (black) and foreground pixels to 1 (white) revealing the actual image. As the images in the table show, this threshold (T = 3) works well. It is easily seen that rest of the resulting images 162 for T = 0, 164 for T = 1 , 166 for T = 2, 170 for T = 4 and 172 for T = 5 are all incorrect. This approach for calculating Otsu's threshold is useful for explaining the theory, which is computationally intensive, especially for a full 8-bit grey scale image.

FIG. 2 illustrates exemplary functional modules 200 of the LPR system in accordance with embodiments of the present disclosure. In an aspect, the present disclosure relates to a system 200 for detecting and recognizing a vehicle identification plate, wherein the system 200 include a plate detection module 202 that is configured to detect location of the plate, a plate segmentation module 204 that is configured to segment characters in the detected plate into respective individual entities; a Plate Recognition Module 206 that is configured to recognize the individual entities of the detected plate as alphabets and/or numerals; and a plate post-analyzer module 208 that is configured to determine full text of the detected plate., The characters are segmented by the plate segmentation module by applying background estimation binarization in order to obtain the respective individual entities. In an aspect, the plate detection module 202 is configured to detect one or more images containing identification plate from videos and/or images that are captured by SVCs. Once the identification plate is detected, the LPR system converts the image into gray using gray processing technique, which involves conversion of color image into a gray image. Gray processing is based on different color transformations, whereby the R, G, B pixel values in the image are replaced by equivalent (calculated) gray pixel values. Following the gray processing, the LPR system removes noise from the detected plate. This noise occurs due to the fact that the videos and/or images are captured in adverse conditions that cause a gray scale value in a character region (foreground) also representing the background region in other part of the image of the detected plate. Post noise removal, the detected plate is skewed whereby the detected plate is skewed and/or tilted to desired orientation in order to have the identification plate aligned horizontally. The output of plate detection module is a skewed gray scaled image that input to the plate segmentation module 204 to produce binarized candidate images.

In an aspect, the plate segmentation module 204 is configured to perform one or a combination of noise removal to remove non-character objects, transformation correction to correct orientation of the detected plate, and application of the background estimation binarization 210 that is configured to isolate the characters from the detected plate by updating the background into values estimated from local statistics followed by Otsu thresholding. In another aspect, the background estimation binarization 210 is configured to estimate a background value that approximates gray scale values to generate an updated background. The updated background is truncated into black background along with the gray scale values in order to isolate the characters into the individual entities.

In another aspect, the background estimation binarization 210 is configured to include a fusion stage or module 212, a background estimation stage or module 214, and a plate quality evaluation stage or module 216, wherein at the fusion stage 212, gray scaled image of the detected plate is converted into a first binary image using Otsu method, and into a second binary image using local adaptive method, followed by multiplying the first binary image and the second binary image to obtain a fused background image. In an aspect, the first binary image is noise filtered before multiplication. In an exemplary implementation of module 212, the skewed gray scaled image is binarized using the fusion module 212 using two different techniques and the resultant binarized images is fused together to produce one binarized candidate image for further processing. In the first technique, the skewed gray scaled image is converted into binarized image using Otsu method followed by noise filtering. In the second technique, the skewed gray scaled image is converted into another binarized image using local adaptive method. In an embodiment, the Otsu method is more generally categorized as Global thresholding since only one threshold reference value is used throughout the image whereas the local thresholding method opts for different reference value in different region using windowing technique. In an embodiment, the two binarized images are fused together, using multiplication method to get one binarized candidate image. The multiplication method involves multiplying the pixel intensity of the two binarized images in order to get a binarized candidate image that is more resilient in uneven illumination condition in the input skewed gray scaled plate image.

In another aspect, the background estimation stage or module 214 is configured to standardize gray scale intensity distribution of the gray scaled image of the detected plate, followed by computing mean and standard deviation of the gray scaled image, wherein based on the mean and standard deviation, background value of the gray scaled image is estimated. In yet another aspect, the standard deviation is scaled using a scaling factor, and wherein by changing the scaling factor, multiple estimated background values are generated. In another aspect, based on the multiple estimated background values, the gray scaled image is converted into corresponding candidate binary images. In an exemplary implementation of module 214, skewed gray scaled image is binarized using background estimation module 214 to produce several binarized candidate images for further processing. The background estimation module 214 is start with computing a histogram of the skewed gray scaled image, followed by histogram equalization in order to standardize gray scale intensity distribution, followed by computing mean, standard deviation, and estimated background value, wherein the estimated background value depends on a scaling factor. In an exemplary embodiment, three different estimated background values (intensities) is generated for instance using 3 different scaling factors, to output three binarized candidate images. For a given estimated background value, pixel intensity of each and every pixel of the input skewed gray scaled image is compared with the given estimated background intensity; replacing the image intensity with the estimated background intensity wherever image intensity is less than the given estimated background intensity resulting in an updated gray scaled image which is then subjected to Otsu thresholding method to produce a binarized candidate image. This process is repeated for the 3 estimated background values to create 3 binarized candidate images, which are more resilient in tackling inconsistencies and variations.

In an aspect, the plate quality evaluation stage or module 216 is configured to process the candidate binary images and the fused background image based on computation of number of isolated blobs in each candidate binary image and fused background image, determination of size of each isolated blob, comparison of the size of each isolated blob with plate height, and assign a score to each blob based on a defined criteria, wherein scores for all blobs of each candidate binary image and fused background image are summed, and candidate binary image and fused background image with highest score is selected as final binary image and sent to the plate recognition module. In an exemplary implementation of module 216, the binarized candidate images can further be processed by the plate quality evaluation module 208, which aims at quantifying the quality of the input binarized candidate images. The evaluation is performed blob by blob wherein each and every blob (connected components of white pixels), for a given binarized candidate image, is evaluated on 3 criteria and a score is assigned to the corresponding blob based on the successful criterion. This is followed by summing up of all metric scores that are assigned to each successful blob, for all the blobs of the given binarized candidate image to get the plate blob metric evaluation score, which process is repeated for all binarized candidate images so that each and every binarized candidate image has a corresponding plate blob metric evaluation score. In an embodiment, the binarized candidate images along with their corresponding plate blob metric evaluation score are returned to plate selection that selects the binarized candidate image with the highest score for plate recognition.

FIG. 3 illustrates an exemplary process diagram 300 of the LPR system in accordance with embodiments of the present disclosure. In an embodiment, when a vehicle 304 having a identification plate bearing number 'WSM 22' 306 approaches or leaves a SVC 302 that is installed at designated areas including car parking, toll road, speed control check-post, prohibited area, etc., the SVC 302 captures video and/or pictures of such vehicles, ensuring to capture people inside the vehicle, place identity (where the vehicle's video and/or pictures were captured) and most importantly the identification plate bearing the registration number of the vehicle as an absolute identity of the vehicle.

In an embodiment, for a video capture, the video is played back image frame 308 wise with a focus on identification plate detection 31 0. Once the identification plate is detected, the LPR system converts the image into gray using the gray processing technique where the R, G, B values in the image is replaced by equivalent gray values resulting in a gray image. The detected plate undergoes a plate segmentation 312 process (to be described in FIG. 4) whereby each and every character on the identification plate is extracted, in turn, and processed for recognition before all characters are individually recognized. The assembly of segmented characters from the detected plate results in the identification plate recognition 314, whereby individual entities on the identification plate is converted into text format of alphabets and/or numbers. In an embodiment, the recognized plate undergo plate post-analyzer 31 6 process that analyze the recognized identification plate in order to determine the final text content of the plate as a unique identity to individual vehicles. Once the identification plate text is revealed, text content of the vehicle identification plate 318 is displayed on the monitor 322 before being stored permanently in the database storage 320 at the backend. FIG. 4 illustrates an exemplary process diagram 312 of the plate segmentation in accordance with embodiments of the present disclosure. As discussed in FIG. 3, one of the major objectives of the LPR system 300 is identification plate detection 310 following which the detected gray scaled plate 404 undergo noise removal 406 process. The noise, here, is referred to non-character or insignificant objects that become part of the detected gray scaled plate 404, caused due to the fact that the video and/or images were captured in adverse conditions, where a gray scale value in a character region (foreground) may also represent the background region in other part of the image of the detected plate 404. In an embodiment, post noise removal 406, the detected gray scaled plate 404 undergoes transformation correction 408. The objective of the transformation correction 408 (also called skewing) is to skew and/or tilt the detected gray scaled plate 404 to desired orientation in order to have the plate contents aligned horizontally as best as possible. In an embodiment, the skewed detected gray scaled plate 404 is subjected to binarization 410 process (to be described in FIG. 5) whereby, deploying the background estimation binarization, the characters on the identification plate is revealed, and finally the character segmentation 412 take place where the characters are extracted from the binarized plate image. In an embodiment, the process is then return to plate segmentation 312.

FIG. 5 illustrates an exemplary background estimation binarization process diagram 41 0 in accordance with embodiments of the present disclosure. As discussed in FIG. 4, the detected gray scaled plate or image 404 undergo noise removal 406 and transformation correction 408 resulting in skewed gray scaled plate 504, wherein the identification plate is horizontally aligned and be ready for binarization 410 processes.

In an embodiment, the proposed binarization method, namely the background estimation binarization, is a binarization method that involves estimating a background value that approximates the gray scale values (ambiguous region) that contribute to connected character problem so that this updated background together with the ambiguous region is truncated into black background after binarization in order to isolate the characters into separate entities. The binarization process is accomplished in three stages namely (i) fusion stage 506, (ii) background estimation stage 510 and (iii) plate quality evaluation stage 514.

In an embodiment, the fusion stage 506 (to be described in FIG. 6) aims at converting input skewed gray scaled plate 504 image into one binarized candidate image 508 that is more resilient in uneven illumination conditions, whereas the background estimation stage 510 (to be described in FIG. 7) aims at converting the input skewed gray scaled plate 504 image into several binarized candidate images 512 that are more resilient in tackling inconsistencies and variations.

The output of the two stages is fed into plate quality evaluation stage 514 (to be described in FIG. 9), an assessment stage that aims to quantify the quality of each binarized candidate image by giving a numeric score for each blob, followed by summing up the scores of all the blobs within a candidate binarized image. Binarized candidate image that possesses the highest score is selected as the final binarized image, i.e., the plate with isolated blob 516, is returned to character segmentation 412. FIG. 6 illustrates an exemplary fusion stage diagram 506 in accordance with an embodiment of the present disclosure. The fusion stage, as illustrated in FIG. 6, involves converting the skewed gray scaled image 504 into one binarized candidate image 508 that is returned to the plate quality evaluation stage 514. In an embodiment, the fusion stage 600 follows two different techniques to create a binarized image each. In the first technique, the skewed gray scaled image 504 is converted into binarized image using Otsu method 604 followed by noise filtering 606. In the second technique, the skewed gray scaled image 504 is converted into another binarized image using local adaptive method 608. In an embodiment, the Otsu method is more generally categorized as Global thresholding since only one threshold reference value is used throughout the image whereas local thresholding method opts for different reference value in different region using windowing technique. In an embodiment, the next step is to fuse both methods 610 to get a binarized candidate image 508. In this step, the two images is combined together using multiplication wherein each pixel in each image is multiplied together. The output of the pixel multiplication, for an 8-bit image, is either 0 (i.e., one of the pixel's value is 0) or 255, a 0 representing black color and a 255 representing a white color. The purpose of this fusion stage is to provide a binarized candidate image 508 that is more resilient in uneven illumination condition in the input skewed gray scaled plate 504 image. The fused image then returns to plate quality evaluation stage 514.

FIG. 7 illustrates an exemplary background estimation stage diagram 510 in accordance with an embodiment of the present disclosure. The background estimation stage, as illustrated in FIG. 7, involves converting the skewed gray scaled image 504 into several binarized candidate images 512 that are returned to plate quality evaluation stage 514. In an embodiment, the background estimation stage starts with computing a histogram of the skewed gray scaled image 504 followed by histogram equalization 704 in order to standardize the gray scale intensity distribution, followed by computing the mean (μ), the standard deviation (σ) of the input skewed gray scaled image 504 and finally the estimated background value. The algorithm used to compute the three values includes, for instance:

l vviuin i neigni

y

i Width Height

\ - Image(x, y) |

Estimated background value = μ + α (σ)

In an embodiment, a is the scaling factor. By using different scaling factors, several estimated background values is generated. The background estimation process 706 is described in FIG. 8, wherein several binarized candidate images 512 is created and returned to plate quality evaluation stage 514. FIG. 8 illustrates an exemplary background estimation diagram 800 for producing several binarized candidate images in accordance with an embodiment of the present disclosure. After computing the estimated background values, as described in FIG. 7, the input skewed gray scaled plate 504 image is transformed by several binarized candidate images using these estimated background values. In an embodiment, 3 different estimated background values (intensities) are used in this exemplary process.

In an embodiment, the first estimated background intensity 1 is selected. In another embodiment, the pixel intensity of each and every pixel of the input skewed gray scaled plate 504 image is read in sequence and compared 804 with the estimated background intensity 1 . If the image intensity is greater than the estimated background intensity 1 , the image intensity is retained 812; else the image intensity is replaced by the estimated background intensity 1 81 0. Once all pixels are read and compared, an updated gray scaled image 1 822 results. Finally, Otsu thresholding method is implemented to transform the updated gray scaled image into binarized image 1 828. The process is repeated for the estimated background intensity 2 and estimated background intensity 3 to get updated gray scaled image 2 824 and updated gray scaled image 3 826 and finally binarized image 2 830 and binarized image 3 832. These binarized candidate images 512 are then returned to plate quality evaluation stage 514.

FIG. 9 illustrates an exemplary plate quality evaluation process diagram 900 in accordance with an embodiment of the present disclosure. As shown in the FIG. 9, the Otsu binarized candidate images 902 (input 508 from fusion stage 506 & input 512 from background estimation stage 510 respectively) are input to the blob metric evaluation 904 which is described in FIG. 10 that returns the binarized candidate images 902 along with their corresponding plate blob metric evaluation scores as input to plate selection 906. The plate blob metric evaluation score of various binarized candidate images 902 is compared, and the one with the highest score is selected at plate selection 906 as the final binarized image, i.e., the plate with isolated blob 516. In an embodiment, the blob metrics evaluation serves to assess the quality of each blob in a candidate plate and is proportional to the score of the plate, i.e., the more the metric evaluation score, the better is the binarization.

FIG. 10 illustrates an exemplary blob metric evaluation diagram 1000 in accordance with an embodiment of the present disclosure. In an embodiment, a blob refers to connected components of white pixels. In an aspect, blob metric evaluation 904 involves evaluating which binarized candidate image 1002 is to be sent to subsequent processes of the LPR system. This evaluation is performed blob by blob, wherein each and every blob for a given binarized candidate image 1002 is evaluated based on 3 criteria (for instance), and a score is assigned to the corresponding blob based on successful criterion. This is followed by summing up of all metric scores that are assigned to each successful blob, for all the blobs of the given binarized candidate image 1 002 to get the plate blob metric evaluation score 1016. This process is repeated for all binarized candidate images 1002 so that each and every binarized candidate image 1002 has a corresponding plate blob metric evaluation score 1016. Finally, the binarized candidate images 1 002 along with their corresponding plate blob metric evaluation score 1016 is returned to plate selection 904.

The blob metrics evaluation serves to assess quality of each blob in a candidate plate where the number of isolated blobs is computed, and size of each blob is computed and compared with plate height. Similarly, width of each blob is computed and compared with plate width to assign a quantified value to each blob according to the criteria. In an embodiment, criterion 1 is said to be successful if blob height is greater than 80% plate height and blob width is greater than 50% plate width 1004 having a success score of +0.5 1010. Criterion 2 is said to be successful if blob height is greater than 80% plate height and blob width is smaller than 50% plate width 1006 having a success score of +1 .0 1012. Similarly, Criterion 3 is said to be successful if blob height is greater than 80% plate height 1008 having a success score of -1 .0 1014. One should appreciate that these are merely exemplary criteria and any other criteria is configured and implemented as part of the proposed system. Also, even the threshold value for these criteria is modified based on the identification plate in context, impact of wrong number determination, among other like parameters.

FIG. 1 1 illustrates an exemplary pictorial diagram in accordance with an embodiment of the present disclosure. As shown in the figure, the identification plate 1 102 is input to fusion stage resulting in a binarized candidate image 1 010 while two binarized candidate images (1 1 16 & 1 120) is produced from the background estimation stage. The plate blob metric evaluation score of the three binarized candidate images are 1 .0 1022, 2.5 1024 and 1 .5 1026 respectively. The binarized candidate image 1 1 16 having the highest score of 2.5 is therefore be selected as the final binarized image 1 028 for plate recognition.

While the foregoing describes various embodiments of the invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof. The scope of the invention is determined by the claims that follow. The invention is not limited to the described embodiments, versions or examples, which are included to enable a person having ordinary skill in the art to make and use the invention when combined with information and knowledge available to the person having ordinary skill in the art.

Claims

1 . A system for detecting and recognizing an identification plate (306), said system comprising a plate detection module (202) configured to detect location of the plate (306) and a plate post-analyzer module (208) configured to determine full text of the detected plate (306), characterized in that, said system further comprises:

i. a plate segmentation module (204) configured to segment characters in the detected plate (306) into respective individual entities, wherein the characters are segmented by applying background estimation binarization

(210) in order to obtain the respective individual entities; and ii. a plate recognition module (206) configured to recognize the individual entities of the detected plate as alphabets or numerals.

2. The system of claim 1 , wherein the plate segmentation module (204) is configured to perform one or a combination of noise removal to remove non- character objects, transformation correction to correct orientation of the detected plate, and application of the background estimation binarization (210) that is configured to isolate the characters from the detected plate by an updating background value into values estimated from local statistics followed by Otsu thresholding.

3. The system of claim 1 , wherein the background estimation binarization (210) is configured to estimate a background value that approximates gray scale values to generate an updated background value, wherein the updated background is truncated into black background along with the gray scale values in order to isolate the characters into the individual entities. A method for detecting and recognizing an identification plate (306), said method comprising the steps of:

i. detecting location of the plate;

ii. segmenting characters in the detected plate into respective individual entities by applying background estimation binarization;

iii. recognizing the individual entities of the detected plate as alphabets or numerals; and

iv. determining full text of the detected plate.

The method of claim 4, wherein the step of segmenting further comprises performing one or a combination of noise removal to remove non-character objects, transformation correction to correct orientation of the detected plate, and application of the background estimation binarization that is configured to isolate the characters from the detected plate by updating the background into values estimated from local statistics followed by Otsu thresholding.

The method of claim 4, wherein the background estimation binarization comprises the steps of:

i. converting gray scale image of the detected plate into a first binary image using Otsu method;

ii. converting gray scale image of the detected plate into a second binary image using local adaptive method; and

iii. multiplying the first binary image and the second binary image to obtain a fused background image.

The method of claim 6, wherein the first binary image is noise filtered before multiplication. The method of claim 4, wherein the background estimation binarization further comprises the steps of:

i. standardizing gray scale intensity distribution of the gray scaled image of the detected plate; and

ii. computing mean and standard deviation of the gray scaled image, wherein a background value of the gray scaled image is estimated based on the mean and standard deviation.

The method of claim 8, wherein the standard deviation is scaled using a scaling factor, which a multiple estimated background values are generated.

The method of claim 9, wherein the multiple estimated background values gray is based on scaled image to convert into corresponding candidate binary images.

The method of claim 4, wherein the background estimation binarization further comprises the steps of:

i. processing the candidate binary images and the fused background image based on computation of number of isolated blobs in each candidate binary image and fused background image;

ii. determining size of each isolated blob;

iii. comparing the size of each isolated blob with plate height; and

iv. assigning a score to each blob based on a defined criteria, wherein scores for all blobs of each candidate binary image and fused background image are summed;

v. selecting candidate binary image and fused background image with highest score as final binary image and sent to the plate recognition module.