CN107506777A - A kind of real-time more licence plate recognition methods and device based on Wavelet transformation and SVMs - Google Patents

Abstract

The invention discloses a real-time multi-license plate recognition method and device based on wavelet variation and support vector machine, comprising the following steps, step 1, collecting vehicle images; step 2, denoising the images; step 3, according to the The position of the license plate image is determined by the ratio of the number of edge pixels to the area of the rectangle; step 4, the horizontal position of the license plate is determined by the method of line scanning to the license plate image, and the vertical position of the license plate is determined by the method of wavelet layered transformation; step 5, the Each license plate image is binarized, and then the processed license plate image is scanned. When the number of edge pixels in a single column exceeds the preset threshold range, it is judged that the current column contains license plate characters, otherwise the column does not contain License plate characters use it as the boundary of character segmentation; step 6, identify the license plate characters. The invention proposes an accurate, fast and effective solution to multi-license plate recognition.

Description

A real-time multi-license plate recognition method based on wavelet transformation and support vector machine and device

technical field

The invention relates to the technical field of computer vision image processing, in particular to a real-time multi-license plate recognition method and device based on wavelet transformation and support vector machine.

Background technique

License plate recognition technology is an algorithm technology that detects the license plate of a vehicle and accurately recognizes its license plate characters in the actual traffic road scene. Use ordinary cameras to obtain actual traffic road scene images, because the influence of complex external environments, such as weather, road conditions, etc., seriously interferes with the camera's detection and recognition of license plates; moreover, most current license plate recognition algorithms can only detect a single license plate in the image For detection and recognition, accurate positioning and recognition cannot be achieved for multiple license plates in the image. Now license plate recognition technology has become the research focus of computer graphics, and is widely used in real life, such as community parking meter system, traffic intersection safety monitoring system, etc.

License plate detection and license plate character recognition are two key technologies of license plate recognition. License plate detection is the front-end work of license plate recognition. For multiple license plate recognition, license plate detection is particularly important. Only by accurately locating the license plate in the image can subsequent processing work such as recognition of multiple license plates in the image be performed. Using the current single-objective algorithm to simply expand the detection and recognition of multiple license plates, first, its detection effect cannot meet the requirements, because a single license plate is relatively clear and the area in the image is relatively large, while multiple license plates Distributed in various areas of the image, each area is not very large, and it is not particularly clear; second, simply adding its algorithm to multi-license plate detection, the algorithm is complex and the real-time effect is not good. Therefore, in view of the existence of illegal events such as vehicle speeding, illegal overtaking, and red light running on the current traffic roads, there is an urgent need for a method that can realize multi-license plate recognition to stop the occurrence of these traffic violations.

Contents of the invention

The invention proposes a real-time multi-license plate recognition method and device based on wavelet variation and support vector machine which can accurately, quickly and effectively solve multi-license plate recognition.

Technical scheme of the present invention is realized like this:

A real-time multi-license plate recognition method based on wavelet variation and support vector machine, comprising the following steps,

Step 1, collecting vehicle images;

Step 2, denoising the image, specifically performing Laplace transform on the image first, enhancing the edge of the image, then converting the color image into a 256-level grayscale image, and then performing Gaussian blur processing on the image;

Step 3, determine the position of the license plate image according to the size of the ratio of the number of edge pixels in the rectangular frame to the area of the rectangle;

Step 4, adopting the method of line scanning to determine the horizontal position of the license plate to the license plate image, adopting the method of wavelet layered transformation to determine the vertical position of the license plate;

Step 5: Carry out binarization processing on each license plate image, and then perform column scanning on the processed license plate image. When the number of edge pixels in a single column exceeds a preset threshold range, it is judged that the current column contains license plate characters, otherwise The column does not contain the license plate character to use it as a character-separated boundary;

Step 6: Recognize the characters of the license plate, specifically, use the pre-trained SVM support vector machine to perform character recognition on the detected license plate image, obtain the license plate number information, and record the position of the license plate in the vehicle image.

The present invention also provides a device for real-time multiple license plate recognition using the above method, including a video acquisition module for image acquisition; a license plate recognition module for identifying license plates; a power supply module for supplying power to the entire system; Module, used to store system programs and important videos; and display module, used to display scene shooting videos and video frames of illegal traffic accidents; where the video acquisition module collects, power supply module, storage module and display module are all identified with the card Module signal connection.

A real-time multi-license plate recognition method and device based on wavelet variation and support vector machine provided by the present invention can realize accurate detection of multiple license plates in the same image, and can quickly identify them, thereby providing a safe monitoring for traffic supervisors Great help.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained according to these drawings without any creative effort.

Fig. 1 is a kind of flow chart of the real-time many license plate recognition method based on wavelet variation and support vector machine of the present invention;

Fig. 2 is a structural block diagram of a real-time multi-license plate recognition device based on wavelet variation and support vector machine;

Fig. 3 is a schematic diagram of one-time decomposition of wavelet transform.

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

Please refer to Figure 1, a real-time multi-license plate recognition method based on wavelet variation and support vector machine, including the following steps,

Step 1, collect vehicle image; it can carry out the collection of vehicle image of actual scene through camera; In this embodiment, it generates collection signal by processor, controls camera to collect and decode measurement image, and its collection speed is in 35 /Second.

Step 2, denoising the image, specifically performing Laplace transform on the image first, enhancing the edge of the image, then converting the color image into a 256-level grayscale image, and then performing Gaussian blur processing on the image;

Step 3, determine the position of the license plate image according to the size of the ratio of the number of edge pixels in the rectangular frame to the area of the rectangle;

Step 4, adopting the method of line scanning to determine the horizontal position of the license plate to the license plate image, adopting the method of wavelet layered transformation to determine the vertical position of the license plate;

Step 5: Carry out binarization processing on each license plate image, and then perform column scanning on the processed license plate image. When the number of edge pixels in a single column exceeds a preset threshold range, it is judged that the current column contains license plate characters, otherwise The column does not contain the license plate character to use it as a character-separated boundary;

Step 6: Recognize the characters of the license plate, specifically, use the pre-trained SVM support vector machine to perform character recognition on the detected license plate image, obtain the license plate number information, and record the position of the license plate in the vehicle image.

Further, when step 2 is executed, the specific operation principle is as follows:

Step 2-1: Transform template according to Laplace:

Step 2-2: Laplacian edge enhancement processing: Where (x, y) is the coordinate of the vehicle image in the time domain, and c represents the influence factor of the Laplacian operator, and its value range is between 0.4 and 0.7. g(x, y) represents the gray value at (x, y) after enhancement, and f(x, y) represents the gray value at (x, y) before enhancement.

Step 2-3: The color image becomes a grayscale image: Gray(x,y)=0.3R(x,y)+0.59G(x,y)+0.11B(x,y); where R(x,y ) is the gray value of the R channel, G(x, y) is the gray value of the G channel, and B(x, y) is the gray value of the B channel;

Step 2-4: Gaussian filter denoising processing, the Gaussian filter template is:

Further, when performing step 4, the specific steps include:

Step 4-1, line scan the license plate image to determine the horizontal position of the license plate. Specifically, set each line from black to white or from white to black as a jump, and each character will appear at least two during line scanning. Jump, set the jump threshold to 14 times; scan from bottom to top, count the number of jumps in the row scan, if the number of jump points in a row is greater than 14, then judge the row where the license plate is located, and record it as the bottom of the license plate , continue to scan upward line by line until the number of jumps is less than 14, and use it as the top of the license plate; in this step, considering that the license plate has 7 characters, affected by factors such as character breaks, blurring, and license plate tilt, the jump The threshold is set to 14 times, so if the number of jumps in a row is greater than 14, it is considered that this row may be the row where the license plate is located, so it is the bottom of the license plate.

Step 4-2, the vertical projection VPj of each column of M rows of images is obtained by the following formula:

Where f(x, y) represents the gray value of point (x, y), M is the number of rows of the image, i is the x coordinate of the pixel, and j is the y coordinate of the pixel. According to the vertical gray value of the license plate whose horizontal position has been determined, the vertical position of the license plate is determined by using the method of wavelet layered transformation for three-layer decomposition and reconstruction. According to the previous experience of those skilled in the art, the vertical area of the license plate character area usually forms dense peak-trough-peak features, so the left and right boundaries of the license plate can be preliminarily obtained. Then according to the wavelet algorithm, the algorithm implemented by one-dimensional wavelet transform is generally the mallat algorithm. First, the signal with a large scale is transformed to obtain the low-frequency part and high-frequency part of the signal, and then the low-frequency part is decomposed twice. Decompose into low-frequency part and high-frequency part, once decomposed as shown in Figure 3, and then extract useful information from the wavelet to construct the signal (useful information is the high-frequency part of the wavelet decomposed signal), and further determine the vertical direction of the license plate by this method Location.

Further, when performing step 6, the execution steps of its pre-trained SVM support vector machine performing character recognition on the detected license plate image include:

Step 6-1, train the support vector machine according to the abbreviations of 34 provinces, 26 English letters and 10 numeric characters. The training process is as follows:

Collect training sample data, where the samples are standard characters and binarized graphics;

Constructing data features of graphic samples, said data features at least including the ratio of the number of character pixels in a character rectangle to the number of non-character pixels, and the aspect ratio;

The sample is constructed according to the above data characteristics to construct a feature vector, from which two or two features are randomly selected to establish a feature space, and all data features are involved;

According to the sample data, find the feature hyperplane that distinguishes the sample from the non-sample in the feature space. The feature hyperplane is used to judge whether the character conforms to the data characteristics corresponding to the sample, so as to judge whether it is the corresponding character. The hyperplane formula in the two-dimensional feature space as follows:

Among them, w represents the slope of the hyperplane for discriminating characters, x represents the spatial x coordinate, b represents the intersection line between the feature plane and the Y plane, y represents the space ordinate, and i represents the feature sequence number. When the feature hyperplane is greater than 1, it means it belongs to the character class, and if it is less than 1, it does not belong to the character class. The coefficients in the optimal hyperplane formula are calculated by the Langrange equation, and the constraints are as follows:

y _i [wx _i +b]-1≥0,i=1,2,3,...,l

in, The reciprocal of represents the distance between the plane and the sample points in the two categories that need to be separated. According to this formula, the greater the distance between the hyperplane and the sample points in the two categories, the better its discrimination ability is.

The Lagrangian discriminant formula is as follows:

According to the Lagrangian discriminant method, and calculate the following formula, we get:

Get the coefficients w and b of the optimal hyperplane, and finally determine the optimal hyperplane in this space;

Step 6-2, when the optimal hyperplane is found in the corresponding pairwise feature space, the SVM support vector machine is trained.

Further, using the SVM support vector machine to recognize the characters, the specific steps are:

S1: Design the SVM support vector machine training sample process, generate a classifier .xml file, and record the feature values of each character in the file;

S2: Load the .xml file into a classifier, and use the classifier to identify characters.

Further, when performing step 3, the specific steps include:

Step 3-1, searching for contours whose edges are rectangles in the image;

Step 3-2, counting the number of edge pixels falling in each rectangle;

Step 3-3, calculate the ratio of the number of edge pixels falling within the rectangular frame to the area, when the ratio is greater than the preset threshold 0.78, it is judged as the position of the license plate image, otherwise it is judged as a non-license plate area.

Further, when performing step 5, the specific steps include:

Perform binarization processing on each license plate image, the binarization is 0 or 255, and the threshold is 80;

Scan the image in columns, count the number of pixels with a pixel value of 255 in each column, and then calculate the ratio of the number of edge pixels in a single column to the length of the column. When the number of edge pixels in a single column exceeds the set threshold range, it is judged that the current column contains license plate characters , otherwise the column does not contain the license plate character to use it as a character splitting boundary.

A real-time multi-license plate recognition device based on wavelet transformation and support vector machine, including a video acquisition module for image acquisition; a license plate recognition module for recognizing license plates; a power supply module for supplying power to the entire system; a storage module , used to store system programs and important videos; and a display module, used to display scene shooting videos and video frames of illegal traffic accidents; where the video acquisition module, power supply module, storage module and display module are all connected with the card recognition module signal connection.

Further, the license plate recognition module includes a Texas Instruments (T1 company) TMS320DM8168 digital media processor, TMS320DM8168 is a DSP processor for multimedia applications, which integrates a 1.2GHz Cortex ^TM -A8 processor and 1GHz C674x+ ^TM floating-point DSP processor, and has 3 high-definition video coprocessors (HDVICP), which can support simultaneous processing of 16 channels of 720p30fps synchronous video stream. Because of its high-speed real-time computing capability, special video interface and rich expansion interface, it has been widely used in the multimedia field.

The video acquisition module consists of a high-definition CCD analog camera and TI's TVP5158 decoder. TVP5158 can automatically control the contrast, reduce noise, improve the compression ratio and the overall video quality. The license plate recognition module is composed of TMS320DM8168 digital media processor, on which the code is written to complete the above-mentioned license plate recognition function. The power supply module is powered by the vehicle power supply, and outputs 1.8V, 3.3V, 5V and 12V voltages through a wide voltage input DC voltage stabilization chip and a conversion chip to complete the power supply for the entire system. The storage module is composed of FLASH memory, DDR3 memory and hard disk with SATA interface. The display module is composed of a high-definition display with an HDMI interface. After completing the high-definition display of the video, the license plate will be clearer, which is beneficial to the license plate recognition of the recognition module. The device provided by the invention adopts a multi-thread programming mode, and realizes parallel operations in acquisition, identification and display, thereby improving the efficiency of the algorithm.

A real-time multi-license plate recognition method and device based on wavelet variation and support vector machine provided by the present invention adopts high-efficiency and parallel processors in hardware, and uses algorithms and support vector machines for accurate and effective recognition in terms of implementation methods. Recognize multiple license plates in the same image, so it can be applied to traffic, parking lots and other places, providing great help to supervisors.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of the present invention. within the scope of protection.

Claims (8)

1. A real-time multi-license plate recognition method based on wavelet change and a support vector machine is characterized by comprising the following steps,

step 1, collecting a vehicle image;

step 2, denoising the image, specifically, performing laplace transform on the image, enhancing the edge of the image, converting the color image into a 256-level gray image, and performing Gaussian blur processing on the image;

step 3, determining the position of the license plate image according to the ratio of the number of edge pixels in the rectangular frame to the rectangular area;

step 4, determining the horizontal position of the license plate by adopting a line scanning method and determining the vertical position of the license plate by adopting a wavelet hierarchical transformation method;

step 5, performing binarization processing on each license plate image, then performing row scanning on the processed license plate image, judging that the current row contains license plate characters when the number of single-row edge pixels exceeds a preset threshold range, and otherwise, judging that the row does not contain the license plate characters and using the license plate characters as character segmentation boundaries;

and 6, recognizing the characters of the license plate, specifically, recognizing the characters of the detected license plate image by using a pre-trained SVM (support vector machine) to obtain license plate information, and recording the position of the license plate in the vehicle image.

2. The real-time multi-license-plate recognition method based on wavelet transformation and support vector machine as claimed in claim 1, wherein when step 2 is executed, the specific operation principle is as follows:

step 2-1: according to the Laplace transform template:

step 2-2: laplace edge enhancement processing:wherein (x, y) is the coordinates of the vehicle image in the time domain, and c represents the influence factor of the Laplace operator, and the value range of the influence factor is 0.4-0.7. g (x, y) represents the grayscale value at (x, y) after the enhancement, and f (x, y) is the grayscale value at (x, y) before the enhancement processing.

Step 2-3, changing the color RGB image into a gray image: gray (x, y) ═ 0.3R (x, y) +0.59G (x, y) +0.11B (x, y); wherein R (x, y) is the gray scale value of the R channel, G (x, y) is the gray scale value of the G channel, and B (x, y) is the gray scale value of the B channel;

step 2-4: and (3) carrying out Gaussian filtering denoising treatment, wherein a Gaussian filtering template is as follows:

3. the real-time multi-license-plate recognition method based on wavelet transformation and support vector machine as claimed in claim 1, wherein when step 4 is executed, the specific steps include:

step 4-1, scanning lines of a license plate image to determine the horizontal position of the license plate, specifically marking each line from black to white or from white to black as one jump, generating at least two jumps on each character during line scanning, and setting a jump threshold value to be 14 times; scanning is carried out from bottom to top, the jumping number of line scanning is counted, if the jumping point number of a certain line is larger than 14, the line is judged to be the line where the license plate is located, the line is marked as the bottom of the license plate, the line is continuously scanned line by line upwards until the jumping number is smaller than 14, and the line is used as the top of the license plate;

step 4-2, the vertical projection VPj of each column of the M row image is obtained by:

where f (x, y) represents the gray scale value of the (x, y) point, M is the number of lines in the image, i is the pixel x coordinate, and j is the pixel y coordinate.

And according to the gray value of the license plate with the determined horizontal position in the vertical direction, performing three-layer decomposition and reconstruction by adopting a wavelet hierarchical transformation method to determine the vertical position of the license plate.

4. The real-time multi-license plate recognition method based on wavelet transformation and support vector machine as claimed in claim 1, wherein when step 6 is executed, the execution step of character recognition of the detected license plate image by the support vector machine of SVM trained in advance comprises:

step 6-1, training the support vector machine according to the abbreviation of 34 provinces, 26 English letters and 10 numeric characters, wherein the training process is as follows:

collecting training sample data, wherein the sample is a standard character and is a binary figure;

constructing data characteristics of a graphic sample, wherein the data characteristics at least comprise the ratio of the number of character pixels in a rectangle of the character to the number of non-character pixels and the aspect ratio;

constructing a feature vector for the sample according to the data features, and randomly selecting two features from the feature vector to establish a feature space, wherein all the data features relate to;

searching a characteristic hyperplane for distinguishing a sample from a non-sample in a characteristic space according to the sample data, wherein the characteristic hyperplane is used for judging whether the character accords with the data characteristic corresponding to the sample so as to judge whether the character is a corresponding character, and a hyperplane formula in a two-dimensional characteristic space is as follows:

wherein w represents the slope of the hyperplane of the discrimination character, x represents the spatial x coordinate, b represents the intersection line of the feature plane and the Y plane, Y represents the spatial ordinate, and i represents the feature serial number. When the characteristic hyperplane is larger than 1, the characteristic hyperplane belongs to the character class, and when the characteristic hyperplane is smaller than 1, the characteristic hyperplane does not belong to the character class, the coefficients in the optimal hyperplane formula are calculated by a Langerian equation, and the constraint conditions are as follows:

y_i[wx_i+b]-1≥0,i＝1,2,3,...,l

wherein,the inverse of (a) represents the distance of the plane from the sample points of the two classes that need to be separated,

the lagrangian discriminant is as follows:

according to the Lagrange discrimination method, calculating the following formula to obtain:

obtaining the coefficients w and b of the optimal hyperplane, and finally determining the optimal hyperplane in the space;

and 6-2, when the optimal hyperplane is found in the corresponding pairwise feature spaces, training the SVM (support vector machine).

5. The real-time multi-license-plate recognition method based on wavelet transformation and support vector machine as claimed in claim 1, wherein when step 3 is executed, the specific steps include:

step 3-1, searching the outline with rectangular edge in the image;

step 3-2, counting the number of edge pixels falling in each rectangle;

and 3-3, calculating the ratio of the number of edge pixels in the rectangular frame to the area, judging the position of the license plate image when the ratio is larger than a preset threshold value of 0.78, and otherwise, judging the license plate image as a non-license plate area.

6. The real-time multi-license-plate recognition method based on wavelet transformation and support vector machine as claimed in claim 1, wherein when step 5 is executed, the specific steps include:

carrying out binarization processing on each license plate image, wherein the binarization value is 0 or 255, and the threshold value is 80;

and (3) performing column scanning on the image, counting the number of pixels of each column with the pixel value of 255, then calculating the ratio of the number of single-column edge pixels to the length of the column, judging that the current column contains license plate characters when the number of single-column edge pixels exceeds a set threshold range, and otherwise, judging that the column does not contain the license plate characters and using the column as a character segmentation boundary.

7. A device for real-time multi-license plate recognition using the method of claim 1, comprising a video capture module for image capture; the license plate recognition module is used for recognizing the license plate; the power supply module is used for supplying power to the whole system; the storage module is used for storing the system program and the important video; the display module is used for displaying scene shooting videos and video frames with illegal traffic accidents; the video acquisition module, the power module, the storage module and the display module are in signal connection with the card identification module.

8. The apparatus of claim 7, wherein the license plate recognition module comprises a texas instruments TMS320DM8168 digital media processor.