WO2019169532A1

WO2019169532A1 - License plate recognition method and cloud system

Info

Publication number: WO2019169532A1
Application number: PCT/CN2018/078044
Authority: WO
Inventors: 梁昊; 南一冰; 廉士国
Original assignee: 深圳前海达闼云端智能科技有限公司
Priority date: 2018-03-05
Filing date: 2018-03-05
Publication date: 2019-09-12
Also published as: CN108701234A

Abstract

Provided in the present application are a license plate recognition method and a cloud system. The method comprises: performing detection with respect to acquired image information to produce a license plate image and boundary information thereof; correcting the license plate image on the basis of the boundary information to produce a corrected license plate image; and, extracting multiple character images in the corrected license plate image and utilizing multiple preset recognition models in recognizing the multiple character images to produce a license plate recognition result. Compared with conventional license plate recognition, the present application satisfies the demand for accurate recognition of license plates in versatile scenarios, is not limited to specific scenarios such as overhead gantries, and provides strengthened robustness.

Description

License plate recognition method and cloud system

Technical field

The present application relates to the field of deep learning technology, and in particular to a license plate recognition method and a cloud system.

Background technique

With the explosive growth of the number of vehicles, the management difficulty of the vehicle is increasing, especially in the monitoring of vehicle violations, and in the process of checking the license plate, the license plate number as the vehicle identity is the key information to be extracted. Therefore, the automatic extraction of the vehicle license plate information by the computer can effectively reduce the workload in the vehicle management process and liberate considerable human resources, and the license plate recognition mainly includes three steps of license plate location, character extraction and character recognition, as follows:

The traditional license plate location mainly relies on a certain feature to find a candidate frame that may be a license plate in the image. All the candidate frames are judged by the classifier whether it is a two-category of the license plate, but the environment of the vehicle is complicated. Variable, using one or more specific features can not effectively locate the license plate in the general scene, so the traditional license plate recognition is mainly for specific scenes such as bayonet, which greatly affects the promotion of license plate recognition in practical applications.

The traditional character extraction is mainly divided into the connected domain method and the projection method. The connected domain method is to binarize the positioned license plate, and realizes the segmentation of characters by analyzing the connected domain, thereby extracting the binarized image of the character; The projection method is to divide and extract characters according to the distribution of peaks and troughs in the grayscale projection curve of the license plate. In the case of low camera pixels or excessive image noise, the traditional character extraction problem is that for the connected domain method, the noise causes the error of the binarization processing to be large, which results in a large influence on the extracted character features. For the projection method, the noise affects the distribution of the peaks and valleys, so that the peaks and troughs in the graph are relatively inconspicuous or redundant peaks and troughs appear, which has a great influence on the correct segmentation of characters. It can be seen that the traditional character extraction has higher requirements on hardware devices, and the application promotion of license plate recognition is hindered.

The traditional character recognition is mainly divided into template matching and feature classifier-based recognition methods. The template matching based recognition method is to match the characters to be recognized in the template library by constructing the template library to realize the recognition of characters; The classifier is identified by extracting a certain feature of the character and using the feature classifier to realize the recognition of the character. The problems with the two recognition methods are: the recognition method based on template matching has higher requirements on the environment, such as the damage of the license plate, or the change of illumination will affect the matching result of the character to be recognized and the template; the recognition method based on the feature classifier needs to be targeted Specific scenes extract specific features, which hinder their promotion in general scenarios.

With the continuous promotion of deep learning in various technical fields, the convolutional neural network is gradually used to realize the recognition of license plates. However, based on the characteristics of small characters in license plate images, it is not possible to use convolutional neural networks with basic functions. The extraction of character features can not effectively identify the characters, and the domestic license plate has more character categories, which leads to the limited promotion of traditional license plate recognition in China.

Summary of the invention

The embodiment of the present application proposes a license plate recognition method and a cloud system, so as to solve the technical problem that the existing license plate recognition is mainly for a single scene such as a bayonet, and the hardware device is required to be high, resulting in poor practicability in a general scenario.

In one aspect, an embodiment of the present application provides a license plate recognition method, including:

Detecting the acquired image information to obtain a license plate image and its boundary information;

Performing correction processing on the license plate image according to the boundary information to obtain a corrected license plate image;

A plurality of character images in the corrected license plate image are extracted, and the plurality of character images are identified by using a plurality of preset recognition models to obtain a license plate recognition result.

In another aspect, an embodiment of the present application provides a license plate recognition cloud system, including:

a detecting module, configured to detect the acquired image information, and obtain a license plate image and boundary information thereof;

a correction module, configured to perform correction processing on the license plate image according to the boundary information, to obtain a corrected license plate image;

The identification module is configured to extract a plurality of character images in the corrected license plate image, and identify the plurality of character images by using a plurality of preset recognition models to obtain a recognition result of the license plate.

In another aspect, an embodiment of the present application provides an electronic device, where the electronic device includes:

Transceiver, memory, one or more processors;

One or more modules, the one or more modules being stored in the memory and configured to be executed by the one or more processors, the one or more modules comprising Instructions for each step.

In another aspect, embodiments of the present application provide a computer program product for use with an electronic device, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism Instructions are included for performing the various steps in the above methods.

The benefits are as follows:

In this embodiment, by detecting the acquired image information, a license plate image and its boundary information are obtained, and the license plate image is corrected according to the boundary information to obtain a corrected license plate image, and the corrected license plate image is extracted. The plurality of character images in the plurality of character images are identified by using a plurality of preset recognition models to obtain a recognition result of the license plate. Compared with the traditional license plate recognition, the present application satisfies the requirement for accurate recognition of the license plate in the general scene, and is not limited to a specific scene such as a bayonet, and has strong robustness.

DRAWINGS

Specific embodiments of the present application will be described below with reference to the accompanying drawings, in which:

1 is a schematic diagram of a method for license plate recognition in the first embodiment of the present application;

2 is a schematic diagram of a license plate angle of a license plate recognition in the first embodiment of the present application;

3 is a structural diagram of a cloud system for license plate recognition in Embodiment 2 of the present application;

FIG. 4 is a schematic structural diagram of an electronic device according to Embodiment 3 of the present application.

Detailed ways

The essence of the technical solution of the embodiment of the present invention is further clarified by specific examples below.

The exemplary embodiments of the present application are further described in detail below with reference to the accompanying drawings, in which the embodiments described are only a part of the embodiments of the present application, but not all embodiments. An exhaustive example. And in the case of no conflict, the features in the embodiments and the embodiments in the description can be combined with each other.

The inventor noticed during the invention:

The three steps of license plate location, character extraction and character recognition included in the traditional license plate recognition have certain drawbacks. For the license plate location, due to the complex and varied environment of the vehicle, it is impossible to use one or more specific features. In the general scene, the license plate is effectively positioned; for character extraction, when the camera pixel is low or the image noise is too large, the extracted character features are greatly affected; for character recognition, the requirements for the environment of the vehicle are more High, or need to extract specific features for a specific scene, so its promotion in the general scene is hindered.

In view of the above disadvantages/based on the above, the embodiment of the present application proposes to obtain a license plate image and its boundary information by using multiple detection techniques according to the acquired image, correcting the license plate image according to the boundary information, and extracting the corrected license plate image. The plurality of character images, and the plurality of recognized recognition models are used to identify the extracted plurality of character images, and the recognition result of the license plate is output, thereby realizing accurate recognition of the large inclination license plate and improving the license plate recognition for different scenarios. Robustness, while meeting the need for accurate identification of license plates in a common scenario.

In order to facilitate the implementation of the present application, the following examples are described.

Example 1

FIG. 1 is a schematic diagram of a method for license plate recognition in the first embodiment of the present application. As shown in FIG. 1 , the method includes:

Step 101: Detect the acquired image information to obtain a license plate image and boundary information thereof.

Step 102: Perform correction processing on the license plate image according to the boundary information to obtain a corrected license plate image.

Step 103: Extract a plurality of character images in the corrected license plate image, and identify the plurality of character images by using a plurality of preset recognition models to obtain a recognition result of the license plate.

In step 101, a coarse license plate image is obtained by using rough detection based on depth learning, and the license plate image and the boundary information are obtained by performing edge detection on the edge information and color information in the rough license plate image, that is, the multiplex detection technology ensures The accurate positioning of the large dip license plate boundary and the license plate detection based on the feature information such as edge and color make the present application more robust to the detection of the license plate for different scenes, so that it is suitable for the general scene without being limited to the bayonet, etc. Specific scenes.

In step 102, unlike the prior art, when the license plate image is direction-corrected, the license plate is usually tilt-corrected by direct rotation. The present application uses a affine transformation process to obtain a rule for the tilting and skewing license plate. The license plate rectangular image, thereby effectively improving the accuracy of extracting the plurality of character images in the corrected license plate image in step 103.

In this embodiment, the acquired image information is detected to obtain a license plate image and boundary information thereof, including:

Using a preset neural network model, obtaining a license plate image to be determined according to the acquired image information;

The license plate image and its boundary information are obtained according to the edge information and the color information in the license plate image to be determined.

In the implementation, the preset neural network model is a pre-trained deep learning-based detection model, that is, correct classification and labeling of the collected image information, and deep learning of the initialization of the construction through forward transmission and reverse conduction. The network parameters in the network are trained to obtain a trained deep learning-based detection model.

In the implementation, using the depth learning-based detection model, the obtained image information is subjected to rough detection of the license plate, and the circumscribed rectangle information of the license plate (for example, the boundary coordinates of the circumscribed rectangle) is obtained, the rough license plate image is located, and the rough license plate image is extracted. The edge information and color information are used to detect the extracted edge information and color information by Sobel operator and color matching method to locate the accurate license plate image and its boundary information.

In this embodiment, the license plate image is corrected according to the boundary information, and the corrected license plate image is obtained, including:

Calculating a tilt angle and a skew angle of the license plate according to the boundary information;

The license plate image is affine transformed according to the inclination angle and the skew angle of the license plate to obtain a corrected license plate image.

In the implementation, FIG. 2 is a schematic diagram of the license plate angle of the license plate recognition in the first embodiment of the present application. As shown in FIG. 2, the boundary information of the license plate image includes the shape and boundary coordinates of the license plate, and is calculated according to the shape and boundary coordinates of the license plate. License plate tilt angle θ and skew angle

And according to the inclination angle θ and the skew angle

The plaque image is affine transformed to obtain a corrected license plate image.

In this embodiment, extracting a plurality of character images in the corrected license plate image includes:

Performing binarization and morphological processing on the corrected license plate image to obtain position information of a plurality of characters in the corrected license plate image;

Performing grayscale processing on the corrected license plate image to obtain a license plate grayscale image;

Character extraction is performed on the license plate grayscale image according to position information of the plurality of characters, and a plurality of character images in the corrected license plate image are obtained.

In the implementation, the corrected license plate image is sequentially binarized, closed, contoured, and minimum circumscribed rectangle processed, and the connected domain in the corrected license plate image is calculated, and according to the connected domain image in the corrected license plate image. The characters are split to get the character position. At the same time, the corrected license plate image is grayscaled to obtain a license plate grayscale image, and a plurality of character grayscale images in the license plate grayscale image are extracted according to the obtained character position.

In this embodiment, the plurality of preset recognition models include a recognition model for recognizing characters, and a recognition model for recognizing English letters and numbers.

In the implementation, the preset recognition model is a pre-trained deep learning-based recognition model, that is, the correct classification and labeling of the collected character images, and the built-in deep learning network through forward transmission and reverse conduction. The network parameters in the training are trained to obtain a first recognition model for recognizing text based on deep learning, a second recognition model for recognizing English letters and numbers, and a number for recognizing characters, English letters and numbers. The three recognition model, that is, by training multiple character classifiers, reduces the category that the single character classifier needs to classify, so as to improve the accuracy of license plate recognition. Among them, the number of identification models can be set according to the actual situation, and the implementation does not limit the number of identification models.

Embodiment, the recognition model using a plurality of the extracted plurality of character recognition gray image, specifically, the character A ₁ grayscale image into a first recognition model, the recognition result to obtain the output corresponding to Z _1; character The grayscale images A ₂ -A ₆ are respectively imported into the second recognition model, and the corresponding recognition results Z ₂ -Z _{6 are} outputted; the character grayscale image A _{7 is} imported into the third recognition model, and the corresponding recognition result is output. Z ₇ , thereby obtaining the recognition result of the license plate.

This application takes a specific scenario as an example to describe the embodiment 1 of the present application in detail. The specific process is as follows:

Step 201: Perform offline training on the deep learning network by using a pre-established data set to obtain a deep learning model M ₁ for online license plate detection, and a deep learning model M ₂ , M ₃ , M for online character recognition. ₄ .

Step 202: acquiring image information, using the depth learning model M ₁ on the image information coarsely detected, to give a rough plate image, extracts edge and color information rough plate image, using the Sobel operator and a color matching method and the extracted Edge information and color information are used to detect the license plate and locate the accurate license plate image and its boundary information.

Step 203: Perform correction processing on the detected tilted or skewed license plate image by using affine transformation according to the boundary information of the license plate image, and perform binarization and morphological processing on the corrected license plate image to obtain correction processing. The position information of 7 characters in the license plate image, at the same time, grayscale the corrected license plate image to obtain a license plate grayscale image, and extract 7-character grayscale images in the license plate grayscale image according to the obtained character position.

Step 204: Identify the extracted 7-character grayscale image by using the pre-trained deep learning models M ₂ , M ₃ , M ₄ , respectively output the license plate recognition result, and jump to step 202 to continue to the next one. Image information for license plate recognition.

Example 2

Based on the same inventive concept, a cloud system for license plate recognition is also provided in the embodiment of the present application. Since the principle of solving the problem of these devices is similar to the method for identifying a license plate, the implementation of these devices can be referred to the implementation of the method, and the method is repeated. I won't go into details here.

FIG. 3 is a structural diagram of a cloud system for license plate recognition in the second embodiment of the present application. As shown in FIG. 3, the license plate recognition cloud system 300 may include: a detection module 301, a correction module 302, and an identification module 303.

The detecting module 301 is configured to detect the acquired image information to obtain a license plate image and boundary information thereof.

The correction module 302 is configured to perform correction processing on the license plate image according to the boundary information to obtain a corrected license plate image.

The identification module 303 is configured to extract a plurality of character images in the corrected license plate image, and identify the plurality of character images by using a plurality of preset recognition models to obtain a recognition result of the license plate.

In this embodiment, the detecting module 301 includes:

In this embodiment, the correction module 302 includes:

Example 3

Based on the same inventive concept, an electronic device is also provided in the embodiment of the present application. Since the principle is similar to the method for identifying a license plate, the implementation of the method may refer to the implementation of the method, and the repeated description is not repeated.

4 is a schematic structural diagram of an electronic device in Embodiment 3 of the present application. As shown in FIG. 4, the electronic device includes: a transceiver device 401, a memory 402, one or more processors 403, and one or more modules. The one or more modules are stored in the memory and configured to be executed by the one or more processors, the one or more modules including steps for performing the steps of any of the above methods instruction.

Example 4

Based on the same inventive concept, the embodiment of the present application further provides a computer program product for use in combination with an electronic device. Since the principle is similar to a method for license plate recognition, the implementation can refer to the implementation of the method, and the repetition is no longer Narration. The computer program product comprises a computer readable storage medium and a computer program mechanism embodied therein, the computer program mechanism comprising instructions for performing the various steps of any of the above methods.

For the convenience of description, the various parts of the above-described apparatus are separately described by functions into various modules. Of course, the functions of each module or unit may be implemented in the same software or hardware in the implementation of the present application.

Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system, or computer program product. Thus, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment in combination of software and hardware. Moreover, the application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

While the preferred embodiment of the present application has been described, it will be apparent that those skilled in the art can make further changes and modifications to the embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and the modifications and

Claims

A license plate recognition method, comprising:

Detecting the acquired image information to obtain a license plate image and its boundary information;

Performing correction processing on the license plate image according to the boundary information to obtain a corrected license plate image;

A plurality of character images in the corrected license plate image are extracted, and the plurality of character images are identified by using a plurality of preset recognition models to obtain a license plate recognition result.
The method according to claim 1, wherein the detecting the acquired image information to obtain a license plate image and boundary information thereof comprises:

Using a preset neural network model, obtaining a license plate image to be determined according to the acquired image information;

The license plate image and its boundary information are obtained according to the edge information and the color information in the license plate image to be determined.
The method according to claim 1, wherein the correcting the license plate image according to the boundary information to obtain a corrected license plate image comprises:

Calculating a tilt angle and a skew angle of the license plate according to the boundary information;

The license plate image is affine transformed according to the inclination angle and the skew angle of the license plate to obtain a corrected license plate image.
The method according to claim 1 or 3, wherein the extracting the plurality of character images in the corrected license plate image comprises:

Performing binarization and morphological processing on the corrected license plate image to obtain position information of a plurality of characters in the corrected license plate image;

Performing grayscale processing on the corrected license plate image to obtain a license plate grayscale image;

Character extraction is performed on the license plate grayscale image according to position information of the plurality of characters, and a plurality of character images in the corrected license plate image are obtained.
The method of claim 1 wherein said plurality of predetermined recognition models comprise an identification model for recognizing text and a recognition model for recognizing English letters and numbers.
A license plate recognition cloud system, comprising:

a detecting module, configured to detect the acquired image information, and obtain a license plate image and boundary information thereof;

a correction module, configured to perform correction processing on the license plate image according to the boundary information, to obtain a corrected license plate image;

The identification module is configured to extract a plurality of character images in the corrected license plate image, and identify the plurality of character images by using a plurality of preset recognition models to obtain a recognition result of the license plate.
The cloud system of claim 6, wherein the detecting module comprises:

Using a preset neural network model, obtaining a license plate image to be determined according to the acquired image information;

The license plate image and its boundary information are obtained according to the edge information and the color information in the license plate image to be determined.
The cloud system of claim 6 wherein said correction module comprises:

Calculating a tilt angle and a skew angle of the license plate according to the boundary information;

The license plate image is affine transformed according to the inclination angle and the skew angle of the license plate to obtain a corrected license plate image.
The cloud system according to claim 6 or 8, wherein the extracting the plurality of character images in the corrected license plate image comprises:

Performing binarization and morphological processing on the corrected license plate image to obtain position information of a plurality of characters in the corrected license plate image;

Performing grayscale processing on the corrected license plate image to obtain a license plate grayscale image;

Character extraction is performed on the license plate grayscale image according to position information of the plurality of characters, and a plurality of character images in the corrected license plate image are obtained.
The cloud system according to claim 6, wherein said plurality of preset recognition models include a recognition model for recognizing characters, and a recognition model for recognizing English letters and numbers.
An electronic device, comprising:

Transceiver, memory, one or more processors;

One or more modules stored in the memory and configured to be executed by the one or more processors, the one or more modules including for performing claim 1 The instructions of the various steps in any of the methods described in 5.
A computer program product for use in conjunction with an electronic device, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism comprising means for performing any of claims 1-5 An instruction for each step in the method.