CN116503848A

CN116503848A - Intelligent license plate recognition method, device, equipment and storage medium

Info

Publication number: CN116503848A
Application number: CN202310760385.0A
Authority: CN
Inventors: 姜华
Original assignee: Shenzhen Qianhai Rheniuting Technology Co ltd
Current assignee: Shenzhen Qianhai Rheniuting Technology Co ltd
Priority date: 2023-06-27
Filing date: 2023-06-27
Publication date: 2023-07-28
Anticipated expiration: 2043-06-27
Also published as: CN116503848B

Abstract

The invention relates to the field of artificial intelligence, and discloses an intelligent license plate recognition method, device, equipment and storage medium, which are used for improving the accuracy of intelligent license plate recognition. Comprising the following steps: extracting vehicle contours from the image information set to generate a plurality of vehicle contour information; carrying out vector diagram conversion on the license plate information set to generate a plurality of license plate vector diagrams; collecting multiple frames of running images of a target vehicle, carrying out license plate positioning on each frame of running image, generating a plurality of license plate position information, carrying out license plate recognition on the multiple frames of running images, and generating an initial license plate recognition result; extracting vehicle contours from the multi-frame driving images to generate vehicle contours to be processed, comparing the vehicle contours to be processed with a plurality of vehicle contour information to generate corresponding target contour information; and correcting the accuracy of the initial license plate recognition result based on the target contour information to generate a target license plate recognition result.

Description

Intelligent license plate recognition method, device, equipment and storage medium

Technical Field

The present invention relates to the field of artificial intelligence, and in particular, to a method, apparatus, device and storage medium for intelligent license plate recognition.

Background

The vehicle recognition and license plate recognition technology has important application value in the fields of traffic management, intelligent traffic systems and security protection. By analyzing and processing the vehicle image, the automatic detection, tracking and recognition of the vehicle can be realized, and effective support is provided for traffic supervision and vehicle management. The license plate recognition technology can automatically extract license plate information of the vehicle and is used for management, tracking and safety monitoring of the vehicle.

However, the existing vehicle contour extraction algorithm has insufficient robustness to factors such as illumination change, shielding, complex background and the like, and errors and inaccuracy of contour extraction are easy to cause. The license plate positioning algorithm is easy to be interfered in a complex environment, such as shielding, blurring and the like, so that positioning errors or missed positioning phenomena are caused. Because of the diversity and complexity of license plates, including different license plate types, fonts, colors, etc., the existing license plate recognition algorithm has a certain limitation on accuracy. Real-time processing of multi-frame running images is very important for scenes such as highways, but the prior art has a certain bottleneck in processing speed.

Disclosure of Invention

The invention provides an intelligent license plate recognition method, device, equipment and storage medium, which are used for improving the accuracy of intelligent license plate recognition.

The first aspect of the invention provides an intelligent license plate recognition method, which comprises the following steps:

collecting an image information set and a license plate information set of a plurality of vehicles at preset positions, extracting vehicle contours from the image information set, and generating a plurality of vehicle contour information;

performing vector diagram conversion on the license plate information set to generate a plurality of license plate vector diagrams;

collecting multi-frame driving images of a target vehicle in a high-speed driving state, and respectively carrying out license plate positioning on each frame of driving image to generate a plurality of license plate position information;

based on the license plate position information, carrying out license plate recognition on a plurality of frames of running images through a plurality of license plate vector diagrams, and generating an initial license plate recognition result;

extracting vehicle contours of the multi-frame driving images, generating corresponding vehicle contours to be processed, and comparing the vehicle contours to be processed with a plurality of pieces of vehicle contour information to generate corresponding target contour information;

and correcting the accuracy of the initial license plate recognition result based on the target contour information to generate a target license plate recognition result.

With reference to the first aspect, in a first implementation manner of the first aspect of the present invention, the performing vector diagram conversion on the license plate information set to generate a plurality of license plate vector diagrams includes:

Character segmentation is carried out on the license plate information sets to generate a plurality of license plate character sets, and each license plate character set is subjected to coding processing to generate a plurality of license plate coding sets;

and carrying out vector diagram conversion on a plurality of license plate coding sets to obtain a license plate vector diagram corresponding to each license plate coding set.

With reference to the first implementation manner of the first aspect, in a second implementation manner of the first aspect of the present invention, the performing character segmentation on the license plate information set to generate a plurality of license plate character sets, and performing encoding processing on each license plate character set to generate a plurality of license plate encoding sets includes:

performing binarization processing on the license plate information set to generate a corresponding binarized license plate data set;

performing character segmentation on the binarized license plate data set through a character segmentation algorithm to generate a plurality of license plate character sets;

character feature extraction is carried out on each license plate character set respectively, and a character feature set corresponding to each license plate character set is generated;

respectively carrying out feature clustering on character feature sets corresponding to each license plate character set to generate a plurality of clustered license plate features;

And carrying out coding processing on each license plate character set based on the plurality of clustered license plate features to generate a plurality of license plate coding sets.

With reference to the first implementation manner of the first aspect, in a third implementation manner of the first aspect of the present invention, performing vector diagram conversion on the plurality of license plate code sets to obtain a license plate vector diagram corresponding to each license plate code set includes:

traversing each license plate code set to determine a plurality of code identifiers;

based on a preset vector diagram template database, vector diagram template matching is carried out through a plurality of code identifiers, and a plurality of vector diagram templates are determined;

and carrying out vector diagram conversion on each license plate coding set based on a plurality of vector diagram templates to obtain a license plate vector diagram corresponding to each license plate coding set.

With reference to the first aspect, in a fourth implementation manner of the first aspect of the present invention, the generating an initial license plate recognition result by performing license plate recognition on a plurality of frames of the running image through a plurality of license plate vector diagrams based on a plurality of license plate position information includes:

performing region segmentation on the multi-frame running image based on a plurality of license plate position information to generate a license plate region image corresponding to each frame of running image;

Based on the license plate vector diagrams corresponding to each license plate coding set, carrying out similarity recognition on license plate region images corresponding to each driving image frame, and determining a corresponding similarity recognition result;

carrying out region segmentation on license plate region images corresponding to each frame of the driving image according to the similarity recognition result to generate a plurality of license plate region images;

and carrying out character recognition on each license plate area image to generate an initial license plate recognition result.

With reference to the fourth implementation manner of the first aspect, in a fifth implementation manner of the first aspect of the present invention, the performing, based on a license plate vector diagram corresponding to each license plate coding set, similarity recognition on a license plate area image corresponding to each frame of the running image, and determining a corresponding similarity recognition result includes:

performing matrix conversion on license plate vector diagrams corresponding to each license plate coding set to generate a plurality of matrixes to be processed;

performing matrix analysis on license plate region images corresponding to the driving images of each frame to generate a plurality of matrixes to be compared;

and carrying out similarity recognition on the matrixes to be processed and the matrixes to be compared, and determining the similarity recognition result.

With reference to the first aspect, in a sixth implementation manner of the first aspect of the present invention, the extracting a vehicle contour from the multi-frame driving image to generate a corresponding vehicle contour to be processed, and comparing the vehicle contour to be processed with a plurality of pieces of vehicle contour information to generate corresponding target contour information includes:

extracting vehicle contours of the multi-frame driving images through an edge detection algorithm, and generating corresponding vehicle contours to be processed;

performing contour comparison on the contour of the vehicle to be processed and the plurality of vehicle contour information through a shape matching algorithm, and determining corresponding comparison results;

performing threshold analysis on the comparison result to determine a corresponding threshold analysis result;

and carrying out contour screening on a plurality of vehicle contours according to the threshold analysis result to generate the target contour information.

The second aspect of the present invention provides an intelligent license plate recognition device, which includes:

the acquisition module is used for acquiring an image information set and a license plate information set of a plurality of vehicles at preset positions, extracting vehicle contours from the image information set and generating a plurality of vehicle contour information;

The conversion module is used for carrying out vector diagram conversion on the license plate information set to generate a plurality of license plate vector diagrams;

the positioning module is used for acquiring multi-frame driving images of the target vehicle in a high-speed driving state, and respectively positioning license plates of each frame of driving images to generate a plurality of license plate position information;

the identification module is used for carrying out license plate identification on a plurality of frames of running images through a plurality of license plate vector diagrams based on the license plate position information, and generating an initial license plate identification result;

the extraction module is used for extracting vehicle contours of the multi-frame driving images, generating corresponding vehicle contours to be processed, and comparing the vehicle contours to be processed with a plurality of pieces of vehicle contour information to generate corresponding target contour information;

and the correction module is used for correcting the accuracy of the initial license plate recognition result based on the target contour information to generate a target license plate recognition result.

The third aspect of the present invention provides an intelligent license plate recognition device, comprising: a memory and at least one processor, the memory having instructions stored therein; and the at least one processor invokes the instruction in the memory to enable the intelligent recognition device of the license plate to execute the intelligent recognition method of the license plate.

A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored therein that, when executed on a computer, cause the computer to perform the above-described intelligent license plate recognition method.

In the technical scheme provided by the invention, an image information set and a license plate information set of a plurality of vehicles at preset positions are collected, and vehicle contour extraction is carried out on the image information set to generate a plurality of vehicle contour information; performing vector diagram conversion on the license plate information set to generate a plurality of license plate vector diagrams; collecting multi-frame driving images of a target vehicle in a high-speed driving state, and respectively carrying out license plate positioning on each frame of driving image to generate a plurality of license plate position information; based on the license plate position information, carrying out license plate recognition on a plurality of frames of running images through a plurality of license plate vector diagrams, and generating an initial license plate recognition result; extracting vehicle contours of the multi-frame driving images, generating corresponding vehicle contours to be processed, and comparing the vehicle contours to be processed with a plurality of pieces of vehicle contour information to generate corresponding target contour information; and correcting the accuracy of the initial license plate recognition result based on the target contour information to generate a target license plate recognition result. In the embodiment of the invention, the vehicle contour is extracted and the vehicle contour information is generated by collecting the image information set and the license plate information set of a plurality of vehicles at the preset positions, and meanwhile, the license plate information is converted into a vector diagram. An accurate reference data set of the vehicle and the license plate can be established, and the accuracy of vehicle contour extraction and license plate identification is improved. The real-time detection and identification of the running vehicle are realized by collecting multi-frame running images of the target vehicle in a high-speed running state, and carrying out license plate positioning and vehicle contour extraction on each frame of image. The vehicle and license plate information can be quickly obtained, the subsequent contour comparison and correction are carried out, the real-time performance and the processing efficiency of the system are improved, and the initial license plate recognition result is generated by carrying out license plate recognition based on a plurality of license plate position information and license plate vector diagrams. Then, the vehicle contour extraction and contour comparison are performed on the running image, and target contour information is generated. And the accuracy correction is carried out on the initial license plate recognition result based on the target contour information, so that the fineness and accuracy of license plate recognition can be improved, and the situations of false recognition and missing recognition are reduced.

Drawings

FIG. 1 is a schematic diagram of an embodiment of a method for intelligently identifying license plates in an embodiment of the present invention;

FIG. 2 is a flow chart of vector diagram conversion in an embodiment of the invention;

FIG. 3 is a flow chart of character feature extraction in an embodiment of the invention;

FIG. 4 is a flow chart of vector diagram template matching in an embodiment of the invention;

FIG. 5 is a schematic diagram of an embodiment of an intelligent recognition device for license plates according to an embodiment of the present invention;

fig. 6 is a schematic diagram of an embodiment of an intelligent license plate recognition device according to an embodiment of the present invention.

Detailed Description

The embodiment of the invention provides an intelligent license plate recognition method, device, equipment and storage medium, which are used for improving the accuracy of intelligent license plate recognition. The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.

For easy understanding, the following describes a specific flow of an embodiment of the present invention, referring to fig. 1, and one embodiment of a method for intelligently identifying a license plate in an embodiment of the present invention includes:

s101, acquiring an image information set and a license plate information set of a plurality of vehicles at preset positions, and extracting vehicle contours of the image information set to generate a plurality of vehicle contour information;

it can be understood that the execution body of the invention can be an intelligent recognition device of a license plate, and can also be a terminal or a server, and the implementation body is not limited in the specific description. The embodiment of the invention is described by taking a server as an execution main body as an example.

For example, a monitoring camera is arranged at a high-speed intersection, images of 10 vehicles passing through the high-speed intersection at preset positions are continuously collected, and license plate numbers of each vehicle are recorded. Each vehicle image is preprocessed, including operations such as adjusting the image size, graying, enhancing contrast, and the like. The area of the vehicle is extracted from each image by a vehicle detection algorithm. For each vehicle region, the contour of the vehicle is extracted by a contour detection algorithm. The extracted vehicle profile is stored as a vehicle profile information set, and a list or an array may be used to store profile information of a plurality of vehicles.

S102, carrying out vector diagram conversion on a license plate information set to generate a plurality of license plate vector diagrams;

specifically, each character image is converted into a vector representation. One common approach is to extract the features of the character image using a feature extraction algorithm, such as a gray level co-occurrence matrix (GLCM) or Local Binary Pattern (LBP), and then represent the features as vectors, combining the vector representations of each character into a complete license plate vector map. The character vectors may be concatenated together sequentially to form a long vector, or may be combined in other ways, such as superimposing a matrix of character vectors.

For example, for a license plate information set, 5 license plate numbers are included: a12345, B6789, C24680, D13579, E98765. The following is an example showing how the license plate information set is subjected to vector diagram conversion, each character image is preprocessed, including operations of adjusting the image size, graying, binarizing, and the like, feature extraction is performed on each character image, for example, a gray level co-occurrence matrix (GLCM) algorithm is used to extract features, and the features of each character image are represented as vector forms. Assuming that the feature vector dimension of each character image is 100, the feature vectors of each character are sequentially stitched together. For the license plate number A12345, the feature vector of the character A, the feature vector of the character 1, the feature vector of the character 2 and the like are spliced together in sequence to form a license plate vector diagram with the length of 500, and for other license plate numbers, the feature vectors of the characters are spliced together in the same way to generate corresponding license plate vector diagrams.

S103, acquiring multi-frame driving images of a target vehicle in a high-speed driving state, and respectively carrying out license plate positioning on each frame of driving image to generate a plurality of license plate position information;

specifically, a monitoring camera or an acquisition device is arranged on the expressway and is used for continuously acquiring multi-frame driving images of the target vehicle in a high-speed driving state. And preprocessing each frame of driving image, such as image denoising, image enhancement and the like, so as to improve the subsequent license plate positioning effect. License plate positioning is carried out on each frame of driving image through a license plate positioning algorithm, for example, a method based on edge detection, color analysis or deep learning. It should be noted that, the license plate positioning algorithm detects a license plate region possibly existing in the image, and determines the position and the bounding box of the license plate. And extracting relevant information such as the position of the license plate, a boundary frame and the like from each frame of driving image according to the license plate position information obtained by a license plate positioning algorithm.

For example, after a plurality of frames of running images of a target vehicle in a high-speed running state are collected for a period of time, license plate position information obtained through a license plate positioning algorithm is as follows for one frame of running image:

license plate position information:

License plate 1: position (x 1, y1, x2, y 2);

license plate 2: position (x 3, y3, x4, y 4);

license plate 3: position (x 5, y5, x6, y 6);

wherein, (x 1, y1, x2, y 2) represents coordinates of upper left and lower right corners of the bounding box of the license plate 1, (x 3, y3, x4, y 4) represents coordinates of upper left and lower right corners of the bounding box of the license plate 2, and (x 5, y5, x6, y 6) represents coordinates of upper left and lower right corners of the bounding box of the license plate 3, and finally a plurality of license plate position information are generated.

S104, based on the position information of a plurality of license plates, carrying out license plate recognition on the multi-frame driving image through a plurality of license plate vector diagrams, and generating an initial license plate recognition result;

specifically, the server performs region segmentation on the multi-frame running image based on a plurality of license plate position information to generate a license plate region image corresponding to each frame of running image; based on the license plate vector diagrams corresponding to each license plate coding set, carrying out similarity recognition on license plate region images corresponding to each driving image frame, and determining a corresponding similarity recognition result; and carrying out region segmentation on license plate region images corresponding to each frame of the driving image through the similarity recognition result to generate a plurality of license plate sub-region images, carrying out character recognition on each license plate region image, and generating an initial license plate recognition result.

S105, extracting vehicle contours of the multi-frame driving images, generating corresponding vehicle contours to be processed, and comparing the vehicle contours to be processed with a plurality of pieces of vehicle contour information to generate corresponding target contour information;

it should be noted that, image preprocessing, such as graying, image enhancement, and the like, is performed on each frame of the running image in order to better extract the vehicle contour. And extracting the vehicle contour in the image by using a contour detection algorithm such as Canny edge detection, sobel operator and the like. For each detected contour, filtering and screening is performed, excluding contours that are too small or do not conform to the shape of the vehicle. And generating a set of vehicle contours to be processed according to the vehicle contours obtained in the vehicle contour extraction step. Each vehicle contour to be processed may be represented by a set of points or bounding boxes of the contour. And comparing the contours of the vehicle to be processed with the multiple pieces of vehicle contour information. And calculating the similarity between the contour of the vehicle to be processed and the known vehicle contour information by using a contour matching algorithm such as Hu moment matching, shape context matching and the like. And determining whether to match the vehicle profile to be processed with the known vehicle profile information according to a threshold value of the similarity or other rules. And generating corresponding target contour information according to the contour comparison result. And if the vehicle contour to be processed is successfully matched with the known vehicle contour information, taking the vehicle contour to be processed as a target contour. It should be noted that the target profile information may include geometric features of the vehicle profile, bounding box coordinates, and the like.

Examples: after the vehicle contour extraction is carried out on the high-speed driving image, the following vehicle contour to be processed and known vehicle contour information are obtained:

vehicle profile to be treated: vehicle 1, vehicle 2, vehicle 3;

vehicle profile information is known: vehicle a, vehicle B, vehicle C;

for each vehicle contour to be processed, a contour comparison algorithm may be used to calculate its similarity to known vehicle contour information. The following results were assumed to be obtained after the comparison: similarity of vehicle 1 to vehicle a: 0.8; similarity of vehicle 1 and vehicle B: 0.2; similarity of vehicle 1 and vehicle C: 0.6; similarity of vehicle 2 to vehicle a: 0.3 similarity of vehicle 2 to vehicle B: 0.7; similarity of vehicle 2 to vehicle C: 0.4; similarity of vehicle 3 to vehicle a: 0.6; similarity of vehicle 3 to vehicle B: 0.5; similarity of vehicle 3 to vehicle C: 0.9.

the best matching known vehicle profile information is determined as target profile information. For example, in the case where the similarity threshold is 0.7, the following target profile information can be obtained:

target profile information: vehicle 1 (matching vehicle a), vehicle 2 (matching vehicle B), vehicle 3 (matching vehicle C).

S106, correcting the accuracy of the initial license plate recognition result based on the target contour information, and generating a target license plate recognition result.

Specifically, for each target contour, a corresponding license plate region is extracted from the corresponding driving image according to the position information of the target contour. And (3) performing image preprocessing, such as graying, binarization, denoising and the like, on the extracted license plate region so as to better perform subsequent license plate recognition. And identifying the preprocessed license plate region by using a license plate identification algorithm to obtain an initial license plate identification result. And performing accuracy correction according to the target contour information and the initial license plate recognition result.

In the invention, the accuracy of the initial recognition result can be judged according to the overlapping degree of the license plate position and the target outline, the consistency of license plate characters and other factors. If the initial recognition result is matched with the target contour information to a higher degree, the initial recognition result is used as a final target license plate recognition result. Otherwise, other methods may be employed for further recognition correction, such as analysis and recognition of license plate characters using character segmentation and character recognition algorithms. For example: it is assumed that contour information of a target vehicle is included in which position information of the vehicle is included. For the vehicle, in the initial license plate recognition stage, the following initial license plate recognition results are obtained: "ABC123". And extracting a corresponding license plate region from the driving image according to the target contour information, and performing preprocessing operation. And then, recognizing the preprocessed license plate region by using a license plate recognition algorithm to obtain an initial license plate recognition result. And then, carrying out accuracy correction according to the target contour information and the initial license plate recognition result. The license plate position and character consistency of the initial recognition result and the target outline information are found to be higher, and no obvious deviation or error exists. Thus, the initial recognition result "ABC123" may be taken as the final target license plate recognition result.

In the embodiment of the invention, an image information set and a license plate information set of a plurality of vehicles at preset positions are collected, and vehicle contour extraction is carried out on the image information set to generate a plurality of vehicle contour information; performing vector diagram conversion on the license plate information set to generate a plurality of license plate vector diagrams; collecting multi-frame driving images of a target vehicle in a high-speed driving state, and respectively carrying out license plate positioning on each frame of driving image to generate a plurality of license plate position information; based on the license plate position information, carrying out license plate recognition on a plurality of frames of running images through a plurality of license plate vector diagrams, and generating an initial license plate recognition result; extracting vehicle contours of the multi-frame driving images, generating corresponding vehicle contours to be processed, and comparing the vehicle contours to be processed with a plurality of pieces of vehicle contour information to generate corresponding target contour information; and correcting the accuracy of the initial license plate recognition result based on the target contour information to generate a target license plate recognition result. In the embodiment of the invention, the vehicle contour is extracted and the vehicle contour information is generated by collecting the image information set and the license plate information set of a plurality of vehicles at the preset positions, and meanwhile, the license plate information is converted into a vector diagram. An accurate reference data set of the vehicle and the license plate can be established, and the accuracy of vehicle contour extraction and license plate identification is improved. The real-time detection and identification of the running vehicle are realized by collecting multi-frame running images of the target vehicle in a high-speed running state, and carrying out license plate positioning and vehicle contour extraction on each frame of image. The vehicle and license plate information can be quickly obtained, the subsequent contour comparison and correction are carried out, the real-time performance and the processing efficiency of the system are improved, and the initial license plate recognition result is generated by carrying out license plate recognition based on a plurality of license plate position information and license plate vector diagrams. Then, the vehicle contour extraction and contour comparison are performed on the running image, and target contour information is generated. And the accuracy correction is carried out on the initial license plate recognition result based on the target contour information, so that the fineness and accuracy of license plate recognition can be improved, and the situations of false recognition and missing recognition are reduced.

In a specific embodiment, the process of executing step S105 may specifically include the following steps:

(1) Extracting vehicle contours of the multi-frame driving images through an edge detection algorithm, and generating corresponding vehicle contours to be processed;

(2) Performing contour comparison on the vehicle contour to be processed and the plurality of vehicle contour information through a shape matching algorithm, and determining a corresponding comparison result;

(3) Threshold analysis is carried out on the comparison result, and a corresponding threshold analysis result is determined;

(4) And carrying out contour screening on the plurality of vehicle contours through a threshold analysis result to generate target contour information.

Specifically, for each frame of running image, an edge detection algorithm (e.g., canny algorithm) is used to extract the vehicle profile. The edge detection algorithm will identify edge information in the image, including the boundaries of the vehicle. The vehicle contour to be processed is matched in shape with a plurality of pieces of known vehicle contour information. The shape matching algorithm may calculate the similarity or distance between the contours to be processed and the known contours for comparison and matching. And (5) carrying out threshold analysis according to the similarity or distance of the shape matching, and determining a comparison result. And judging the matching degree of the contour to be processed and the known contour according to the threshold setting standard. And screening the plurality of vehicle contours to be processed based on the threshold analysis result, and determining target contour information. A profile that matches a known vehicle profile to a degree higher than a threshold value may be selected as the target profile. For example, if the degree of matching of a certain contour to be processed with the contour of the known vehicle a is higher than the threshold value of 0.8, it is regarded as the contour of the target vehicle a.

When the contour of the vehicle to be processed and the plurality of vehicle contour information are compared by the shape matching algorithm and the corresponding comparison result is determined, firstly, an edge detection or other contour extraction algorithm is used for a frame of driving image to extract the contour of the vehicle to be processed. And for the vehicle contour to be processed and a plurality of known vehicle contour information sets, a shape matching algorithm is applied to perform contour comparison. For each known vehicle profile information, its corresponding Hu moment feature vector is calculated. Then, a Hu moment feature vector of the vehicle contour to be processed is calculated. And determining the similarity or matching degree by comparing the Hu moment feature vector of the contour of the vehicle to be processed with the feature vector of each contour in the known vehicle contour information set. A distance metric method (such as euclidean distance or cosine similarity) may be used to calculate the similarity of the vehicle profile to be processed to the known vehicle profile. From the value of the similarity, the best matching known vehicle profile is determined.

For example: assume that there is one vehicle profile to be processed and three known vehicle profile information.

Hu moment eigenvectors of the vehicle contour to be processed: [0.25,0.15,0.1];

the Hu moment eigenvector of the vehicle contour 1 is known: [0.22,0.13,0.12];

The Hu moment eigenvector of the vehicle contour 2 is known: [0.20,0.14,0.08];

the Hu moment eigenvectors of the vehicle contour 3 are known: [0.28,0.17,0.09];

by calculating the distance or similarity between the vehicle profile to be processed and the known vehicle profile, the following results can be obtained:

similarity of the vehicle profile to be processed to the known vehicle profile 1: 0.02;

similarity of the vehicle profile to be processed to the known vehicle profile 2: 0.05;

similarity of the vehicle profile to be processed to the known vehicle profile 3: 0.03;

from the value of the similarity, it can be determined that the best-matching known vehicle profile is the known vehicle profile 2, since it has the highest similarity to the vehicle profile to be processed.

In a specific embodiment, as shown in fig. 2, the process of executing step S102 may specifically include the following steps:

s201, character segmentation is carried out on license plate information sets to generate a plurality of license plate character sets, and each license plate character set is subjected to coding processing to generate a plurality of license plate coding sets;

s202, vector diagram conversion is carried out on a plurality of license plate code sets, and a license plate vector diagram corresponding to each license plate code set is obtained.

Note that license plate character segmentation is a process of cutting characters in the entire license plate image into independent character parts. Character segmentation may be achieved based on spacing and shape characteristics between characters using image processing techniques such as thresholding, edge detection, connected region analysis, and the like. After segmentation, each character portion is saved as a separate image. And for each license plate character set, converting the character image into a numerical characteristic representation by adopting a coding processing method. The encoding process may convert the character into an encoding in the form of a number or string, for example, encoding the character "a" as 1, "B" as 2, etc. And carrying out vector diagram conversion on each license plate code set, and converting each code into a corresponding license plate vector diagram. License plate vector diagrams are image representations obtained by mapping license plate codes into a high-dimensional vector space. License plate codes are converted to vector representations using feature extraction methods, such as Convolutional Neural Networks (CNNs) or other machine learning algorithms. For example: assuming that a license plate image, such as "Beijing A12345", is required to perform character segmentation, encoding processing and vector diagram conversion. The license plate image is divided into six independent character images of A, 1, 2, 3, 4 and 5 according to the interval between the characters and the shape characteristics by using an image processing method. A character recognition algorithm is applied to each character image to recognize the characters as corresponding identifications. For example, the character "a" is identified as the letter "a", the character "1" is identified as the number "1", and so on.

And carrying out vector diagram conversion on each character code, and converting the character codes into corresponding license plate vector diagrams. For example, code 1 of character "a" is converted into a high-dimensional vector representation, represented in the form of a vector diagram. Likewise, code 1 of the character "1" is converted into another vector diagram.

In a specific embodiment, as shown in fig. 3, the process of executing step S201 may specifically include the following steps:

s301, carrying out binarization processing on a license plate information set to generate a corresponding binarized license plate data set;

s302, performing character segmentation on the binarized license plate data set through a character segmentation algorithm to generate a plurality of license plate character sets;

s303, respectively extracting character features of each license plate character set to generate a character feature set corresponding to each license plate character set;

s304, respectively carrying out feature clustering on character feature sets corresponding to each license plate character set to generate a plurality of clustered license plate features;

s305, carrying out coding processing on each license plate character set based on a plurality of clustered license plate features to generate a plurality of license plate coding sets.

The image in the license plate information set is subjected to binarization processing, and the image is converted into a binary image containing only two colors of black and white. And separating the license plate region from the background in the image according to the pixel value through threshold segmentation. And (3) using a character segmentation algorithm to segment out characters in each license plate image in the binarized license plate data set. According to the characteristics of the interval between characters, pixel connectivity and the like, the adjacent black areas are divided into independent characters, character characteristic extraction is carried out on character images in each license plate character set, and numerical values or vectors representing the characteristics of character shapes, textures and the like are extracted. Character features are extracted using shape descriptors (e.g., hu moments, zernike moments) or texture descriptors (e.g., gray co-occurrence matrix, gabor filter response), etc. And carrying out cluster analysis on character feature sets corresponding to each license plate character set, and classifying similar features into the same category. Clustering algorithms (e.g., K-means clustering, hierarchical clustering) are used to cluster character features into multiple categories, each category representing a license plate feature. And distributing corresponding codes for each license plate character set according to the plurality of clustered license plate features. The license plate character set can be encoded by using modes such as independent thermal encoding, digital encoding and the like, and characters are converted into numerical representation.

For example, assume that there is a license plate information set, which includes the following three license plate images and corresponding character sequences:

license plate 1 character sequence: AB1234;

license plate 2 character sequence: CD5678;

license plate 3 character sequence: EF9012;

firstly, binarization processing is carried out on a license plate information set. The pixel gradation value is compared with a set threshold value by a thresholding method, and pixels larger than the threshold value are set to white and pixels smaller than the threshold value are set to black. Taking the threshold value as 100 as an example, the binarized license plate image is as follows:

binarized character sequence 1:11110000111100001111000011110000111100001111;

binarized character sequence 2:11110000111100001111000011110000111100001111;

binarized character sequence 3:11110000111100001111000011110000111100001111;

next, character segmentation is performed on the binarized license plate dataset, and each character region is segmented into independent character images. Taking a character width of 10 pixels as an example, the result after character segmentation is as follows:

license plate character set 1: [ Character A, character B, character 1, character 2, character 3, character 4];

license plate character set 2: [ Character C, character D, character 5, character 6, character 7, character 8];

License plate character set 3: [ Character E, character F, character 9, character 0, character 1, character 2];

and then, extracting character features from each license plate character set, and extracting key feature information of each character. Taking the HOG feature extraction algorithm as an example, the extracted character features are as follows:

character feature set of license plate character set 1: [ Feature 1A, feature 1B, feature 11, feature 12, feature 13, feature 14];

character feature set of license plate character set 2: [ Feature 2C, feature 2D, feature 25, feature 26, feature 27, feature 28];

character feature set of license plate character set 3: [ Feature 3E, feature 3F, feature 39, feature 30, feature 31, feature 32];

and then, carrying out feature clustering on character feature sets corresponding to each license plate character set, and aggregating the characters with similar features into the same category. Taking a K-means clustering algorithm as an example, the result after feature clustering is as follows:

clustering license plate features 1: [ Cluster 1A, cluster 1B, cluster 1C ];

clustering license plate features 2: [ Cluster 2A, cluster 2B, cluster 2C ];

clustering license plate features 3: [ Cluster 3A, cluster 3B, cluster 3C ];

And finally, based on a plurality of clustered license plate features, carrying out coding processing on each license plate character set to generate a plurality of license plate coding sets. Taking the encoding method as a simple character connection as an example, the result after encoding is as follows:

license plate code set 1: ABC;

license plate code set 2: DEF;

license plate code set 3:123.

in a specific embodiment, as shown in fig. 4, the process of executing step S202 may specifically include the following steps:

s401, performing traversal processing on each license plate code set to determine a plurality of code identifiers;

s402, carrying out vector diagram template matching through a plurality of coding identifiers based on a preset vector diagram template database, and determining a plurality of vector diagram templates;

s403, carrying out vector diagram conversion on each license plate code set based on a plurality of vector diagram templates to obtain a license plate vector diagram corresponding to each license plate code set.

It is noted that, each license plate code set is traversed, and characteristic information of each code character is extracted.

And determining the code identification corresponding to each code character according to the characteristic information. The coded identifier may be a serial number of a character, an ASCII code of a character, or the like.

For example: assume that there is one license plate code set: [ ABC, DEF, 123]. By traversing the process, the code identification can be determined as follows:

Coding identifier 1: a=1, b=2, c=3;

coding identifier 2: d=4, e=5, f=6;

coding an identification 3: 1=7, 2=8, 3=9;

based on a preset vector diagram template database, a corresponding vector diagram template is found according to a plurality of coding identifiers. The vector diagram template database stores a vector diagram template of each character, which may be a template image generated in advance or a feature vector obtained by a feature extraction method.

For example: the assumed vector diagram template database contains the following vector diagram templates:

template 1 corresponds to code identifier 1: [ Vector 1A, vector 1B, vector 1C ];

template 2 corresponds to code identifier 2: [ Vector 2D, vector 2E, vector 2F ];

template 3 corresponds to code identifier 3: [ Vector 31, vector 32, vector 33];

and carrying out vector diagram conversion on each license plate coding set, and converting each coding character into a corresponding vector diagram. And according to the code identification and the vector diagram template which are determined before, each character code is converted into a corresponding vector diagram.

For example: taking license plate coding set [ ABC, DEF, 123] as an example, according to the previous coding identification and vector diagram template, vector diagram conversion is carried out as follows:

vector diagram conversion result of license plate code set 1: [ Vector 1A, vector 1B, vector 1C ];

Vector diagram conversion result of license plate code set 2: [ Vector 2D, vector 2E, vector 2F ];

vector diagram conversion result of license plate code set 3: [ Vector 31, vector 32, vector 33].

In a specific embodiment, the process of executing step S104 may specifically include the following steps:

(1) Performing region segmentation on a plurality of frames of running images based on a plurality of license plate position information to generate license plate region images corresponding to each frame of running images;

(2) Based on license plate vector diagrams corresponding to each license plate coding set, carrying out similarity recognition on license plate region images corresponding to each frame of driving image, and determining a corresponding similarity recognition result;

(3) Carrying out region segmentation on license plate region images corresponding to each frame of driving image through a similarity recognition result to generate a plurality of license plate region images;

(4) And carrying out character recognition on each license plate sub-region image to generate an initial license plate recognition result.

Specifically, based on the plurality of license plate position information, the multi-frame running image is subjected to region segmentation, and the license plate region in each frame running image is extracted. This is accomplished by a bounding box or outline of license plate location information.

For example: the position information of one vehicle in the multi-frame driving image is assumed as follows:

Image 1: license plate position information is (x 1, y1, x2, y 2);

image 2: license plate position information is (x 3, y3, x4, y 4);

image 3: license plate position information is (x 5, y5, x6, y 6);

through region segmentation, license plate region images corresponding to each frame of running image can be generated:

license plate area image of image 1: ROI1;

license plate area image of image 2: ROI2;

license plate area image of image 3: ROI3;

and carrying out similarity recognition on license plate region images corresponding to each frame of driving image based on license plate vector diagrams corresponding to each license plate coding set.

For example: the following license plate code set and corresponding license plate vector diagram are assumed:

license plate code set 1: ABC, the corresponding license plate Vector diagram is Vector1;

license plate code set 2: DEF, corresponding license plate Vector diagram is Vector2;

for each license plate region image, the similarity with the license plate vector diagram can be calculated:

similarity 1 = calculated similarity (ROI 1, vector 1);

similarity 2 = calculate similarity (ROI 1, vector 2);

similarity 3 = calculated similarity (ROI 2, vector 1);

similarity 4 = calculate similarity (ROI 2, vector 2);

similarity 5 = calculated similarity (ROI 3, vector 1);

similarity 6 = calculated similarity (ROI 3, vector 2);

and based on the similarity recognition result, carrying out region segmentation on license plate region images corresponding to each frame of driving image, and generating a plurality of license plate region images.

And judging whether the matching is successful or not by setting a threshold value or selecting a matching result with the highest similarity.

For example: assuming that a similarity threshold is set, if the similarity is greater than the threshold, the matching is considered successful.

When the similarity 1> is threshold, the ROI1 is considered to be matched with the license plate coding set 1, and the license plate sub-region image 1 is generated.

When the similarity 2> is threshold, the ROI1 is considered to be matched with the license plate code set 2, and the license plate region image 2 is generated.

When the similarity 3> is threshold, the ROI2 is considered to be matched with the license plate coding set 1, and a license plate sub-region image 3 is generated.

When the similarity 4> is threshold, the ROI2 is considered to be matched with the license plate code set 2, and a license plate region image 4 is generated.

When the similarity 5> is threshold, the ROI3 is considered to be matched with the license plate code set 1, and a license plate region image 5 is generated.

When the similarity 6> is threshold, the ROI3 is considered to be matched with the license plate code set 2, and a license plate region image 6 is generated.

And carrying out character recognition on each license plate sub-region image, extracting characters in each sub-region, and generating an initial license plate recognition result.

For example: for each generated license plate area image, the initial license plate recognition result obtained after character recognition is as follows:

License plate sub-region image 1: the character recognition results are A, B and C;

license plate area image 2: the character recognition results are D, E and F;

license plate sub-region image 3: the character recognition results are A, B and C;

license plate sub-region image 4: the character recognition results are D, E and F;

license plate area image 5: the character recognition results are A, B and C;

license plate area image 6: the character recognition results are D, E and F;

in the embodiment, the region segmentation of the multi-frame driving image based on the license plate position information is realized, the matching result is obtained through similarity recognition, a plurality of license plate sub-region images are finally generated, and character recognition is carried out on the license plate sub-region images to obtain an initial license plate recognition result.

In a specific embodiment, the step of performing similarity recognition on the license plate region image corresponding to each frame of the driving image based on the license plate vector diagram corresponding to each license plate coding set and determining the corresponding similarity recognition result may specifically include the following steps:

(1) Performing matrix conversion on license plate vector diagrams corresponding to each license plate coding set to generate a plurality of matrixes to be processed;

(2) Performing matrix analysis on license plate region images corresponding to each frame of driving image to generate a plurality of matrixes to be compared;

(3) And carrying out similarity recognition on the multiple matrixes to be processed and the multiple matrixes to be compared, and determining a similarity recognition result.

Specifically, the license plate vector diagram corresponding to each license plate coding set is subjected to matrix conversion, and is converted into a matrix to be processed. This can be achieved by representing the image pixels in a matrix form, and then preprocessing and normalizing the matrix.

vector1 and Vector2 can be converted into a matrix to be processed:

matrix to be processed 1: matrix1;

matrix to be processed 2: matrix2;

and (3) carrying out matrix analysis on license plate region images corresponding to each frame of driving image, and converting the license plate region images into a matrix to be compared. Also, this is achieved by representing the image pixels in a matrix form, and performing preprocessing and normalization.

For example: assume that there is a license plate region image of each of the following running images:

license plate area image of image 1: ROI1;

license plate area image of image 2: ROI2;

ROI1 and ROI2 can be converted into alignment matrix to be aligned:

The matrix to be compared 1: matrix3;

the matrix to be compared 2: matrix4;

and carrying out similarity recognition on the multiple matrices to be processed and the multiple matrices to be compared to determine the similarity between the multiple matrices to be processed and the multiple matrices to be compared. By calculating similarity indexes (such as euclidean distance, correlation, etc.) between the matrices. For example: the following matrices to be processed and the matrices to be aligned are assumed:

matrix to be processed 1: matrix1;

matrix to be processed 2: matrix2;

the matrix to be compared 1: matrix3;

the matrix to be compared 2: matrix4;

the similarity between the matrix to be processed and the matrix to be compared can be calculated:

similarity 1=calculation (Matrix 1, matrix 3);

similarity 2=calculation (Matrix 1, matrix 4);

similarity 3=calculation (Matrix 2, matrix 3);

similarity 4=calculation (Matrix 2, matrix 4);

and finally, determining a similarity recognition result according to the similarity.

In this embodiment, a matrix conversion may be performed on a license plate vector diagram corresponding to each license plate code set to generate a matrix to be processed, and a matrix analysis may be performed on a license plate area image corresponding to each frame of running image to generate a matrix to be compared. And then, comparing the similarity between the matrix to be processed and the matrix to be compared through similarity recognition, and determining a similarity recognition result.

The method for intelligently identifying the license plate in the embodiment of the present invention is described above, and the device for intelligently identifying the license plate in the embodiment of the present invention is described below, referring to fig. 5, and one embodiment of the device for intelligently identifying the license plate in the embodiment of the present invention includes:

the acquisition module 501 is configured to acquire an image information set and a license plate information set of a plurality of vehicles at a preset position, extract a vehicle contour from the image information set, and generate a plurality of vehicle contour information;

the conversion module 502 is configured to perform vector diagram conversion on the license plate information set, and generate a plurality of license plate vector diagrams;

the positioning module 503 is configured to collect multiple frames of running images of a target vehicle in a high-speed running state, and respectively perform license plate positioning on each frame of running image to generate multiple license plate position information;

the recognition module 504 is configured to perform license plate recognition on a plurality of frames of the running image through a plurality of license plate vector diagrams based on a plurality of license plate position information, and generate an initial license plate recognition result;

the extracting module 505 is configured to extract a vehicle contour from a multi-frame driving image, generate a corresponding vehicle contour to be processed, and compare the vehicle contour to be processed with a plurality of pieces of vehicle contour information to generate corresponding target contour information;

And the correction module 506 is configured to correct the accuracy of the initial license plate recognition result based on the target profile information, and generate a target license plate recognition result.

Collecting an image information set and a license plate information set of a plurality of vehicles at preset positions through the cooperative cooperation of the components, extracting vehicle contours from the image information set, and generating a plurality of vehicle contour information; performing vector diagram conversion on the license plate information set to generate a plurality of license plate vector diagrams; collecting multi-frame driving images of a target vehicle in a high-speed driving state, and respectively carrying out license plate positioning on each frame of driving image to generate a plurality of license plate position information; based on the license plate position information, carrying out license plate recognition on a plurality of frames of running images through a plurality of license plate vector diagrams, and generating an initial license plate recognition result; extracting vehicle contours of the multi-frame driving images, generating corresponding vehicle contours to be processed, and comparing the vehicle contours to be processed with a plurality of pieces of vehicle contour information to generate corresponding target contour information; and correcting the accuracy of the initial license plate recognition result based on the target contour information to generate a target license plate recognition result. In the embodiment of the invention, the vehicle contour is extracted and the vehicle contour information is generated by collecting the image information set and the license plate information set of a plurality of vehicles at the preset positions, and meanwhile, the license plate information is converted into a vector diagram. An accurate reference data set of the vehicle and the license plate can be established, and the accuracy of vehicle contour extraction and license plate identification is improved. The real-time detection and identification of the running vehicle are realized by collecting multi-frame running images of the target vehicle in a high-speed running state, and carrying out license plate positioning and vehicle contour extraction on each frame of image. The vehicle and license plate information can be quickly obtained, the subsequent contour comparison and correction are carried out, the real-time performance and the processing efficiency of the system are improved, and the initial license plate recognition result is generated by carrying out license plate recognition based on a plurality of license plate position information and license plate vector diagrams. Then, the vehicle contour extraction and contour comparison are performed on the running image, and target contour information is generated. And the accuracy correction is carried out on the initial license plate recognition result based on the target contour information, so that the fineness and accuracy of license plate recognition can be improved, and the situations of false recognition and missing recognition are reduced.

The intelligent recognition device of the license plate in the embodiment of the invention is described in detail from the angle of the modularized functional entity in fig. 5, and the intelligent recognition device of the license plate in the embodiment of the invention is described in detail from the angle of hardware processing.

Fig. 6 is a schematic structural diagram of a smart identification device for a license plate according to an embodiment of the present invention, where the smart identification device 600 for a license plate may have a relatively large difference due to different configurations or performances, and may include one or more processors (CPU) 610 (e.g., one or more processors) and a memory 620, and one or more storage media 630 (e.g., one or more mass storage devices) storing applications 633 or data 632. Wherein the memory 620 and the storage medium 630 may be transitory or persistent storage. The program stored on the storage medium 630 may include one or more modules (not shown), each of which may include a series of instruction operations in the smart identification device 600 for license plates. Still further, the processor 610 may be configured to communicate with the storage medium 630 to execute a series of instruction operations in the storage medium 630 on the smart identification device 600 of the license plate.

The smart identification device 600 for license plates may also include one or more power sources 640, one or more wired or wireless network interfaces 650, one or more input/output interfaces 660, and/or one or more operating systems 631, such as WindowsServe, macOSX, unix, linux, freeBSD, etc. It will be appreciated by those skilled in the art that the smart identification device structure of the license plate shown in fig. 6 does not constitute a limitation of the smart identification device of the license plate, and may include more or fewer components than shown, or may combine certain components, or may have a different arrangement of components.

The invention also provides intelligent recognition equipment of the license plate, which comprises a memory and a processor, wherein the memory stores computer readable instructions, and when the computer readable instructions are executed by the processor, the processor executes the steps of the intelligent recognition method of the license plate in the embodiments.

The invention also provides a computer readable storage medium, which can be a nonvolatile computer readable storage medium, and can also be a volatile computer readable storage medium, wherein the computer readable storage medium stores instructions, and when the instructions run on a computer, the instructions cause the computer to execute the steps of the intelligent license plate recognition method.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

The integrated units, if implemented in the form of software functional units and sold or passed as separate products, may be stored in a computer readable storage medium. Based on the understanding that the technical solution of the present invention may be embodied in essence or in a part contributing to the prior art or in whole or in part in the form of a software product stored in a storage medium, comprising instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, or other various media capable of storing program codes.

The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. The intelligent license plate recognition method is characterized by comprising the following steps of:

2. The intelligent license plate recognition method according to claim 1, wherein the performing vector diagram conversion on the license plate information set to generate a plurality of license plate vector diagrams includes:

3. The intelligent license plate recognition method according to claim 2, wherein the character segmentation is performed on the license plate information set to generate a plurality of license plate character sets, and each license plate character set is subjected to coding processing to generate a plurality of license plate code sets, and the method comprises the steps of:

4. The intelligent license plate recognition method according to claim 2, wherein the performing vector diagram conversion on the plurality of license plate code sets to obtain a corresponding license plate vector diagram of each license plate code set comprises:

5. The intelligent license plate recognition method according to claim 4, wherein the generating an initial license plate recognition result based on the license plate position information by performing license plate recognition on a plurality of frames of the running image through a plurality of license plate vector diagrams comprises:

6. The intelligent license plate recognition method according to claim 5, wherein the performing similarity recognition on the license plate region image corresponding to each frame of the driving image based on the license plate vector diagram corresponding to each license plate code set, and determining the corresponding similarity recognition result includes:

7. The intelligent license plate recognition method according to claim 1, wherein the steps of extracting vehicle contours from the multi-frame driving image to generate corresponding vehicle contours to be processed, comparing the vehicle contours to be processed with a plurality of pieces of vehicle contour information to generate corresponding target contour information include:

8. The utility model provides an intelligent identification device of license plate, its characterized in that, intelligent identification device of license plate includes:

9. The utility model provides an intelligent identification equipment of license plate, its characterized in that, intelligent identification equipment of license plate includes: a memory and at least one processor, the memory having instructions stored therein;

the at least one processor invokes the instructions in the memory to cause the intelligent recognition device of the license plate to perform the intelligent recognition method of the license plate of any one of claims 1-7.

10. A computer readable storage medium having instructions stored thereon, wherein the instructions when executed by a processor implement the intelligent identification method of a license plate of any of claims 1-7.