CN114299503A

CN114299503A - License plate recognition method and device, electronic equipment and storage medium

Info

Publication number: CN114299503A
Application number: CN202111538897.XA
Authority: CN
Inventors: 李海龙
Original assignee: Shenzhen Intellifusion Technologies Co Ltd
Current assignee: Shenzhen Intellifusion Technologies Co Ltd
Priority date: 2021-12-15
Filing date: 2021-12-15
Publication date: 2022-04-08

Abstract

The invention discloses a license plate recognition method and device, electronic equipment and a storage medium, and is suitable for the technical field of image recognition. The method comprises the following steps: acquiring a license plate image to be recognized corresponding to a target vehicle; detecting a license plate region in a license plate image to be recognized, and determining position information and angle information of the license plate region; the angle information is used for representing the deflection angle of the license plate area relative to the reference position; adjusting the position of the license plate region according to the position information and the angle information, and acquiring a target license plate region after the position is adjusted; and identifying the target license plate area, and determining the license plate information of the target vehicle. By adopting the method, the accuracy of identifying the license plate information can be improved.

Description

License plate recognition method and device, electronic equipment and storage medium

Technical Field

The invention relates to the technical field of image recognition, in particular to a license plate recognition method and device, electronic equipment and a storage medium.

Background

The problem that how to manage various large vehicles is urgently solved is that the construction of cities can not leave various large vehicles.

In the prior art, the management of the vehicle is usually realized by identifying the license plate of the vehicle. However, because the number plate types are many, and the scenes are complex and changeable, including factors such as reflection, weather, and shielding, the number plate recognition system in the prior art sometimes has a situation of recognition error when the vehicle enters or exits a specific scene. At present, the accuracy of license plate recognition equipment based on video image license plate recognition is not high, and the recognition rate of a license plate recognition system still has more space for improvement.

Disclosure of Invention

In view of this, embodiments of the present invention provide a license plate recognition method, a license plate recognition device, an electronic device, and a storage medium, and aim to solve the problem of low recognition rate of a license plate recognition system.

According to a first aspect, an embodiment of the present invention provides a license plate recognition method, including:

acquiring a license plate image to be recognized corresponding to a target vehicle;

detecting a license plate region in a license plate image to be recognized, and determining position information and angle information of the license plate region; the angle information is used for representing the deflection angle of the license plate area relative to the reference position;

adjusting the position of the license plate region according to the position information and the angle information, and acquiring a target license plate region after the position is adjusted;

and identifying the target license plate area, and determining the license plate information of the target vehicle.

The license plate recognition method provided by the embodiment of the application obtains the license plate image to be recognized corresponding to the target vehicle, detects the license plate area in the license plate image to be recognized, and determines the position information and the angle information of the license plate area. According to the position information and the angle information of the license plate area, the position of the license plate of the target vehicle in the license plate image to be recognized can be accurately determined, so that the license plate of the target vehicle in the license plate image to be recognized can be recognized conveniently. In addition, the position of the license plate region is adjusted according to the position information and the angle information, and the target license plate region after the position is adjusted is obtained. The adjusted target license plate area is easier to identify, the condition that the license plate area is inclined upwards and downwards or the license plate area is inclined inwards and outwards is improved, and therefore the accuracy of identifying the license plate of the target vehicle is improved when the target license plate area is identified. The problem that the license plate of the target vehicle is identified inaccurately due to the position information of the license plate of the target vehicle in the license plate image to be identified is avoided. Therefore, the license plate identification method can improve the accuracy of identifying the license plate information.

With reference to the first aspect, in a first implementation manner of the first aspect, acquiring at least two license plate images to be recognized corresponding to a target vehicle includes:

when a target vehicle enters a target scene, acquiring a candidate image of the target vehicle;

identifying the position of a target vehicle in the candidate image, and judging whether the target vehicle in the candidate image enters a target area in a target scene;

and when the target vehicle in the candidate image enters the target area, determining the candidate image as the license plate image to be recognized so as to obtain at least two license plate images to be recognized.

According to the license plate recognition method provided by the embodiment of the application, when a target vehicle enters a target scene, a candidate image of the target vehicle is obtained. And judging whether the target vehicle in the candidate image enters a target area in the target scene or not by identifying the position of the target vehicle in the candidate image. The position of the target vehicle in the target scene is related to the position of the license plate of the target vehicle in the candidate image. Therefore, it is necessary to determine whether the target vehicle in the candidate image enters the target area in the target scene according to the position of the target vehicle in the candidate image. And when the target vehicle in the candidate image enters the target area, determining the candidate image as the license plate image to be recognized. Because, only when the target vehicle enters the target area, the position of the license plate of the target vehicle in the license plate image to be recognized is most easily recognized; when the target vehicle does not enter the target area, the position of the license plate of the target vehicle in the license plate image to be recognized is small, and the license plate area of the target vehicle is not convenient to recognize; when the target vehicle leaves the target area, the license plate of the target vehicle may not be completely displayed in the license plate image to be recognized, and the license plate area of the target vehicle is not convenient to recognize. Therefore, when a target vehicle in the candidate images enters the target area, the candidate images are determined to be the license plate images to be recognized so as to obtain at least two license plate images to be recognized, and the accuracy of the license plate areas recognized according to the obtained at least two license plate images to be recognized can be ensured. In addition, the license plate region is more accurately identified from at least two license plate images to be identified than from one license plate image to be identified.

With reference to the first embodiment of the first aspect, in a second embodiment of the first aspect, identifying a target license plate region and determining license plate information of a target license plate includes:

identifying target license plate areas corresponding to at least two license plate images to be identified, and determining corresponding predicted license plate information and probability thereof;

and comparing the probabilities and determining the predicted license plate information of the maximum probability as the license plate information of the target license plate.

According to the license plate recognition method provided by the embodiment of the application, the target license plate areas corresponding to at least two license plate images to be recognized are recognized, and corresponding predicted license plate information and probability thereof are determined; then, the probability is compared, and the predicted license plate information with the maximum probability is determined as the license plate information of the target license plate. Instead of only one license plate image to be recognized, the license plate image is determined as the license plate information of the target license plate. Only one license plate image to be recognized is recognized, and the license plate information determined as the license plate information of the target license plate may be inaccurate in recognition. Therefore, the license plate identification method can improve the accuracy of the license plate information of the determined target license plate.

With reference to the first aspect, in a third implementation manner of the first aspect, the position information and the angle information of the license plate region are determined by a license plate region detection model, and a training process of the license plate region detection model includes the following steps:

acquiring a sample license plate image and a label thereof, wherein the label comprises target position information and target angle information of a license plate region in the sample license plate image;

inputting a sample license plate image into a preset license plate region detection model, and determining predicted position information and predicted angle information of the sample license plate region;

calculating a location loss based on the predicted location information;

calculating an angle loss based on the predicted angle information;

and updating parameters of the license plate region detection model based on the position loss and the angle loss, and determining the license plate region detection model.

According to the license plate identification method provided by the embodiment of the application, the sample license plate image and the label thereof are obtained, the sample license plate image is input into the preset license plate region detection model, and the prediction position information and the prediction angle information of the sample license plate region are determined. And then, calculating position loss based on the predicted position information, calculating angle loss based on the predicted angle information, updating parameters of the license plate region detection model based on the position loss and the angle loss, and determining the license plate region detection model. The preset license plate region detection model is trained according to the sample license plate image and the label thereof, so that the accuracy of the trained license plate region detection model can be ensured, the accuracy of the position information and the angle information of the license plate region acquired based on the license plate region detection model is further ensured, and the accuracy of recognizing the license plate of the target vehicle is ensured.

With reference to the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the determining the angle loss based on the predicted angle information includes:

calculating a cosine value by using the predicted angle information;

the angle loss is calculated based on the cosine value.

According to the license plate identification method provided by the embodiment of the application, the cosine value is calculated by utilizing the prediction angle information;

the angle loss is calculated based on the cosine value, and the accuracy of the calculated angle loss can be ensured.

With reference to the first aspect, in a fifth implementation manner of the first aspect, the performing position adjustment on the license plate region according to the position information and the angle information, and acquiring the target license plate region after the position adjustment includes:

and performing affine transformation on the license plate region by taking the reference position as a reference according to the position information and the angle information, correcting the angle of the license plate region, and obtaining the angle-adjusted target license plate region.

According to the license plate identification method provided by the embodiment of the application, affine transformation is carried out by taking the reference position as the reference according to the position information and the angle information of the license plate region, the angle of the license plate region is corrected, the target license plate region after angle adjustment is obtained, the corrected target license plate region can be ensured to coincide with the reference position, the problem that the target license plate region inclines inwards or outwards is solved, and the accuracy of identifying the target license plate region is ensured.

With reference to the first aspect, in a sixth implementation manner of the first aspect, identifying a target license plate region and determining license plate information of a target vehicle includes:

inputting the target license plate area into a target license plate recognition model; the target license plate recognition model is obtained by compressing the license plate recognition model;

the target license plate recognition model recognizes a target license plate region and outputs a recognition result corresponding to the target license plate region; the recognition result comprises at least one of characters, numbers and letters;

and analyzing the recognition result based on license plate rule analysis, and outputting license plate information of the target license plate.

According to the license plate recognition method provided by the embodiment of the application, the target license plate area is input into the target license plate recognition model, and the recognition result corresponding to the target license plate area is output. And analyzing the recognition result based on license plate rule analysis, and outputting the license plate information of the target license plate, so that the accuracy of the output license plate information of the target license plate can be ensured. In addition, the target license plate recognition model is obtained by compressing the license plate recognition model, so that the structure of the target license plate recognition model is small. If the license plate recognition model is huge in structure, when a target license plate area is recognized, the recognition result is inaccurate due to redundancy in the calculation process, and a large amount of running memory of the electronic equipment is occupied, so that the electronic equipment runs slowly, and the license plate recognition efficiency is influenced. The target license plate recognition model is small in structure, the calculation process is simplified, and accuracy of recognition of the target license plate region can be guaranteed. In addition, the target license plate recognition model can ensure good running condition of the electronic equipment under the condition of improving the accuracy of the target license plate recognition model, and the license plate recognition efficiency is improved.

With reference to the sixth implementation manner of the first aspect, in the seventh implementation manner of the first aspect, the process of compressing the license plate recognition model to obtain the target license plate recognition model includes:

pruning the license plate recognition model;

performing model training on the vehicle license plate recognition model after pruning;

carrying out model accuracy detection on the trained license plate recognition model;

and when the accuracy of the trained license plate recognition model is greater than or equal to a preset accuracy threshold, obtaining a target license plate recognition model.

According to the license plate recognition method provided by the embodiment of the application, the structure of the license plate recognition model can be reduced by pruning the license plate recognition model, so that the calculation process of the license plate recognition model is simplified. After the vehicle license plate recognition model is pruned, the accuracy of the vehicle license plate recognition model may be reduced. Therefore, model training is carried out on the vehicle license plate recognition model after pruning, and accuracy of the vehicle license plate recognition model after pruning can be improved. Then, carrying out model accuracy detection on the trained license plate recognition model; and when the accuracy of the trained license plate recognition model is greater than or equal to a preset accuracy threshold, obtaining a target license plate recognition model. Therefore, the obtained target license plate recognition model is small in structure and simple in calculation process, and the accuracy of the target license plate recognition model can meet the requirement.

According to a second aspect, an embodiment of the present invention further provides a license plate recognition apparatus, including:

the acquisition module is used for acquiring a license plate image to be recognized corresponding to a target vehicle;

the detection module is used for detecting a license plate region in a license plate image to be recognized and determining position information and angle information of the license plate region; the angle information is used for representing the deflection angle of the license plate area relative to the reference position;

the adjusting module is used for adjusting the position of the license plate region according to the position information and the angle information to obtain a target license plate region after the position is adjusted;

and the determining module is used for identifying the target license plate area and determining the license plate information of the target vehicle.

The license plate recognition device provided by the embodiment of the application acquires a license plate image to be recognized corresponding to a target vehicle, detects a license plate region in the license plate image to be recognized, and determines position information and angle information of the license plate region. According to the position information and the angle information of the license plate area, the position of the target license plate in the license plate image to be recognized can be accurately determined, so that the license plate of the target vehicle in the license plate image to be recognized can be recognized conveniently. In addition, the position of the license plate region is adjusted according to the position information and the angle information, and the target license plate region after the position is adjusted is obtained. The adjusted target license plate area is easier to identify, the condition that the license plate area is inclined upwards and downwards or the license plate area is inclined inwards and outwards is improved, and therefore the accuracy of identifying the license plate of the target vehicle is improved when the target license plate area is identified. The problem that the license plate of the target vehicle is identified inaccurately due to the position information of the license plate of the target vehicle in the license plate image to be identified is avoided. Therefore, the license plate identification method can improve the accuracy of identifying the license plate information.

According to a third aspect, an embodiment of the present invention provides an electronic device, which includes a memory and a processor, where the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions, so as to execute the license plate recognition method in the first aspect or any one of the implementation manners of the first aspect.

According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions for causing a computer to execute the license plate recognition method in the first aspect or any one of the implementation manners of the first aspect.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.

FIG. 1 is a flow chart of a license plate recognition method according to an embodiment of the present invention;

fig. 2(a) is a schematic diagram of a license plate region before adjusting the position of the license plate region in a license plate recognition method according to another embodiment of the present invention;

fig. 2(b) is a schematic diagram after the position of the license plate region is adjusted by applying the license plate recognition method according to another embodiment of the present invention;

FIG. 3 is a flowchart illustrating a license plate recognition method according to another embodiment of the present invention;

FIG. 4 is a flowchart illustrating a license plate recognition method according to another embodiment of the present invention;

FIG. 5 is a flowchart illustrating a license plate recognition method according to another embodiment of the present invention;

FIG. 6 is a functional block diagram of a license plate recognition device according to an embodiment of the present invention;

fig. 7 is a schematic diagram of a hardware structure of an electronic device to which an embodiment of the present invention is applied.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

With the development of science and technology, the quantity of vehicles kept is also increasing. More and more scenes need to identify the license plate of the vehicle, so that the effective management of the vehicle is achieved.

For example, in some large-scale construction site scenes, in order to ensure the safety of a construction site, license plate recognition is performed on vehicles entering and exiting the construction site, so as to determine whether each vehicle is allowed to enter and exit the construction site, thereby avoiding external vehicles from entering the construction site and ensuring the safety of the construction site.

In some indoor and outdoor parking lots, license plates of vehicles entering and exiting the parking lots need to be identified, and the time of the vehicles entering and exiting the parking lots is determined, so that reasonable charging is conveniently carried out on the parking time of the vehicles.

In some confidential units, in order to achieve strict management of the unit, it is necessary to reject the entrance of a foreign vehicle, and therefore, it is necessary to identify the license plate of each vehicle entering and exiting the confidential unit to determine whether the vehicle is the unit vehicle.

In a large-scale activity site, in order to maintain the order of activities and ensure the safety of the activity site, it is strict that foreign vehicles enter, and therefore, the license plates of the vehicles need to be identified so as to determine whether the vehicles are inside.

In the prior art, the accuracy of the license plate recognition equipment based on video image license plate recognition is not high, and the recognition rate of a license plate recognition system still has more space for improvement.

Therefore, the application provides a license plate recognition method, a license plate recognition device, electronic equipment and a storage medium. The license plate region in the license plate image to be recognized is detected by acquiring the license plate image to be recognized corresponding to the target vehicle, and the position information and the angle information of the license plate region are determined. According to the position information and the angle information of the license plate area, the position of the target license plate in the license plate image to be recognized can be accurately determined, so that the license plate of the target vehicle in the license plate image to be recognized can be recognized conveniently. In addition, the position of the license plate region is adjusted according to the position information and the angle information, and the target license plate region after the position is adjusted is obtained. The adjusted target license plate area is easier to identify, the condition that the license plate area is inclined upwards and downwards or the license plate area is inclined inwards and outwards is improved, and therefore the accuracy of identifying the license plate of the target vehicle is improved when the target license plate area is identified. The problem that the license plate of the target vehicle is identified inaccurately due to the position information of the license plate of the target vehicle in the license plate image to be identified is avoided. Therefore, the license plate identification method can improve the accuracy of identifying the license plate information.

It should be noted that in the method for recognizing a license plate provided in the embodiment of the present application, an execution subject may be a device for recognizing a license plate, and the device for recognizing a license plate may be implemented as part or all of an electronic device in a software, hardware, or a combination of software and hardware, where the electronic device may be a server or a terminal, where the server in the embodiment of the present application may be one server or a server cluster composed of multiple servers, and the terminal in the embodiment of the present application may be another intelligent hardware device such as a smart phone, a personal computer, a tablet computer, a wearable device, and an intelligent robot. In the following method embodiments, the execution subject is an electronic device as an example.

In an embodiment of the present application, as shown in fig. 1, a license plate recognition method is provided, which is described by taking an application and an electronic device of the method as an example, and includes the following steps:

and S11, acquiring a license plate image to be recognized corresponding to the target vehicle.

In an optional implementation manner, the electronic device may acquire the license plate image to be recognized, which is sent by the image acquisition device, based on the connection with the image acquisition device.

In another alternative embodiment, the electronic device may further acquire the license plate image to be recognized by using an image acquisition device provided in the electronic device.

In another alternative embodiment, the electronic device may receive the license plate image to be recognized sent by the other device based on the connection with the other device.

S12, detecting the license plate area in the license plate image to be recognized, and determining the position information and the angle information of the license plate area.

The angle information is used for representing the deflection angle of the license plate area relative to the reference position.

In an optional implementation manner, the electronic device may detect coordinate information of four corners corresponding to a license plate region in a license plate image to be recognized, and determine position information and angle information of the license plate region according to the coordinate information of the four corners corresponding to the license plate region.

In another optional implementation manner, the electronic device may input the license plate image to be recognized into the license plate region detection model, and determine the position information and the angle information of the license plate region.

For a detailed explanation of this step, please see below.

And S13, adjusting the position of the license plate region according to the position information and the angle information, and acquiring the target license plate region after the position is adjusted.

Specifically, the electronic device can adjust the position of the license plate region in the license plate image to be recognized according to the position information of the license plate region. And then, according to the angle information of the license plate region, adjusting the deflection angle of the license plate region relative to the reference position, and acquiring the target license plate region after the position adjustment. The reference position can be the position of a plane reference plane corresponding to the license plate image to be recognized.

For example, fig. 2(a) shows the license plate region before the adjustment, and fig. 2(b) shows the adjusted target license plate region.

And S14, identifying the target license plate area and determining the license plate information of the target vehicle.

In an optional implementation manner, the electronic device may identify a license plate in the target license plate region by using the target detection model, and determine license plate information of the target vehicle. The target detection Model may be a Model based on manual features, such as a DPM (Deformable Parts Model), or a Model based on a Convolutional Neural network, such as a YOLO (You Look Only Once) detector, an R-CNN (Region-based Convolutional Neural network), an SSD (Single Shot multi box) detector, a Mask R-CNN (Mask Region-based Convolutional Neural network), and the like. The embodiment of the present application does not specifically limit the target detection model.

In another optional implementation, the electronic device may further identify license plate information in the target license plate region by using an OCR algorithm, so as to determine the license plate information of the target vehicle.

For example, the electronic device may obtain a license plate image to be recognized corresponding to the construction vehicle on a construction site, detect a license plate region of the construction vehicle in the license plate image to be recognized, and determine position information and angle information of the license plate region. And adjusting the position of the license plate region according to the position information and the angle information of the license plate region of the construction vehicle, and acquiring the target license plate region after the position is adjusted. And then, determining the license plate information of the construction vehicle for the license plate information in the target license plate area by using a target detection model or a character recognition algorithm.

The license plate recognition method provided by the embodiment of the application obtains the license plate image to be recognized corresponding to the target vehicle, detects the license plate area in the license plate image to be recognized, and determines the position information and the angle information of the license plate area. According to the position information and the angle information of the license plate area, the position of the target license plate in the license plate image to be recognized can be accurately determined, so that the license plate of the target vehicle in the license plate image to be recognized can be recognized conveniently. In addition, the position of the license plate region is adjusted according to the position information and the angle information, and the target license plate region after the position is adjusted is obtained. The adjusted target license plate area is easier to identify, the condition that the license plate area is inclined upwards and downwards or the license plate area is inclined inwards and outwards is improved, and therefore the accuracy of identifying the license plate of the target vehicle is improved when the target license plate area is identified. The problem that the license plate of the target vehicle is identified inaccurately due to the position information of the license plate of the target vehicle in the license plate image to be identified is avoided. Therefore, the license plate identification method can improve the accuracy of identifying the license plate information.

In an alternative embodiment of the present application, as shown in fig. 3, a license plate recognition method is provided, which is described by taking an application and an electronic device of the method as an example, and includes the following steps:

and S21, acquiring a candidate image of the target vehicle when the target vehicle enters the target scene.

The candidate image may be an image corresponding to the target vehicle acquired after the target vehicle performs the target scene.

In an alternative embodiment, the electronic device obtains the candidate image of the target vehicle in real time when the target vehicle enters the target scene. The target scene can be a construction site, a parking lot, a private unit, an activity site and the like, and the target scene is not particularly limited in the embodiment of the application.

For example, when the target vehicle travels a distance of 20 meters from the parking lot, the acquired image corresponding to the target vehicle may be referred to as a candidate image.

And S22, identifying the position of the target vehicle in the candidate image, and judging whether the target vehicle in the candidate image enters a target area in the target scene.

In an alternative embodiment, the electronic device may identify the target vehicle in the candidate image using the target detection model to determine the position of the target vehicle in the candidate image. And then judging whether the target vehicle in the candidate image enters a target area in the target scene or not according to the position of the target vehicle in the candidate image.

In another alternative embodiment, the electronic device may identify the target vehicle in the candidate image using the target detection model, thereby determining the size of the target vehicle in the candidate image. And then judging whether the target vehicle in the candidate image enters a target area in the target scene or not according to the size of the target vehicle in the candidate image.

And S23, when the target vehicle in the candidate image enters the target area, determining the candidate image as the license plate image to be recognized so as to obtain at least two license plate images to be recognized.

Specifically, when the electronic equipment determines that the target vehicle in the candidate images enters the target area, the candidate images are determined to be license plate images to be recognized, and at least two license plate images to be recognized of the target vehicle in the target area are obtained again

Illustratively, when a target vehicle enters the door of a confidential institution, a candidate image of the target vehicle is acquired. And then, the position of the target vehicle in the candidate image is identified, and whether the target vehicle in the candidate image enters a range of 10 meters away from the doorway of the confidential unit is judged. And if the target vehicle in the candidate image enters a range of 10 meters away from the doorway of the confidential unit, determining the candidate image as the license plate image to be recognized, and acquiring at least two license plate images to be recognized again.

S24, detecting the license plate area in the license plate image to be recognized, and determining the position information and the angle information of the license plate area.

The detailed description of this step can be referred to the description of S12 in fig. 1.

And S25, adjusting the position of the license plate region according to the position information and the angle information, and acquiring the target license plate region after the position is adjusted.

Wherein, the above S25 may include the following contents:

Specifically, the electronic device may perform affine transformation based on the reference position according to the position information and the angle information of the license plate region, correct the angle of the license plate region, and obtain the angle-adjusted target license plate region. Thereby making the location of the target license plate region easier to identify.

According to the position information and the angle information of the license plate region, affine transformation is carried out by taking the reference position as the reference, the angle of the license plate region is corrected, the target license plate region after angle adjustment is obtained, the corrected target license plate region can be ensured to be coincident with the reference position, the problem that the target license plate region inclines inwards or outwards is solved, and the accuracy of recognizing the target license plate region is ensured.

And S26, identifying the target license plate area and determining the license plate information of the target vehicle.

Wherein, the step S26 may include the following steps:

s261, identifying target license plate areas corresponding to at least two license plate images to be identified, and determining corresponding predicted license plate information and probability thereof.

Specifically, the electronic device may identify target license plate regions corresponding to at least two license plate images to be identified by using a target identification model or a character identification algorithm, and output predicted license plate information and probabilities thereof corresponding to each target license plate region.

Exemplarily, the electronic device identifies a target license plate region corresponding to a first license plate image to be identified, and outputs first predicted license plate information corresponding to the first license plate image to be identified as F01 with a probability of 80%, second predicted license plate information as F02 with a probability of 20%; the electronic equipment identifies a target license plate region corresponding to the second license plate image to be identified, outputs third predicted license plate information corresponding to the second license plate image to be identified as F01, and has a probability of 75%; the fourth predicted license plate information is F03, and the probability is 25%; the electronic equipment identifies a target license plate region corresponding to the third license plate image to be identified, outputs fifth predicted license plate information corresponding to the third license plate image to be identified as F01, and has a probability of 85%; the sixth predicted license plate information is F02, and the probability is 15%.

S262, comparing the probabilities and determining the predicted license plate information with the maximum probability as the license plate information of the target license plate.

Specifically, the electronic device compares the probabilities of the predicted license plate information corresponding to each license plate image to be recognized, and determines the predicted license plate information with the maximum probability as the license plate information of the target license plate.

In addition, the target license plate regions corresponding to at least two license plate images to be recognized are recognized, and corresponding predicted license plate information and the probability thereof are determined; then, the probability is compared, and the predicted license plate information with the maximum probability is determined as the license plate information of the target license plate. Instead of only one license plate image to be recognized, the license plate image is determined as the license plate information of the target license plate. Only one license plate image to be recognized is recognized, and the license plate information determined as the license plate information of the target license plate may be inaccurate in recognition. Therefore, the license plate identification method can improve the accuracy of the license plate information of the determined target license plate.

In an optional embodiment of the application, the electronic device may further include an infrared device or a radar device, and the electronic device detects a position of the target vehicle in the target scene by transmitting infrared light or radar in real time, and then determines whether the target vehicle enters a target area in the target scene according to the position of the target vehicle in the target scene. When a target vehicle enters a target area in a target scene, the target vehicle is photographed, and at least two license plate images to be recognized are obtained.

In an optional embodiment of the present application, as shown in fig. 4, the position information and the angle information of the license plate region in the above steps are determined by a license plate region detection model, and a training process of the license plate region detection model includes the following steps:

and S31, acquiring a sample license plate image and a label thereof.

The label comprises target position information and target angle information of a license plate area in a sample license plate image.

Specifically, the electronic equipment can receive a sample license plate image and a label thereof input by a user; and receiving a sample license plate image and a label thereof sent by other equipment.

Target position information of the license plate area in the sample license plate image label can be identified by a user, and a detection frame is added for marking. The target angle information of the license plate region in the sample license plate image label can be obtained by calculation based on the target position information of the license plate region.

For example, assuming that the coordinates of the upper left corner of the license plate region are (x1, y1) and the coordinates of the upper right corner of the license plate region are (x2, y2), the target angle information θ of the license plate region is calculated in the formula (1):

θ＝arctan((y2-y1)/(x2-x1)) (1)。

s32, inputting the sample license plate image into a preset license plate region detection model, and determining the predicted position information and the predicted angle information of the sample license plate region.

Specifically, the electronic equipment inputs a sample license plate image into a preset license plate region detection model, the preset license plate region detection model performs target detection on the sample license plate image, so that a sample license plate region is determined, and prediction position information and prediction angle information of the sample license plate region are output according to the determined license plate region. The preset license plate Region detection Model may be a Model based on manual features, such as a DPM (Deformable Parts Model), or a Model based on a Convolutional Neural network, such as a YOLO (You Only see Once) detector, an R-CNN (Region-based Convolutional Neural network), an SSD (Single Shot multi box) detector, a Mask R-CNN (Mask Region-based Convolutional Neural network) Model, and the like. The embodiment of the application does not specifically limit the preset license plate region detection model.

And S33, calculating the position loss based on the predicted position information.

Specifically, the electronic device may calculate the position loss based on a difference between the predicted position information and the target position information.

And S34, calculating the angle loss based on the predicted angle information.

In an alternative embodiment, the electronic device may calculate the angle loss based on a difference between the predicted angle information and the target angle information.

In another alternative embodiment, step S34 may include the following steps:

and S341, calculating cosine values by using the predicted angle information.

And S342, calculating the angle loss based on the cosine value.

In an alternative embodiment, the electronic device calculates a cosine value using the predicted angle information, and then uses the calculated cosine value as the angle loss.

In another alternative embodiment, the electronic device calculates the cosine value using the predicted angle information, and then calculates the angle loss using 1 minus the cosine value.

Illustratively, the angle function is of formula (2):

Loss(θ)＝1-cos(θ) (2)；

and theta is the difference value between the actual license plate angle of the license plate area and the model prediction angle.

And S35, updating parameters of the license plate region detection model based on the position loss and the angle loss, and determining the license plate region detection model.

In an optional implementation manner, the electronic device may update parameters of the license plate region detection model according to the position loss and the angle loss, respectively, and determine the license plate region detection model.

In an optional implementation manner, the electronic device may further multiply the position loss and the angle loss by corresponding weights respectively, add the position loss and the angle loss to generate a new loss, and then update parameters of the license plate region detection model by using the new loss to determine the license plate region detection model.

According to the license plate identification method provided by the embodiment of the application, the sample license plate image and the label thereof are obtained, the sample license plate image is input into the preset license plate region detection model, and the prediction position information and the prediction angle information of the sample license plate region are determined. And then, calculating position loss based on the predicted position information, calculating angle loss based on the predicted angle information, updating parameters of the license plate region detection model based on the position loss and the angle loss, and determining the license plate region detection model. The cosine value can be calculated by utilizing the predicted angle information, the angle loss can be calculated based on the cosine value, and the accuracy of the calculated angle loss can be ensured. According to the license plate recognition method, the preset license plate region detection model is trained according to the sample license plate image and the label thereof, so that the accuracy of the trained license plate region detection model can be ensured, the accuracy of the position information and the angle information of the license plate region acquired based on the license plate region detection model is further ensured, and the accuracy of recognizing the license plate of the target vehicle is ensured.

In an alternative embodiment of the present application, as shown in fig. 5, a license plate recognition method is provided, which is described by taking an application and an electronic device of the method as an example, and includes the following steps:

and S41, acquiring a license plate image to be recognized corresponding to the target vehicle.

For a detailed description of this step, please refer to the description of S11 in fig. 1, and this step will not be described again.

S42, detecting the license plate area in the license plate image to be recognized, and determining the position information and the angle information of the license plate area.

For a detailed description of this step, please refer to fig. 3, which is not repeated herein.

And S43, adjusting the position of the license plate region according to the position information and the angle information, and acquiring the target license plate region after the position is adjusted.

For a detailed description of this step, please refer to the description of S13 in fig. 1 and the description of S25 in fig. 2, and this step will not be described again.

And S44, identifying the target license plate area and determining the license plate information of the target vehicle.

Wherein, S44 may include the following steps:

s441, inputting the target license plate area into the target license plate recognition model.

And the target license plate recognition model is obtained by compressing the license plate recognition model.

The process of compressing the license plate recognition model to obtain the target license plate recognition model comprises the following steps:

(1) and pruning the license plate recognition model.

(2) And performing model training on the vehicle license plate recognition model after pruning.

(3) Carrying out model accuracy detection on the trained license plate recognition model;

(4) and when the accuracy of the trained license plate recognition model is greater than or equal to a preset accuracy threshold, obtaining a target license plate recognition model.

Specifically, the electronic device may perform pruning processing on the license plate recognition model by using a preset pruning method. The Pruning method may include, but is not limited to, Network Pruning (Network Pruning), Target Pruned architecture (Target Pruned architecture), and the like, where the Network Pruning may include a sparse training method based on a battnorm parameter, and may also include other methods.

The electronic equipment inputs the license plate recognition sample into the license plate recognition model after pruning, model training is carried out on the license plate recognition model after pruning, and parameters of the license plate recognition model after pruning are adjusted to obtain the trained license plate recognition model.

And the electronic equipment inputs the license plate recognition sample into the trained license plate recognition model again, and model accuracy detection is carried out on the trained license plate recognition model. And when the accuracy of the trained license plate recognition model is greater than or equal to a preset accuracy threshold, obtaining a target license plate recognition model. If the accuracy of the trained license plate recognition model is smaller than the preset accuracy threshold, pruning the license plate recognition model again, performing model training on the license plate recognition model after pruning, and repeating the steps until the accuracy of the trained license plate recognition model is larger than or equal to the preset accuracy threshold, so as to obtain the target license plate recognition model.

According to the method, the license plate recognition model is pruned, so that the structure of the license plate recognition model can be reduced, and the calculation process of the license plate recognition model is simplified. After the vehicle license plate recognition model is pruned, the accuracy of the vehicle license plate recognition model may be reduced. Therefore, model training is carried out on the vehicle license plate recognition model after pruning, and accuracy of the vehicle license plate recognition model after pruning can be improved. Then, carrying out model accuracy detection on the trained license plate recognition model; and when the accuracy of the trained license plate recognition model is greater than or equal to a preset accuracy threshold, obtaining a target license plate recognition model. Therefore, the obtained target license plate recognition model is small in structure and simple in calculation process, and the accuracy of the target license plate recognition model can meet the requirement.

S442, the target license plate recognition model recognizes the target license plate area and outputs a recognition result corresponding to the target license plate area.

Wherein, the recognition result comprises at least one of characters, numbers and letters.

Specifically, the electronic device identifies a target license plate region by using a target license plate identification model to obtain a license plate in the target license plate region, then identifies each character in the target license plate, and outputs an identification result corresponding to the target license plate region.

The target license plate recognition Model may be a Model based on manual features, such as a DPM (Deformable Parts Model), or a Model based on a Convolutional Neural network, such as a YOLO (You Only see Once) detector, an R-CNN (Region-based Convolutional Neural network), an SSD (Single-Shot multiple box) detector, a Mask R-CNN (Mask Region-based Convolutional Neural network), and the like. The embodiment of the application does not specifically limit the target license plate recognition model.

Illustratively, the recognition result may include "jing", "a", "0", "3", "5", "U", and the like.

S443, analyzing the recognition result based on license plate rule analysis, and outputting license plate information of the target license plate.

Specifically, the electronic device splices characters included in a recognition result output by the target license plate recognition model based on license plate rule analysis, and outputs license plate information of the target license plate.

The license plate rule analysis can be used for sequencing all characters included in the recognition result output by the target license plate recognition model according to a preset sequence.

For example, assume that the recognition results are "jing", "a", "0", "3", "5", "U". The electronic equipment analyzes the recognition result based on the license plate rule analysis, and the license plate information of the output target license plate is 'Jing A035U'.

It should be understood that although the various steps in the flowcharts of fig. 1, and 3-5 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1, and 3-5 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least some of the other steps.

As shown in fig. 6, the present embodiment provides a license plate recognition device, including:

the acquiring module 51 is configured to acquire a license plate image to be recognized corresponding to a target vehicle;

the detection module 52 is configured to detect a license plate region in a license plate image to be recognized, and determine position information and angle information of the license plate region; the angle information is used for representing the deflection angle of the license plate area relative to the reference position;

the adjusting module 53 is configured to perform position adjustment on the license plate region according to the position information and the angle information, and obtain a target license plate region after the position adjustment;

and the determining module 54 is configured to identify the target license plate region and determine license plate information of the target vehicle.

In an embodiment of the present application, the number of license plate images to be recognized is at least two, and the obtaining module 51 is specifically configured to obtain a candidate image of a target vehicle when the target vehicle enters a target scene; identifying the position of a target vehicle in the candidate image, and judging whether the target vehicle in the candidate image enters a target area in a target scene; and when the target vehicle in the candidate image enters the target area, determining the candidate image as the license plate image to be recognized so as to obtain at least two license plate images to be recognized.

In an embodiment of the present application, the determining module 54 is specifically configured to identify target license plate regions corresponding to at least two license plate images to be identified, and determine corresponding predicted license plate information and probability thereof; and comparing the probabilities and determining the predicted license plate information of the maximum probability as the license plate information of the target license plate.

In an embodiment of the present application, the position information and the angle information of the license plate region are determined by a license plate region detection model, and the detection module 52 is specifically configured to obtain a sample license plate image and a tag thereof, where the tag includes target position information and target angle information of the license plate region in the sample license plate image; inputting a sample license plate image into a preset license plate region detection model, and determining predicted position information and predicted angle information of the sample license plate region; calculating a location loss based on the predicted location information; calculating an angle loss based on the predicted angle information; and updating parameters of the license plate region detection model based on the position loss and the angle loss, and determining the license plate region detection model.

In an embodiment of the present application, the detecting module 52 is specifically configured to calculate a cosine value by using the predicted angle information; the angle loss is calculated based on the cosine value.

In an embodiment of the present application, the adjusting module 53 is specifically configured to, for a license plate region, perform affine transformation based on a reference position according to the position information and the angle information, correct an angle of the license plate region, and obtain an angle-adjusted target license plate region.

In an embodiment of the present application, the determining module 54 is specifically configured to input the target license plate region into the target license plate recognition model; the target license plate recognition model is obtained by compressing the license plate recognition model; the target license plate recognition model recognizes a target license plate region and outputs a recognition result corresponding to the target license plate region; the recognition result comprises at least one of characters, numbers and letters; and analyzing the recognition result based on license plate rule analysis, and outputting license plate information of the target license plate.

In an embodiment of the present application, the determining module 54 is specifically configured to perform pruning processing on the license plate recognition model; performing model training on the vehicle license plate recognition model after pruning; carrying out model accuracy detection on the trained license plate recognition model; and when the accuracy of the trained license plate recognition model is greater than or equal to a preset accuracy threshold, obtaining a target license plate recognition model.

For specific limitations and beneficial effects of the license plate recognition device, reference may be made to the above limitations on the license plate recognition method, which are not described herein again. All or part of the modules in the license plate recognition device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent of a processor in the electronic device, or can be stored in a memory in the electronic device in a software form, so that the processor can call and execute operations corresponding to the modules.

An embodiment of the present invention further provides an electronic device, which has the license plate recognition apparatus shown in fig. 6.

As shown in fig. 7, fig. 7 is a schematic structural diagram of an electronic device according to an alternative embodiment of the present invention, and as shown in fig. 7, the electronic device may include: at least one processor 61, such as a CPU (Central Processing Unit), at least one communication interface 63, memory 64, at least one communication bus 62. Wherein a communication bus 62 is used to enable the connection communication between these components. The communication interface 63 may include a Display (Display) and a Keyboard (Keyboard), and the optional communication interface 63 may also include a standard wired interface and a standard wireless interface. The Memory 64 may be a high-speed RAM Memory (volatile Random Access Memory) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The memory 64 may optionally be at least one memory device located remotely from the processor 61. Wherein the processor 61 may be in connection with the apparatus described in fig. 6, an application program is stored in the memory 64, and the processor 61 calls the program code stored in the memory 64 for performing any of the above-mentioned method steps.

The communication bus 62 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus 62 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 7, but this is not intended to represent only one bus or type of bus.

The memory 64 may include a volatile memory (RAM), such as a random-access memory (RAM); the memory may also include a non-volatile memory (english: non-volatile memory), such as a flash memory (english: flash memory), a hard disk (english: hard disk drive, abbreviated: HDD) or a solid-state drive (english: SSD); the memory 64 may also comprise a combination of the above types of memory.

The processor 61 may be a Central Processing Unit (CPU), a Network Processor (NP), or a combination of CPU and NP.

The processor 61 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a Programmable Logic Device (PLD), or a combination thereof. The PLD may be a Complex Programmable Logic Device (CPLD), a field-programmable gate array (FPGA), a General Array Logic (GAL), or any combination thereof.

Optionally, the memory 64 is also used to store program instructions. The processor 61 may call a program instruction to implement the license plate recognition method shown in fig. 1 and fig. 3-5 of the present application.

The embodiment of the invention also provides a non-transitory computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions can execute the license plate identification method in any method embodiment. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD) or a Solid State Drive (SSD), etc.; the storage medium may also comprise a combination of memories of the kind described above.

Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A license plate recognition method is characterized by comprising the following steps:

detecting a license plate region in the license plate image to be recognized, and determining position information and angle information of the license plate region; the angle information is used for representing the deflection angle of the license plate area relative to a reference position;

adjusting the position of the license plate region according to the position information and the angle information to obtain a target license plate region with the adjusted position;

and identifying the target license plate area, and determining license plate information of the target vehicle.

2. The method according to claim 1, wherein the number of the license plate images to be recognized is at least two, and the obtaining of the license plate images to be recognized corresponding to the target vehicle comprises:

when the target vehicle enters a target scene, acquiring a candidate image of the target vehicle;

identifying the position of the target vehicle in the candidate image, and judging whether the target vehicle in the candidate image enters a target area in the target scene;

3. The method of claim 2, wherein the identifying the target license plate region and determining license plate information of the target license plate comprises:

identifying the target license plate areas corresponding to at least two license plate images to be identified, and determining corresponding predicted license plate information and probability thereof;

and comparing the probabilities and determining the predicted license plate information with the maximum probability as the license plate information of the target license plate.

4. The method of claim 1, wherein the position information and the angle information of the license plate region are determined by a license plate region detection model, and the training process of the license plate region detection model comprises the following steps:

inputting the sample license plate image into a preset license plate region detection model, and determining predicted position information and predicted angle information of the sample license plate region;

calculating a location loss based on the predicted location information;

calculating an angle loss based on the predicted angle information;

5. The method of claim 4, wherein determining an angle loss based on the predicted angle information comprises:

calculating a cosine value using the predicted angle information;

calculating the angle loss based on the cosine value.

6. The method of claim 1, wherein the adjusting the position of the license plate region according to the position information and the angle information to obtain a position-adjusted target license plate region comprises:

and performing affine transformation on the license plate region by taking the reference position as a reference according to the position information and the angle information, correcting the angle of the license plate region, and obtaining the target license plate region after angle adjustment.

7. The method of claim 1, wherein the identifying the target license plate region and determining license plate information of the target vehicle comprises:

inputting the target license plate area into a target license plate recognition model; the target license plate recognition model is obtained by compressing a license plate recognition model;

the target license plate recognition model recognizes the target license plate region and outputs a recognition result corresponding to the target license plate region; the recognition result comprises at least one of characters, numbers and letters;

8. The method of claim 7, wherein the step of compressing the license plate recognition model to obtain the target license plate recognition model comprises:

pruning the license plate recognition model;

and when the accuracy of the trained license plate recognition model is greater than or equal to a preset accuracy threshold, obtaining the target license plate recognition model.

9. A license plate recognition device, the device comprising:

the detection module is used for detecting a license plate region in the license plate image to be recognized and determining the position information and the angle information of the license plate region; the angle information is used for representing the deflection angle of the license plate area relative to a reference position;

10. An electronic device, comprising a memory and a processor, wherein the memory stores computer instructions, and the processor executes the computer instructions to perform the license plate recognition method of any one of claims 1-8.

11. A computer-readable storage medium storing computer instructions for causing a computer to perform the license plate recognition method according to any one of claims 1 to 8.