CN115376119A - License plate recognition method and device, license plate recognition equipment and storage medium - Google Patents
License plate recognition method and device, license plate recognition equipment and storage medium Download PDFInfo
- Publication number
- CN115376119A CN115376119A CN202211309201.0A CN202211309201A CN115376119A CN 115376119 A CN115376119 A CN 115376119A CN 202211309201 A CN202211309201 A CN 202211309201A CN 115376119 A CN115376119 A CN 115376119A
- Authority
- CN
- China
- Prior art keywords
- license plate
- network
- feature
- trained
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention is suitable for the technical field of computers, and provides a license plate recognition method, a license plate recognition device and a storage medium, wherein the method comprises the following steps: performing feature extraction on the license plate image to be recognized by adopting a trained feature extraction network to obtain a plurality of feature maps of the license plate image to be recognized at different levels; acquiring the category information of the license plate from a deep characteristic map in the characteristic maps through a trained classification network; obtaining the peak information of the license plate from a deep characteristic diagram in the characteristic diagrams through a trained regression network, and performing peak adjustment on the peak information; and intercepting the corresponding license plate feature map from a shallow feature map in the multiple feature maps according to the adjusted vertex information, and identifying the license plate feature map by using a trained license plate identification network to obtain the license plate number, so that the vertex information of the license plate is obtained from the deep feature map by using the trained regression network, and the vertex information is adjusted, thereby improving the accuracy of multi-directional license plate identification.
Description
Technical Field
The invention belongs to the technical field of artificial intelligence, and particularly relates to a license plate recognition method and device, license plate recognition equipment and a storage medium.
Background
License plate recognition has extremely wide application in scenes such as high-speed gates, parking lot access, road safety and the like. In the license plate recognition in the field of computer vision, a shot picture is taken as input, a license plate number is taken as output, the position of the license plate needs to be positioned firstly in the recognition process, and then the license plate is intercepted for recognition. However, in a real scene, the directions of the license plates in the picture are different due to different shooting angles of the monitoring device, so that the license plates have perspective transformation in different degrees, and certain challenges are brought to license plate detection. The traditional license plate detection method needs to perform characteristic engineering according to the remarkable characteristics of the license plate, namely, characteristics such as edges, textures, colors and characters are designed manually. However, the methods have weak anti-interference capability and poor generalization capability, can cause detection errors under the interference of illumination, other characters and the like, are only suitable for horizontal license plates, and can cause subsequent multidirectional license plate recognition errors. Meanwhile, the traditional license plate recognition method needs to detect a single character of the license plate on the basis of intercepting the license plate, and then recognize the character to obtain the license plate number. The method depends on the detection results of the license plate and the characters, the problem of error accumulation is easily caused, a plurality of independent models are needed, and the license plate recognition efficiency is reduced. In addition, the method requires a license plate label at a character level, which results in a large amount of manpower and material resources.
Since the development of deep learning methods, a large number of license plate recognition methods based on a deep neural network are emerging continuously, the methods can be used for feature learning based on data, corresponding features do not need to be designed artificially, and the anti-jamming capability and the generalization capability are strong. However, the existing license plate detection method based on the deep neural network still has certain problems in multi-direction license plate detection, for example, the positioning of each vertex of the license plate is taken as an independent task, and the mutual relation between the vertices is ignored, so that the detection efficiency and accuracy of the multi-direction license plate detection are reduced. Meanwhile, the existing license plate recognition method based on the deep neural network also has certain problems, for example, the license plate features are extracted from the deep features of the network, and the license plate features are seriously lost due to low resolution of the deep features.
Disclosure of Invention
The invention aims to provide a license plate recognition method, a license plate recognition device, license plate detection equipment and a storage medium, and aims to solve the problem that the multi-direction license plate recognition accuracy is low because an effective multi-direction license plate recognition method cannot be provided in the prior art.
In one aspect, the present invention provides a license plate recognition method, including:
performing feature extraction on a license plate image to be recognized by adopting a trained feature extraction network to obtain a plurality of feature maps of the license plate image to be recognized at different levels;
acquiring category information of the license plate from a deep feature map in the feature maps through a trained classification network, acquiring vertex information of the license plate from the deep feature map in the feature maps through a trained regression network, and adjusting the vertex information to obtain the adjusted vertex information;
and intercepting a corresponding license plate feature map from a shallow feature map in the feature maps according to the adjusted vertex information, and identifying the license plate feature map by using a trained license plate identification network to obtain a license plate number corresponding to the license plate image to be identified.
In another aspect, the present invention provides a license plate recognition apparatus, including:
the system comprises a feature extraction unit, a feature extraction unit and a feature extraction unit, wherein the feature extraction unit is used for extracting features of a license plate image to be recognized by adopting a trained feature extraction network to obtain a plurality of feature maps of the license plate image to be recognized at different levels;
the frame adjusting unit is used for acquiring the category information of the license plate from a deep feature map in the feature maps through a trained classification network, acquiring the vertex information of the license plate from the deep feature map in the feature maps through a trained regression network, and adjusting the vertex information to obtain the adjusted vertex information; and
and the license plate recognition unit is used for intercepting corresponding license plate feature maps from shallow feature maps in the feature maps according to the adjusted vertex information, and recognizing the license plate feature maps by using a trained license plate recognition network so as to obtain the license plate number corresponding to the license plate image to be recognized.
In another aspect, the present invention further provides a license plate recognition apparatus, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method when executing the computer program.
In another aspect, the present invention also provides a computer-readable storage medium, which stores a computer program that, when executed by a processor, implements the steps of the method as described above.
The method comprises the steps of extracting features of a license plate image to be recognized by adopting a trained feature extraction network to obtain a plurality of feature maps of the license plate image to be recognized at different levels, obtaining category information of the license plate from deep feature maps in the feature maps by using the trained classification network, obtaining peak information of the license plate from the deep feature maps in the feature maps by using the trained regression network, adjusting the peak information to obtain adjusted peak information, intercepting corresponding license plate feature maps from shallow feature maps in the feature maps according to the adjusted peak information, and recognizing the license plate feature maps by using the trained license plate recognition network to obtain the license plate number corresponding to the license plate image to be recognized, so that the peak information of the license plate is obtained from the deep feature maps by using the trained regression network, and the peak information is adjusted in multiple directions, thereby improving the accuracy of license plate recognition.
Drawings
Fig. 1 is a flowchart illustrating an implementation of a license plate recognition method according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating an implementation of a license plate recognition method according to a second embodiment of the present invention;
fig. 3 is a flowchart illustrating an implementation of a license plate recognition method according to a third embodiment of the present invention;
fig. 4 is a flowchart illustrating an implementation of a license plate recognition method according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of a license plate recognition device according to a fifth embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of a license plate recognition device according to a sixth embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The following detailed description of specific implementations of the present invention is provided in conjunction with specific embodiments:
the first embodiment is as follows:
fig. 1 shows an implementation flow of a license plate recognition method provided in an embodiment of the present invention, and for convenience of description, only parts related to the embodiment of the present invention are shown, which are detailed as follows:
in the step S101, a trained feature extraction network is adopted to extract features of the license plate image to be recognized, so that a plurality of feature maps of the license plate image to be detected in different levels are obtained;
the embodiment of the invention is suitable for license plate recognition equipment, such as a monitoring camera with a license plate recognition function and a license plate recognition computer, and is used for obtaining a license plate (number) from an acquired license plate image to be recognized. In the embodiment of the invention, a feature extraction network is adopted to extract features of the license plate image to be recognized, so that a plurality of feature maps of the license plate image to be recognized in different levels, such as a deep feature map and a shallow feature map, are obtained, the deep feature map has low resolution and large receptive field, and is convenient for detecting large-size license plates, and the shallow feature map has high resolution and small receptive field, and is convenient for detecting small-size license plates. The feature extraction network may be a backbone network for feature extraction, such as a convolutional neural network, and in a specific embodiment, the backbone network may include 21 convolutional layers, the sizes of all convolutional cores are 3*3, the number of convolutional channels increases continuously from 64 channels in the first layer to 1024 channels in the last layer, and the convolutional feature size is reduced through the largest pooling layer, where the convolutional feature size in the first layer is 512 × 512, and the convolutional feature size in the last layer is 1*1. As an example, the backbone network may be VGGNet, resNet.
In step S102, obtaining category information of the license plate from a deep feature map of the plurality of feature maps through the trained classification network, obtaining vertex information of the license plate from the deep feature map of the plurality of feature maps through the trained regression network, and adjusting the vertex information of the license plate to obtain adjusted vertex information;
in the embodiment of the invention, the trained classification network obtains the category of the image, such as the license plate or the background, from the deep feature map of the license plate image to be recognized, obtains the category information of the license plate according to the classification result, obtains the vertex information of the license plate from the deep feature map of the license plate image to be recognized through the trained regression network, and adjusts the vertex information of the license plate, so that the adjusted vertex information of the license plate is accurately obtained, and the positioning accuracy of the license plate region is improved.
In step S103, the corresponding license plate feature map is intercepted from the shallow feature map in the plurality of feature maps according to the adjusted vertex information, and the trained license plate recognition network is used to recognize the license plate feature map, so as to obtain the license plate number corresponding to the license plate image to be recognized.
In the embodiment of the invention, the vertex information of the labeling frame is obtained, the corresponding license plate feature map is intercepted from the shallow feature map in the feature maps according to the vertex information, and the trained license plate recognition network is used for recognizing the license plate feature map so as to obtain the license plate number corresponding to the license plate image to be recognized. Therefore, when the license plate feature map is identified, the shallow feature map of the license plate image to be identified is used, the license plate information is well reserved, the license plate is positioned and identified as far as possible, and the accuracy of license plate identification is improved.
In a preferred embodiment, when the trained license plate recognition network is used for recognizing the license plate feature map, the intercepted license plate feature map is horizontally corrected, and then the trained license plate recognition network is used for recognizing the license plate feature map, so that the interference of the multi-direction license plate on recognition is reduced, the robustness of the license plate feature input into the license plate recognition network is improved, and the accuracy of license plate recognition is further improved.
In a specific embodiment, the feature extraction network, the classification network, the regression network and the license plate recognition network form an end-to-end license plate recognition network, wherein the feature extraction network can be a backbone network, and the classification network and the regression network can form a detection head, so that end-to-end recognition of a license plate is realized, and finally a license plate number corresponding to a license plate image to be recognized is obtained.
Example two:
fig. 2 shows an implementation flow of a license plate detection method provided in the second embodiment of the present invention, and for convenience of description, only the relevant parts related to the second embodiment of the present invention are shown, which are detailed as follows:
in step S201, training a feature extraction network, a classification network, and a regression network to be trained using a training license plate image, and updating parameters of the feature extraction network, the classification network, and the regression network using a classification network loss function, a regression network loss function, and a vertex adjustment loss of the regression network;
the embodiment of the invention is suitable for license plate recognition equipment, such as a monitoring camera with a license plate recognition function and a license plate recognition computer, so that before a license plate image to be recognized is recognized, a feature extraction network, a classification network and a regression network are trained to obtain the trained feature extraction network, classification network and regression network.
In the training process, the loss is adjusted by using the classification network loss function, the regression network loss function and the peak of the regression networkAnd updating parameters of the feature extraction network, the classification network and the regression network to accelerate the training of the feature extraction network, the classification network and the regression network. In a preferred embodiment, when the classification network loss function, the regression network loss function and the vertex adjustment loss of the regression network are used to update the parameters of the feature extraction network, the classification network and the regression network, the first loss function is used
Updating parameters of the feature extraction network, the classification network and the regression network, wherein N represents the number of anchor frames for license plate matching,
the loss-balance parameter is expressed in terms of,
a function representing the loss of the classification network,
a function representing the loss of the regression network,
the vertex adjustment loss is expressed, and the accuracy of license plate vertex detection can be improved by utilizing the correlation among the license plate vertexes through the function, so that the accuracy of multi-direction license plate detection is improved.
It is further preferred that the first and second liquid crystal compositions,
，
，
the license plate vertexes can be accurately detected through the loss functions, so that the accuracy of multi-direction license plate detection is improved. Wherein c represents a class, p i c Representing the classification confidence of the object belonging to the class c corresponding to the anchor box i, c + A category of a license plate is represented,
denotes the first
Whether or not the anchor frame is connected with the first
The number of the real frames is matched,
and
respectively represent the predicted value and the true value,
，（
respectively represents four vertexes of the left upper part, the right lower part and the left lower part of the license plate,
to representSmooth L1 Loss (ref: ross B. Girshick. Fast R-CNN [ C)]. International Conference on Computer Vision, 2015: 1440-1448.），
The minimum horizontal surrounding rectangular frame formed by four vertexes of the license plate obtained by regression is shown,
a true horizontal rectangular bounding box representing the license plate,IoUindicating the degree of overlap of two horizontal rectangular boxes,
representing the center points of two rectangular frames
And
the Euclidean distance between the two electrodes,
represents the diagonal length of the smallest rectangular box surrounding the two rectangular boxes,
used for measuring the similarity of the aspect ratio of two rectangular frames,
the hyperparameters are balanced for loss. In particular, the amount of the solvent to be used,
，
，
，
。
in step S202, judging whether the training times reach the preset training times, if so, executing step S203, otherwise, skipping to step S201, and continuing training by using the feature extraction network, the classification network and the regression network to be trained by using the train license plate images until the preset training times are reached;
in step S203, a trained feature extraction network, classification network, and regression network are obtained.
In the embodiment of the invention, the training times are used for recording the times of training the feature extraction network, the classification network and the regression network, after the feature extraction network, the classification network and the regression network to be trained are trained by using the training license plate image, if the current training times reach the preset training times, the training of the feature extraction network, the classification network and the regression network can be considered to be finished, so that the trained feature extraction network, the classification network and the regression network are obtained, and if the training times do not reach the preset training times, the feature extraction network, the classification network and the regression network to be trained by using the training license plate image are required to be trained continuously until the preset training times are reached, and the training is finished.
Example three:
fig. 3 shows an implementation process of a license plate recognition method provided in the third embodiment of the present invention, and for convenience of description, only the parts related to the third embodiment of the present invention are shown, which are detailed as follows:
in step S301, a feature extraction network and a license plate recognition network to be trained are trained using a training license plate image, and the feature extraction network and the license plate recognition network are updated with parameters using a second loss function;
the embodiment of the invention is suitable for license plate recognition equipment, such as a monitoring camera with a license plate recognition function and a license plate recognition computer, so that a feature extraction network and a license plate recognition network are trained before a license plate image to be recognized is recognized, and the trained feature extraction network and the trained license plate recognition network are obtained. In the training process, parameter updating is carried out on the feature extraction network and the license plate recognition network by using a second loss function, so that training of the feature extraction network and the license plate recognition network is accelerated.
In a preferred embodiment, when a training license plate image is used for training a feature extraction network and a license plate recognition network to be trained, the feature extraction network to be trained is used for carrying out feature extraction on the training license plate image to obtain a plurality of feature maps of the training license plate image at different levels, a license plate feature map marked by a license plate marking frame is intercepted from a shallow feature map in the feature maps, and the license plate feature map to be trained is used for recognizing the license plate feature map to obtain a license plate number corresponding to the license plate image to be detected, so that license plate information is completely reserved through the shallow feature map, and the accuracy of license plate recognition is improved. Meanwhile, the license plate feature map marked on the license plate marking frame is directly used as input during training, so that the diversity of the input of the license plate recognition network can be improved, and the robustness of the license plate recognition network is improved.
In a preferred embodiment, a second loss function is utilized
Updating parameters of the feature extraction network and the license plate recognition network, wherein the parameters are updated
A set of training data is represented that is,
is represented by input features of
Output symbol sequence in the case of
The function can realize the automatic alignment of the input characteristics and the output characters under the condition of not marking license plate characters, thereby realizing the automatic identification of the license plate numbers with indefinite length.
In step S302, judging whether the training times reach the preset training times, if so, executing step S303, otherwise, jumping to step S301, and training the feature extraction network and the license plate recognition network to be trained by using the license plate images for training until the preset training times are reached;
in step S303, a trained feature extraction network and a license plate recognition network are obtained.
In the embodiment of the invention, the training times are used for recording the times of training the feature extraction network and the license plate recognition network, after the feature extraction network and the license plate recognition network to be trained are trained by using the training license plate image, if the current training times reach the preset training times, the training of the feature extraction network and the license plate recognition network can be considered to be completed, so that the trained feature extraction network and the trained license plate recognition network are obtained, and if the training times do not reach the preset training times, the feature extraction network and the license plate recognition network to be trained need to be trained by using the training license plate image continuously until the preset training times are reached, and the training is completed.
Example four:
fig. 4 shows an implementation process of a license plate recognition method provided by the fourth embodiment of the present invention, and for convenience of description, only parts related to the fourth embodiment of the present invention are shown, which are detailed as follows:
in step S401, training a feature extraction network, a classification network, a regression network, and a license plate recognition network to be trained using a training license plate image, updating parameters of the feature extraction network, the classification network, and the regression network using a first loss function, and updating parameters of the feature extraction network and the license plate recognition network using a second loss function;
the embodiment of the invention is suitable for license plate recognition equipment, such as a monitoring camera with a license plate recognition function and a license plate recognition computer, so that before a license plate image to be recognized is recognized, a feature extraction network, a classification network, a regression network and a license plate recognition network are trained to obtain the trained feature extraction network, classification network, regression network and license plate recognition network.
In the training process, parameter updating is carried out on the feature extraction network, the classification network and the regression network by utilizing the classification network loss function, the regression network loss function and the vertex adjustment loss of the regression network, so that the training of the feature extraction network, the classification network and the regression network is accelerated. In a preferred embodiment, when the classification network loss function, the regression network loss function and the vertex adjustment loss of the regression network are used to update the parameters of the feature extraction network, the classification network and the regression network, the first loss function is used
Updating parameters of the feature extraction network, the classification network and the regression network, wherein N represents the number of anchor frames for license plate matching,
the loss-balance parameter is expressed in terms of,
a function representing the loss of the classification network,
a function representing the loss of the regression network,
the vertex adjustment loss is expressed, and the accuracy of license plate vertex detection can be improved by utilizing the correlation among the license plate vertexes through the function, so that the accuracy of multi-direction license plate detection is improved.
In a preferred embodiment, when a training license plate image is used for training a feature extraction network and a license plate recognition network to be trained, the feature extraction network to be trained is used for carrying out feature extraction on the training license plate image to obtain a plurality of feature maps of the training license plate image at different levels, a license plate feature map marked by a license plate marking frame is intercepted from a shallow feature map in the feature maps, and the license plate feature map to be trained is used for recognizing the license plate feature map to obtain a license plate number corresponding to the license plate image to be detected, so that license plate information is completely reserved through the shallow feature map, and the accuracy of license plate recognition is improved. Meanwhile, the license plate feature map marked on the license plate marking frame is directly used as input during training, so that the diversity of the input of the license plate recognition network can be improved, and the robustness of the license plate recognition network is improved.
Further, in a preferred embodiment, a second loss function is utilized
Updating parameters of the feature extraction network and the license plate recognition network, wherein the parameters are updated
A set of training data is represented that is,
is expressed in the input features of
Output symbol sequence in the case of
The function can realize the automatic alignment of the input characteristics and the output characters under the condition of not marking license plate characters, thereby realizing the automatic identification function of the license plate numbers with indefinite length.
In step S402, judging whether the training times reach the preset training times, if so, executing step S403, otherwise, jumping to step S401, and continuing training by using the feature extraction network, the classification network, the regression network and the license plate recognition network to be trained by using the license plate images for training until the preset training times are reached;
in step S403, a trained feature extraction network, classification network, regression network, and license plate recognition network are obtained.
In the embodiment of the invention, the training times are used for recording the times of training the feature extraction network, the classification network, the regression network and the license plate recognition network, after the feature extraction network, the classification network, the regression network and the license plate recognition network to be trained are trained by using the training license plate image, if the current training times reach the preset training times, the training on the feature extraction network, the classification network, the regression network and the license plate recognition network can be considered to be finished, so that the trained feature extraction network, classification network, regression network and license plate recognition network are obtained, and if the training times do not reach the preset training times, the feature extraction network, the classification network, the regression network and the license plate recognition network to be trained by using the training license plate image need to be continuously trained until the preset training times are reached, and the training is finished.
Example five:
fig. 5 shows a structure of a license plate recognition device according to a fifth embodiment of the present invention, and for convenience of description, only parts related to the fifth embodiment of the present invention are shown, where the structures include:
the feature extraction unit 51 is configured to perform feature extraction on the license plate image to be recognized by using the trained feature extraction network to obtain a plurality of feature maps of the license plate image to be recognized at different levels;
the frame adjusting unit 52 is configured to obtain category information of the license plate from a deep feature map in the feature maps through the trained classification network, obtain vertex information of the license plate from the deep feature map in the feature maps through the trained regression network, and adjust the vertex information of the license plate to obtain adjusted vertex information;
and the license plate recognition unit 53 is configured to intercept a corresponding license plate feature map from a shallow feature map in the plurality of feature maps according to the adjusted vertex information, and recognize the license plate feature map by using a trained license plate recognition network to obtain a license plate number corresponding to the license plate image to be recognized.
In a preferred embodiment, the license plate recognition unit includes a license plate recognition subunit, configured to intercept a corresponding license plate feature map from a shallow feature map in the multiple feature maps according to the vertex information, perform horizontal correction on the license plate feature map, and recognize the license plate feature map by using a trained license plate recognition network.
In the embodiment of the present invention, each unit of the license plate recognition device may be implemented by a corresponding hardware or software unit, and each unit may be an independent software or hardware unit, or may be integrated into a software or hardware unit, which is not limited herein. The detailed implementation of each unit can refer to the description of the foregoing method embodiment, and is not repeated herein.
Example six:
fig. 6 shows a structure of a license plate recognition device according to a sixth embodiment of the present invention, and for convenience of description, only a part related to the embodiment of the present invention is shown.
The license plate recognition apparatus 6 of the embodiment of the present invention includes a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and operable on the processor 60. The processor 60 executes the computer program 62 to implement the steps of the license plate recognition method embodiments, such as the steps S101 to S103 shown in fig. 1. Alternatively, the processor 60, when executing the computer program 62, implements the functions of the units in the device embodiments described above, such as the functions of the units 51 to 53 shown in fig. 5.
The license plate recognition device provided by the embodiment of the invention can be a monitoring camera with a license plate recognition function and a license plate recognition computer. The steps of the license plate recognition method implemented by the processor 60 executing the computer program 62 in the license plate recognition device 6 can refer to the description of the foregoing method embodiments, and are not described herein again.
Example seven:
in an embodiment of the present invention, a computer-readable storage medium is provided, where a computer program is stored, and the computer program, when executed by a processor, implements the steps in the embodiment of the license plate recognition method described above, for example, steps S101 to S103 shown in fig. 1. Alternatively, the computer program, when executed by a processor, implements the functionality of the units in the device embodiments described above, such as the functionality of units 51 to 53 shown in fig. 5.
In the embodiment of the invention, a trained feature extraction network is adopted to extract features of a license plate image to be recognized to obtain a plurality of feature maps of the license plate image to be recognized at different levels, category information of the license plate is obtained from deep feature maps in the feature maps through the trained classification network, peak information of the license plate is obtained from the deep feature maps in the feature maps through the trained regression network, the peak information is adjusted to obtain the adjusted peak information, the corresponding license plate feature map is intercepted from the shallow feature maps in the feature maps according to the adjusted peak information, the license plate feature map is recognized through the trained license plate recognition network to obtain the license plate number corresponding to the license plate image to be recognized, so that the peak information of the license plate is obtained from the deep feature maps through the trained regression network, the peak information is adjusted, and the accuracy of multi-direction license plate recognition is improved.
The computer readable storage medium of the embodiments of the present invention may include any entity or device capable of carrying computer program code, a recording medium, for example, a ROM/RAM, a magnetic disk, an optical disk, a flash memory, or the like.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (10)
1. A license plate recognition method is characterized by comprising the following steps:
performing feature extraction on a license plate image to be recognized by adopting a trained feature extraction network to obtain a plurality of feature maps of the license plate image to be recognized at different levels;
acquiring category information of the license plate from a deep feature map in the feature maps through a trained classification network, acquiring vertex information of the license plate from the deep feature map in the feature maps through a trained regression network, and adjusting the vertex information to obtain the adjusted vertex information;
and intercepting a corresponding license plate feature map from a shallow feature map in the feature maps according to the adjusted vertex information, and identifying the license plate feature map by using a trained license plate identification network to obtain a license plate number corresponding to the license plate image to be identified.
2. The method of claim 1, wherein the step of extracting a corresponding license plate feature map from a shallow feature map of the feature maps according to the adjusted vertex information, and recognizing the license plate feature map using a trained license plate recognition network comprises:
and intercepting a corresponding license plate feature map from a shallow feature map in the feature maps according to the vertex information, horizontally correcting the license plate feature map, and identifying the license plate feature map by using a trained license plate identification network.
3. The method of claim 1, wherein before the step of extracting the features of the license plate image to be recognized by using the trained feature extraction network, the method further comprises:
training the feature extraction network, the classification network and the regression network to be trained by using a license plate image for training, and updating parameters of the feature extraction network, the classification network and the regression network by using a classification network loss function, a regression network loss function and vertex adjustment loss of the regression network;
and judging whether the training times reach preset training times, if so, finishing the training, otherwise, skipping to the step of training the feature extraction network, the classification network and the regression network to be trained by using the license plate image for training until the preset training times are reached.
4. The method of claim 3, wherein the step of updating the parameters of the feature extraction network, the classification network, and the regression network with a classification network loss function, a regression network loss function, and a vertex adjustment loss of the regression network comprises:
using a first loss function
Updating parameters of the feature extraction network, the classification network and the regression network, wherein N represents the number of anchor frames for license plate matching,
the loss-balance parameter is expressed in terms of,
a function representing a loss of the classification network,
representing a function of loss of said regression network,
representing the vertex adjustment penalty.
5. The method of claim 1, wherein before the step of extracting the features of the license plate image to be recognized by using the trained feature extraction network, the method further comprises:
training the feature extraction network and the license plate recognition network to be trained by using a license plate image for training, and updating parameters of the feature extraction network and the license plate recognition network by using a second loss function;
judging whether the training times reach preset training times, if so, finishing the training, otherwise, skipping to the step of training the feature extraction network and the license plate recognition network to be trained by using the license plate images for training until the preset training times are reached;
the step of training the feature extraction network and the license plate recognition network to be trained by using the license plate image for training includes:
performing feature extraction on the license plate image for training by adopting the feature extraction network to be trained to obtain a plurality of feature maps of the license plate image for training at different levels;
and intercepting the license plate feature map marked by the license plate marking frame from the shallow feature map in the feature maps, and identifying the license plate feature map by using the license plate identification network to be trained to obtain the license plate number corresponding to the license plate image to be identified.
6. The method of claim 5, wherein the second loss function is
Wherein, the
A set of training data is represented that is,
is expressed in the input features of
Output symbol sequence in the case of
All paths of (1).
7. A license plate recognition device, the device comprising:
the system comprises a feature extraction unit, a feature extraction unit and a feature extraction unit, wherein the feature extraction unit is used for extracting features of a license plate image to be recognized by adopting a trained feature extraction network to obtain a plurality of feature maps of the license plate image to be recognized at different levels;
the frame adjusting unit is used for acquiring the category information of the license plate from a deep feature map in the feature maps through a trained classification network, acquiring the vertex information of the license plate from the deep feature map in the feature maps through a trained regression network, and adjusting the vertex information to obtain the adjusted vertex information; and
and the license plate recognition unit is used for intercepting corresponding license plate feature maps from shallow feature maps in the feature maps according to the adjusted vertex information, and recognizing the license plate feature maps by using a trained license plate recognition network so as to obtain the license plate number corresponding to the license plate image to be recognized.
8. The apparatus of claim 7, wherein the license plate recognition unit comprises:
and the license plate recognition subunit is used for intercepting the corresponding license plate feature map from the shallow feature map in the feature maps according to the vertex information, horizontally correcting the license plate feature map, and recognizing the license plate feature map by using a trained license plate recognition network.
9. A license plate detection apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the method of any one of claims 1 to 6 are implemented when the computer program is executed by the processor.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of a method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211309201.0A CN115376119B (en) | 2022-10-25 | 2022-10-25 | License plate recognition method and device, license plate recognition equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211309201.0A CN115376119B (en) | 2022-10-25 | 2022-10-25 | License plate recognition method and device, license plate recognition equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115376119A true CN115376119A (en) | 2022-11-22 |
| CN115376119B CN115376119B (en) | 2023-04-14 |
Family
ID=84073853
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211309201.0A Active CN115376119B (en) | 2022-10-25 | 2022-10-25 | License plate recognition method and device, license plate recognition equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115376119B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109344825A (en) * | 2018-09-14 | 2019-02-15 | 广州麦仑信息科技有限公司 | A kind of licence plate recognition method based on convolutional neural networks |
| CN112085018A (en) * | 2020-08-27 | 2020-12-15 | 南京佑驾科技有限公司 | License plate recognition system based on neural network |
| CN112380913A (en) * | 2020-10-21 | 2021-02-19 | 苏州怡林城信息科技有限公司 | License plate detection and identification method based on combination of dynamic adjustment and local feature vector |
| CN113392837A (en) * | 2021-07-09 | 2021-09-14 | 超级视线科技有限公司 | License plate recognition method and device based on deep learning |
| CN113850253A (en) * | 2021-08-24 | 2021-12-28 | 中国船舶重工集团公司第七0九研究所 | License plate recognition method and system based on deep learning, server and medium |
| CN114155523A (en) * | 2021-12-13 | 2022-03-08 | 大连海事大学 | End-to-end efficient and accurate license plate detection and identification method |
| CN114998883A (en) * | 2022-05-30 | 2022-09-02 | 阿波罗智联(北京)科技有限公司 | License plate recognition method and device, electronic equipment and intelligent transportation equipment |
-
2022
- 2022-10-25 CN CN202211309201.0A patent/CN115376119B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109344825A (en) * | 2018-09-14 | 2019-02-15 | 广州麦仑信息科技有限公司 | A kind of licence plate recognition method based on convolutional neural networks |
| CN112085018A (en) * | 2020-08-27 | 2020-12-15 | 南京佑驾科技有限公司 | License plate recognition system based on neural network |
| CN112380913A (en) * | 2020-10-21 | 2021-02-19 | 苏州怡林城信息科技有限公司 | License plate detection and identification method based on combination of dynamic adjustment and local feature vector |
| CN113392837A (en) * | 2021-07-09 | 2021-09-14 | 超级视线科技有限公司 | License plate recognition method and device based on deep learning |
| CN113850253A (en) * | 2021-08-24 | 2021-12-28 | 中国船舶重工集团公司第七0九研究所 | License plate recognition method and system based on deep learning, server and medium |
| CN114155523A (en) * | 2021-12-13 | 2022-03-08 | 大连海事大学 | End-to-end efficient and accurate license plate detection and identification method |
| CN114998883A (en) * | 2022-05-30 | 2022-09-02 | 阿波罗智联(北京)科技有限公司 | License plate recognition method and device, electronic equipment and intelligent transportation equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115376119B (en) | 2023-04-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110580723B (en) | Method for carrying out accurate positioning by utilizing deep learning and computer vision | |
| CN113989450B (en) | Image processing method, device, electronic equipment and medium | |
| CN113822278B (en) | License plate recognition method for unlimited scene | |
| US11367195B2 (en) | Image segmentation method, image segmentation apparatus, image segmentation device | |
| CN110021033A (en) | A kind of method for tracking target based on the twin network of pyramid | |
| CN111091023B (en) | Vehicle detection method and device and electronic equipment | |
| WO2021155558A1 (en) | Road marking identification method, map generation method and related product | |
| CN110689000B (en) | Vehicle license plate recognition method based on license plate sample generated in complex environment | |
| CN110781962B (en) | Target detection method based on lightweight convolutional neural network | |
| Wu et al. | Psdet: Efficient and universal parking slot detection | |
| CN111126205A (en) | Optical remote sensing image airplane target detection method based on rotary positioning network | |
| CN112001298B (en) | Pedestrian detection method, device, electronic equipment and storage medium | |
| CN110544268B (en) | Multi-target tracking method based on structured light and SiamMask network | |
| CN111784737A (en) | Automatic target tracking method and system based on unmanned aerial vehicle platform | |
| CN112215190A (en) | Illegal building detection method based on YOLOV4 model | |
| CN114782770A (en) | License plate detection and recognition method and system based on deep learning | |
| CN113850136A (en) | Yolov5 and BCNN-based vehicle orientation identification method and system | |
| WO2024093641A1 (en) | Multi-modal-fused method and apparatus for recognizing high-definition map element, and device and medium | |
| CN105631849B (en) | The change detecting method and device of target polygon | |
| CN113673515A (en) | Computer vision target detection algorithm | |
| CN111062393B (en) | Natural scene Chinese character segmentation method based on spectral clustering | |
| CN115546221B (en) | Reinforcing steel bar counting method, device, equipment and storage medium | |
| CN115376119B (en) | License plate recognition method and device, license plate recognition equipment and storage medium | |
| CN115690770A (en) | License plate recognition method based on space attention characteristics in non-limited scene | |
| CN114332814A (en) | Parking frame identification method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |