CN111368816A - License plate recognition method, system, device and computer readable storage medium - Google Patents

License plate recognition method, system, device and computer readable storage medium Download PDF

Info

Publication number
CN111368816A
CN111368816A CN202010123472.1A CN202010123472A CN111368816A CN 111368816 A CN111368816 A CN 111368816A CN 202010123472 A CN202010123472 A CN 202010123472A CN 111368816 A CN111368816 A CN 111368816A
Authority
CN
China
Prior art keywords
license plate
detected
information
picture
result
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010123472.1A
Other languages
Chinese (zh)
Inventor
申燚
欧阳一村
罗英群
吕令广
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE ICT Technologies Co Ltd
Original Assignee
ZTE ICT Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE ICT Technologies Co Ltd filed Critical ZTE ICT Technologies Co Ltd
Priority to CN202010123472.1A priority Critical patent/CN111368816A/en
Publication of CN111368816A publication Critical patent/CN111368816A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/24Aligning, centring, orientation detection or correction of the image
    • G06V10/245Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/285Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/22Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • G06V10/23Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition based on positionally close patterns or neighbourhood relationships
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/60Type of objects
    • G06V20/62Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625License plates

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Character Discrimination (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention provides a license plate recognition method, a license plate recognition system, a license plate recognition device and a computer readable storage medium, wherein the license plate recognition method comprises the following steps: acquiring information of a picture to be detected; positioning the picture information to be detected to obtain the position of the license plate; identifying license plate characters in the license plate position to acquire license plate information; and outputting the license plate information. The license plate recognition system, the license plate recognition device and the computer-readable storage medium are respectively used for realizing the license plate recognition method. The invention improves the precision of license plate recognition in the picture to be detected and improves the accuracy and efficiency of license plate information recognition.

Description

License plate recognition method, system, device and computer readable storage medium
Technical Field
The invention relates to the technical field of vehicle license plate recognition, in particular to a license plate recognition method, a license plate recognition system, a license plate recognition device and a computer readable storage medium.
Background
The license plate of the vehicle is used as the unique identification of the vehicle and becomes an important solution for effectively carrying out traffic management. At present, a Chinese license plate recognition module (HyperLPR) is usually adopted to realize end-to-end license plate recognition, but under the actual scene test, the precision can not meet the requirement, and further optimization is needed.
Moreover, any discussion of the prior art throughout the specification is not an admission that the prior art is necessarily known to a person of ordinary skill in the art, and any discussion of the prior art throughout the specification is not an admission that the prior art is necessarily widely known or forms part of common general knowledge in the field.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art or the related art.
In view of the above, a first objective of the present invention is to provide a license plate recognition method.
The second object of the present invention is to provide a license plate recognition system.
The third object of the present invention is to provide a license plate recognition device.
A fourth object of the present invention is to provide a computer-readable storage medium.
The first object of the invention is achieved, and the embodiment of the invention provides a license plate identification method, which comprises the following steps: acquiring information of a picture to be detected; positioning the picture information to be detected to obtain the position of the license plate; identifying license plate characters in the license plate position to acquire license plate information; and outputting the license plate information.
According to the technical scheme, the accuracy of recognizing the license plate information in the license plate can be improved by acquiring the information of the picture to be detected and accurately positioning the position of the license plate in the picture to be detected. After the license plate position is obtained, the license plate characters in the license plate position are identified, and because the license plate characters of each vehicle are different, the license plate information can be obtained by reading the license plate characters, so that the output license plate information is more accurate, and the accuracy of obtaining the license plate information is improved.
In addition, the technical solution provided by the above embodiment of the present invention may further have the following additional technical features:
in the above technical solution, the positioning the information of the picture to be detected to obtain the license plate position specifically includes the following steps: carrying out primary positioning processing on the picture information to be detected to obtain a primary positioning result; according to the primary positioning result, performing secondary positioning processing on the picture information to be detected to obtain a secondary positioning result; and according to the secondary positioning result, correcting the information of the picture to be detected to obtain a correction result.
In the technical scheme, the acquired picture information is subjected to primary positioning processing to quickly perform rough positioning on the license plate, and a primary positioning result is acquired to improve the speed of positioning the license plate. Further, secondary positioning processing is carried out on the picture information, and the license plate is accurately positioned, so that the position of the license plate in the picture in the obtained secondary positioning result is more accurate. Therefore, the technical scheme improves the accuracy of positioning the license plate position. In addition, according to the technical scheme, the image information to be detected is further corrected according to the secondary positioning result, and the license plate which is roughly positioned is corrected to obtain more accurate license plate information, so that the accuracy of obtaining the license plate information is improved. By carrying out positioning operation of two levels on the license plate, the technical scheme can obtain more accurate license plate information on the premise of quickly positioning the license plate.
In any of the above technical solutions, after the step of performing correction processing on the image information to be detected according to the secondary positioning result to obtain a correction result is executed, the license plate recognition method further includes the following steps: evaluating the availability degree of the correction result to obtain an availability degree evaluation result; and acquiring the position of the license plate according to the usability degree evaluation result.
Because in the process of carrying out secondary positioning treatment on the picture to be detected, shearing is used when the picture to be detected is corrected, part of the license plate can be sheared, so that the license plate is damaged and is not complete any more. And evaluating the usability of the corrected picture to be detected, wherein the evaluation is mainly used for evaluating the integrity of the picture to be detected. When the evaluation result is available, the next step can be carried out to obtain the position of the license plate; and when the evaluation result is unavailable, the license plate position is abandoned, so that the license plate position is obtained according to the evaluation result of the availability degree, and the license plate position information in the picture to be detected can be effectively and quickly obtained.
In any of the above technical solutions, the obtaining of the license plate position according to the usability degree evaluation result specifically includes the following steps: judging that the evaluation result of the availability degree is greater than the evaluation threshold of the availability degree; and acquiring the position of the license plate according to the correction result.
The integrity degree of the license plate position in the picture to be detected can be used as an evaluation index of the availability degree of the picture to be detected, an availability degree evaluation threshold value of the picture to be detected can be set according to the integrity degree of the license plate position in the picture to be detected, and when the availability degree evaluation result is judged to be greater than the availability degree evaluation threshold value, the picture to be detected is judged to be available, and a correction step is carried out; and when the usability degree evaluation result is judged to be less than or equal to the usability degree evaluation threshold, the picture to be detected is judged to be unusable, and the correction step is not carried out any more, so that only the picture to be detected with the usability degree evaluation result greater than the usability degree evaluation threshold is corrected, the efficiency and the accuracy of obtaining the license plate position in the picture to be detected are improved, and the program is saved.
In any of the above technical solutions, the obtaining of the license plate position according to the usability degree evaluation result specifically includes the following steps: determining that the usability evaluation result is less than or equal to a usability evaluation threshold; carrying out three-level positioning processing and background amplification correction on the picture information to be detected to obtain a correction result; and acquiring the position of the license plate according to the correction result.
According to the technical scheme, according to the integrity of the position of the license plate in the picture to be detected, when the availability of the picture to be detected is judged to be less than or equal to the availability by virtue of the threshold, the current license plate is in an unclear state, then the picture to be detected is subjected to three-stage positioning treatment, namely, the picture to be detected is subjected to repositioning, and then the picture to be detected is subjected to background amplification correction so as to appropriately correct the missing part of the picture to be detected, wherein the background amplification correction is to further expand other parts except the surrounding part of the license plate in the picture so as to avoid missing characters of the license plate. The correction is an operation of aligning the shifted license plate position. And obtaining a correction result after correction so that the correction result can meet the condition of obtaining the license plate position information, thereby further more accurately obtaining the license plate position.
In any of the above technical solutions, the obtaining of the license plate position according to the correction result specifically includes the following steps: according to the correction result, carrying out secondary positioning processing on the picture information to be detected again, and obtaining a secondary positioning result again; correcting the picture information to be detected again according to the secondary positioning result, and acquiring a correction result again; and acquiring the position of the license plate according to the correction result.
In the technical scheme, because the corrected picture to be detected is very close to the required license plate, after appropriate correction, secondary positioning treatment, namely accurate positioning, can be carried out. If the rough positioning is continuously carried out, the detection omission phenomenon can occur because only the pictures to be detected which pass through all the primary positioning can enter the step of accurate positioning. And the picture to be detected is corrected again, so that the accuracy of obtaining the license plate information is further improved.
In the above technical solution, the obtaining of the license plate position according to the correction result specifically includes the following steps: according to the correction result, carrying out primary positioning processing on the picture information to be detected again, and obtaining a primary positioning result again; according to the first-stage positioning result, carrying out second-stage positioning processing on the picture information to be detected again, and obtaining a second-stage positioning result again; correcting the picture information to be detected again according to the secondary positioning result, and acquiring a correction result again; and acquiring the position of the license plate according to the correction result.
In the technical scheme, after the correction result is obtained, the primary positioning processing and the secondary positioning processing can be performed on the detected picture information again, the picture information is corrected, and then accurate license plate information is obtained, so that the accuracy of obtaining the license plate information is improved. In addition, the technical scheme that the result after the three-level positioning processing and the background amplification correction is directly subjected to the first-level positioning processing can be realized by adopting a simpler program. The calculation model for realizing the primary positioning processing and the secondary positioning processing is called as a whole, so that a program needs to be written, and the calculation model is easier to build.
The second object of the present invention is achieved, and an embodiment of the present invention provides a license plate recognition system, which implements the license plate recognition method according to any embodiment of the present invention, and the license plate recognition system includes: the information acquisition unit is suitable for acquiring the information of the picture to be detected; the license plate recognition unit is in communication connection with the information acquisition unit and is suitable for positioning the picture information to be detected and acquiring the position of a license plate; the character recognition unit is in communication connection with the license plate recognition unit and is suitable for recognizing license plate characters in a license plate position and acquiring license plate information; and the information output unit is in communication connection with the character recognition unit and is suitable for outputting license plate information.
According to the technical scheme, the image including the vehicle license plate can be obtained through the information obtaining unit, the obtained image information can be positioned through the license plate recognition unit, and the position of the license plate in the image is accurately positioned. Furthermore, the technical scheme is that the license plate characters in the license plate position are identified through the character identification unit. And finally, the technical scheme adopts an information output unit to output the acquired license plate information. Therefore, the technical scheme enables the output license plate information to be more accurate, and improves the accuracy of obtaining the license plate information.
In the above technical solution, the license plate recognition unit includes: the first license plate identification unit is suitable for carrying out primary positioning processing, secondary positioning processing and correction processing on the picture information to be detected; and the second license plate identification unit is in communication connection with the first license plate identification unit and is suitable for performing three-level positioning processing and background amplification correction on the picture information to be detected.
In the technical scheme, the first license plate recognition unit and the second license plate recognition unit are arranged in the license plate recognition system, so that the position of a license plate can be effectively detected, and the detection accuracy is improved. The first license plate identification unit is used for carrying out primary positioning processing, secondary positioning result and correction processing on the picture information to be detected. The second license plate identification unit is used for being in communication connection with the first license plate identification unit and is suitable for carrying out three-level positioning processing and background amplification correction on the picture information to be detected. According to the technical scheme, the three-level positioning processing is carried out on the detected picture information so as to further position the license plate position, and the accuracy of positioning the license plate position is improved. In addition, the technical scheme further expands other parts except the surrounding part of the license plate in the picture through background expansion correction so as to avoid missing characters of the license plate.
To achieve the third object of the present invention, an embodiment of the present invention provides a license plate recognition device, including: a memory storing a computer program; a processor executing a computer program; when executing the computer program, the processor implements the steps of the license plate recognition method according to any embodiment of the present invention.
The license plate recognition device provided by the embodiment of the invention realizes the steps of the license plate recognition method according to any embodiment of the invention, so that the license plate recognition device has all the beneficial effects of the license plate recognition method according to any embodiment of the invention.
To achieve the fourth object of the present invention, an embodiment of the present invention provides a computer-readable storage medium including: the computer-readable storage medium stores a computer program which, when executed, implements the steps of the license plate recognition method according to any one of the embodiments of the present invention.
The readable storage medium provided in the embodiments of the present invention implements the steps of the license plate recognition method according to any one of the embodiments of the present invention, and thus has all the advantageous effects of the license plate recognition method according to any one of the embodiments of the present invention.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 shows a first flowchart of a license plate recognition method according to an embodiment of the present invention;
FIG. 2 is a second flowchart of a license plate recognition method according to an embodiment of the invention;
FIG. 3 is a third flowchart of a license plate recognition method according to an embodiment of the invention;
FIG. 4 is a fourth flowchart illustrating a license plate recognition method according to an embodiment of the invention;
FIG. 5 is a fifth flowchart illustrating a license plate recognition method according to an embodiment of the invention;
FIG. 6 is a sixth flowchart illustrating a license plate recognition method according to an embodiment of the invention;
FIG. 7 is a seventh flowchart illustrating a license plate recognition method according to an embodiment of the invention;
FIG. 8 is a schematic diagram showing the components of a license plate recognition unit of a license plate recognition system according to an embodiment of the present invention;
fig. 9 is a schematic diagram illustrating a system configuration of a license plate recognition device according to an embodiment of the present invention.
Wherein, the correspondence between the reference numbers and the part names in fig. 7 to 9 is:
100: license plate recognition system, 110: information acquisition unit, 120: license plate recognition unit, 122: first license plate recognition unit, 124: second card recognition unit, 130: character recognition unit, 140: information output unit, 200: license plate recognition device, 210: memory, 220: a processor.
Detailed Description
So that the manner in which the above recited objects, features and advantages of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to the embodiments thereof which are illustrated in the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
A license plate recognition method, a license plate recognition system, a license plate recognition apparatus, and a computer-readable storage medium according to some embodiments of the present invention will be described with reference to fig. 1 to 9.
With the continuous increase of the number of automobiles in various countries around the world, the urban traffic conditions are more and more emphasized by people. How to effectively conduct traffic management is becoming a focus of increasing concern for government departments of various countries. The vehicle license plate is used as the unique identity of the vehicle, so that the research of a license plate recognition system becomes an important solution for improving traffic management. The license plate recognition system is an analysis method for processing images of images containing license plates shot by a camera, extracts license plate regions, and further performs character segmentation and recognition on the license plate regions to finally finish recognition of the license plates. The license plate recognition technology is an important link of traffic management automation, plays an important role in traffic monitoring and control, and can be applied to expressways, parking lot management systems, traffic place distribution and control management systems and expressway overspeed automatic supervision systems. One such large integrated License Plate data set (CCPD) is introduced in the End-to-End License Plate detection and identification (handware End-to-End License Plate detection and Recognition: image data and base) of large data sets. All pictures are taken manually by the staff of the roadside parking management company and carefully noted. CCPD is the largest public license plate data set to date, with over 250k unique car images, and only one providing vertex position annotations. With CCPD, a new neural network model is proposed that can quickly and accurately predict bounding boxes and identify the corresponding license plate number. Through comparison experiments, the model is proved to be superior to the current target detection and identification method in precision and speed. However, most of data sets processed by the model are scenes relatively close to each other, and the license plates are blue plates, so that only a single license plate can be recognized.
Example 1:
as shown in fig. 1, an embodiment of the present invention provides a license plate recognition method, including:
step S102: acquiring information of a picture to be detected;
step S104: positioning the picture information to be detected to obtain the position of the license plate;
step S106: identifying license plate characters in the license plate position to acquire license plate information;
step S108: and outputting the license plate information.
The license plate characters of the embodiment comprise at least one of Chinese characters, English letters and Arabic numerals or a combination thereof.
License plate identification is one of important components in modern intelligent traffic systems, and is very widely applied. The method is based on technologies such as digital image processing, mode recognition and computer vision, and analyzes vehicle images or video sequences shot by a camera to obtain a unique license plate number of each vehicle, so that the recognition process is completed. The functions of parking lot charging management, traffic flow control index measurement, vehicle positioning, automobile anti-theft, highway overspeed automatic supervision, red light running electronic police, highway toll stations and the like can be realized through some subsequent processing means. The method has practical significance for maintaining traffic safety and urban public security, preventing traffic jam and realizing automatic traffic management.
In order to improve the accuracy of license plate recognition, in the embodiment, the picture to be detected can be obtained by shooting, the picture to be detected including the license plate of the vehicle can be obtained, and the picture to be detected generally includes information of the vehicle, the surrounding background of the vehicle and the like, so that the information of the picture to be detected is formed. Furthermore, the acquired picture information is subjected to license plate positioning processing, so that the position of the license plate in the picture to be detected can be accurately positioned by acquiring the picture information to be detected of the picture to be detected, and the accuracy of recognizing the license plate information in the license plate can be improved. After the license plate position is obtained, the license plate characters in the license plate position are identified, and because the license plate characters of each vehicle are different, the license plate information can be obtained by reading the license plate characters, so that the output license plate information is more accurate, and the accuracy of obtaining the license plate information is improved.
In this embodiment, a chinese license plate recognition library may be used to recognize license plate information, for example, a hyper lpr, which is an open-source, deep learning-based, high-performance chinese license plate recognition library, light in weight, with a total code amount not exceeding 1k lines, supporting PHP, C/C + +, Python languages, and may be applied to platforms such as Windows/Mac/Linux/Android/IOS. The HyperLPR is based on end-to-end license plate recognition, character segmentation is not needed, the recognition rate is high, the zero error rate reaches 95.2%, and the license plate recognition rate after successful positioning can also reach 97.4%. The currently identifiable license plates are single-row blue plates, single-row yellow plates, new energy license plates, white police license plates, Shichan/Gangao license plates, coach license plates, armed police license plates and the like. However, the recognition accuracy of the HyperLPR cannot meet the requirement, so that the recognition accuracy of the picture to be detected can be improved by combining the HyperLPR with the network model. For example, MobileNet-SSD is a lightweight network architecture framework model, where MobileNet is a lightweight deep network model mainly proposed for being applied to a mobile terminal, and mainly uses Deep Separable Convolution (DSC) to perform decomposition calculation on a standard Convolution kernel, so as to reduce the amount of calculation, and MobileNet based on a target detection algorithm SSD (SSD) is a MobileNet-SSD, which can implement a target detection function, and is applicable to a general computer optic nerve network of a mobile device, such as functions of vehicle license plate detection, pedestrian detection, and the like.
In this embodiment, the process of implementing step 106 and step 108 by the HyperLPR is as follows:
the CNN-based sequence model ocr _ plate _ all _ w _ rnn _2.h5 is used to perform character recognition on the picture to be detected in step 106. For example, when a blurred picture to be detected containing the license plate number of cinnamon N15851 is subjected to license plate recognition, step 106 outputs (14, 84) tensors to the input picture to be detected, wherein 14 represents 14 parts of the license plate, each row represents the probability that the predicted character is 83 characters, the one with the highest probability is selected, and then a vector with the dimension of 14 is obtained. The vector with the dimension of 14 comprises 7 license plate numbers and 7 backgrounds, wherein the background represents 83, and the rest characters are coded into 0-82. The license plate information finally output by the HyperLPR is as follows: gui N15851.
In this embodiment, the combined part of the HyperLPR and the MobileNet-SSD is applied to the step S104, so that the accuracy of license plate recognition in the picture to be detected can be improved.
Example 2:
as shown in fig. 2, the present embodiment provides a license plate recognition method, which includes the following technical features in addition to the technical features of the foregoing embodiment.
The method for positioning the picture information to be detected and acquiring the license plate position specifically comprises the following steps:
step S1042: carrying out primary positioning processing on the picture information to be detected to obtain a primary positioning result;
step S1044: according to the primary positioning result, performing secondary positioning processing on the picture information to be detected to obtain a secondary positioning result;
step S1046: and according to the secondary positioning result, correcting the detected picture information to obtain a correction result.
The steps of license plate recognition are generally divided into license plate detection and character recognition. Firstly, detecting the position of a license plate in a picture, and then carrying out character recognition on the detected license plate. In the process of Character Recognition, two methods are mainly used, one is conventional Optical Character Recognition (OCR), and the other is a trained convolutional neural network processing output result. Among them, OCR refers to a process in which an electronic device (e.g., a scanner or a digital camera) checks a character printed on paper, determines its shape by detecting dark and light patterns, and then translates the shape into a computer word by a character recognition method. The method is characterized in that characters in a paper document are converted into an image file with a black-white dot matrix in an optical mode aiming at print characters, and the characters in the image are converted into a text format through recognition software for further editing and processing by word processing software.
The picture to be detected is usually obtained by shooting, because the shooting angles adopted during shooting are different, the distance between a shot object and a photographer is different, and the shot object and other objects with similar or same shapes or colors are shot together, so that the picture to be detected enters factors such as the picture to be detected together, the license plate position is different in the shot picture to be detected, for example, some license plate positions are obvious in the picture and are easy to find, some license plate positions can be easily confused with other picture information in the picture and are not easy to find, therefore, the information in the picture to be detected needs to be firstly subjected to primary positioning processing, so as to preliminarily judge and identify the license plate position.
In the related technology of license plate recognition, a detection model in Computer Assisted Strain screening and Development Engineering (hereinafter, referred to as "CASCADE") in HyperLPR is usually adopted to perform primary positioning processing on a license plate in a picture, that is, to roughly position the picture to be detected. The CASCADE detection model can improve the detection speed and the detection precision of the picture to be detected. The CASCADE detection model is provided with a classifier and is used for analyzing and identifying information in a picture to be detected, and in order to improve the detection speed and precision, the final CASCADE classification system can be obtained only by cascading a plurality of strong classifiers. In a cascade classification system obtained by cascading a plurality of strong classifiers, each input picture to be detected sequentially passes through each strong classifier in sequence, wherein the strong classifiers which pass through the picture to be detected and are positioned in front are relatively simple, and the weak classifiers contained in the classifiers are relatively few, so that the omission of some pictures to be detected can be avoided, and more pictures to be detected can pass through the strong classifiers successively. However, the strong classifiers with the later positions through the picture to be detected are complex step by step. That is, only the image to be detected after the preceding strong classification detection can be sent to the following strong classifier for detection, so as to improve the detection efficiency of the image to be detected. In the cascade classification system, most unqualified pictures can be filtered out by comparing the front classifiers, so that only the picture area detected by all the strong classifiers is the effective area, the picture to be detected is subjected to primary positioning treatment, and a primary positioning result is obtained, so that the subsequent efficiency of further detection and identification can be ensured, and the accuracy of detection and identification of the picture to be detected can be improved.
The primary positioning result obtained from the picture to be detected cannot guarantee smooth subsequent detection and identification, for example, the position of the license plate in the roughly positioned picture to be detected is inclined, the color of the license plate is closer to the surrounding colors, and the like. Therefore, secondary positioning processing is required to achieve the purpose of accurately positioning the picture to be detected, that is, after the secondary positioning processing, the position of the license plate in the picture to be detected is accurately found, so that the information in the license plate can be clearly identified.
After the image to be detected is subjected to secondary positioning processing, if the position of the license plate in the image to be detected has factors such as a large relative inclination angle, excessive background colors and the like, the image to be detected needs to be corrected, so that the definition of the image to be detected is further improved. For example, the periphery of the license plate in the picture to be detected can be cut to eliminate the recognition influence caused by the surrounding background of the license plate; the position of the inclined picture to be detected can be corrected, so that the license plate is positioned in the picture to be detected. When the inclined license plate is corrected, Hough transformation can be used for correction. Or performing boundary regression on the picture to be detected by utilizing a left and right boundary regression model, further cutting a peripheral background, and further enhancing the effect of accurate positioning. The final purpose is to improve the definition of the license plate and facilitate the subsequent identification of the license plate information.
Example 3:
as shown in fig. 3, the present embodiment provides a license plate recognition method, which includes the following technical features in addition to the technical features of the above embodiments.
After the step of correcting the information of the picture to be detected according to the secondary positioning result and acquiring the correction result is executed, the license plate recognition method further comprises the following steps:
step S1048: evaluating the availability degree of the correction result to obtain an availability degree evaluation result;
step S1050: and obtaining the position of the license plate according to the available degree evaluation result.
Because in the process of carrying out secondary positioning treatment on the picture to be detected, shearing is used when the picture to be detected is corrected, part of the license plate can be sheared, so that the license plate is damaged and is not complete any more. The method is characterized in that related modules or programs are arranged in the HyperLPR, so that the usability degree of the corrected picture to be detected is evaluated, and the method is mainly used for evaluating the integrity degree of the picture to be detected. When the evaluation result is available, the next step can be carried out to obtain the position of the license plate; and when the evaluation result is unavailable, the license plate position is abandoned, so that the HyperLPR acquires the license plate position according to the availability evaluation result, and can effectively and quickly acquire the license plate position information in the picture to be detected.
Example 4:
as shown in fig. 4, the present embodiment provides a license plate recognition method, which includes the following technical features in addition to the technical features of the foregoing embodiment.
According to the usability degree evaluation result, the license plate position obtaining method specifically comprises the following steps:
step S202: judging that the evaluation result of the availability degree is greater than the evaluation threshold of the availability degree;
step S204: and acquiring the position of the license plate according to the correction result.
The integrity of the license plate position in the picture to be detected can be used as an evaluation index of the availability of the picture to be detected, and an availability evaluation threshold of the picture to be detected can be set according to the integrity of the license plate position in the picture to be detected, for example, the availability evaluation threshold can be 0.7. When the evaluation result of the usability degree is judged to be larger than 0.7, the picture to be detected is judged to be usable, and a correction step is carried out; and when the evaluation result of the availability degree is judged to be less than or equal to 0.7, the picture to be detected is judged to be unavailable, and the correction step is not carried out any more, so that only the picture to be detected with the evaluation result of the availability degree greater than the evaluation threshold of the availability degree is corrected, the efficiency and the accuracy of obtaining the license plate position in the picture to be detected are improved, and the program is saved.
Example 5:
as shown in fig. 5, the present embodiment provides a license plate recognition method, which includes the following technical features in addition to the technical features of the foregoing embodiment.
According to the usability degree evaluation result, the license plate position obtaining method specifically comprises the following steps:
step S302: determining that the usability evaluation result is less than or equal to a usability evaluation threshold;
step S304: carrying out three-level positioning processing and background amplification correction on the picture information to be detected to obtain a correction result;
step S306: and acquiring the position of the license plate according to the correction result.
According to the integrity of the position of the license plate in the picture to be detected, when the availability of the picture to be detected is judged to be less than or equal to the availability by virtue of the threshold, the current license plate is in an unclear state, then the picture to be detected is subjected to three-stage positioning processing, namely, the picture to be detected is subjected to repositioning, and then the picture to be detected is subjected to background amplification correction so as to appropriately correct the missing part of the picture to be detected, wherein the background amplification correction is to further expand other parts except the surrounding part of the license plate in the picture so as to avoid missing characters of the license plate. The correction is an operation of aligning the shifted license plate position. And obtaining a correction result after correction so that the correction result can meet the condition of obtaining the license plate position information, thereby further more accurately obtaining the license plate position.
In this embodiment, when the HyperLPR is adopted to perform primary positioning processing and secondary positioning processing on the picture to be detected, and the usability evaluation result is determined to be less than or equal to the usability evaluation threshold according to the usability evaluation result, the mobile net-SSD can be entered to perform tertiary positioning processing and background expansion correction. The reason is that the accuracy of the CASCADE detection model in the HyperLPR is not high enough, so that some pictures to be detected cannot be detected. The MobileNet-SSD uses the depth separable convolution to carry out decomposition calculation on the standard convolution kernel, so that the calculation amount is reduced, and therefore the license plate of the picture to be detected can be quickly positioned to quickly acquire the license plate position. And then correcting the position of the license plate at the detected position through the MobileNet-SSD. The detected license plate is sometimes cut incompletely, only a part of the license plate is detected, so that the result is slightly enlarged in background so as to be sent to subsequent accurate positioning, and the identification precision is improved. And the MobileNet-SSD transmits the corrected picture to be detected into the HyperLPR for second positioning processing and correction processing in the step S1046.
In the embodiment, the license plate recognition precision is improved by combining the MobileNet-SSD and the HyperLPR, and the detection precision can be improved by 20% compared with a single HyperLPR module.
Specific and complete steps of identifying the license plate of the picture to be detected by combining the MobileNet-SSD and the HyperLPR are shown in fig. 6, and include:
step S402: inputting a picture containing a license plate, and enabling k to be 0;
wherein, the picture containing the license plate is the picture to be detected, and the step is realized in HyperLPR;
step S404: carrying out rough positioning on the license plate;
this step is implemented in the HyperLPR;
step S406: accurately positioning and correcting the position of the license plate;
this step is implemented in the HyperLPR;
step S408: carrying out character recognition on the corrected picture;
this step is implemented in the HyperLPR;
step S410: whether there is an identification result;
this step is implemented in the HyperLPR, wherein if the determination result is yes, step S412 is performed, if the determination result is not yes, step S414 is performed,
step S412: outputting the recognition result;
this step is implemented in the HyperLPR;
step S414: whether k is 1;
if the identification result is determined to be negative, executing step S416, and if the identification result is determined to be positive, executing step S412, wherein the step is implemented in MobileNet-SSD;
step S416: inputting the original picture to be detected into a MobileNet-SSD for detection when K is equal to 1;
step S418: correcting the detected license plate frame;
subsequently, the corrected picture to be detected is input to step S406.
Example 6:
the embodiment provides a license plate recognition method, which includes the following technical features in addition to the technical features of the embodiment.
According to the correction result, the method for acquiring the license plate position specifically comprises the following steps: according to the correction result, carrying out secondary positioning processing on the picture information to be detected again, and obtaining a secondary positioning result again; correcting the picture information to be detected again according to the secondary positioning result, and acquiring a correction result again; and acquiring the position of the license plate according to the correction result.
The method for performing the secondary positioning processing on the picture information to be detected is the same as that in embodiment 2, that is, the corrected picture to be detected enters the step of S1044 and the step of S1046 in embodiment 2. Because the corrected picture to be detected is very similar to the required license plate, after appropriate correction, secondary positioning treatment, namely accurate positioning, can be carried out. If the rough positioning is continuously carried out, the detection omission phenomenon can occur because the accurate positioning step is only carried out on the pictures to be detected passing through all the strong classifiers. The method for correcting the picture to be detected after the secondary positioning is performed is the same as the method in the embodiment 2, so as to acquire the parking space position of the corrected picture to be detected.
In the embodiment, the image to be detected extracted from the MobileNet-SSD is subjected to more background expansion through the HyperLPR, the license plate information is obtained by using the precise positioning and character recognition of the HyperLPR, and only the rough positioning model of the HyperLPR is not used.
Example 7:
the embodiment provides a license plate recognition method, which includes the following technical features in addition to the technical features of the embodiment.
According to the correction result, the method for acquiring the license plate position specifically comprises the following steps: according to the correction result, carrying out primary positioning processing on the picture information to be detected again, and obtaining a primary positioning result again; according to the first-stage positioning result, carrying out second-stage positioning processing on the picture information to be detected again, and obtaining a second-stage positioning result again; correcting the picture information to be detected again according to the secondary positioning result, and acquiring a correction result again; and acquiring the position of the license plate according to the correction result.
In this embodiment, the method for performing the primary positioning processing on the information of the picture to be detected is the same as that in embodiment 2, that is, the corrected picture to be detected enters all the steps in embodiment 2. When more backgrounds of the picture to be detected extracted from the MobileNet-SSD are expanded, and then the picture enters the HyperLPR for rough positioning, which is equivalent to integrally calling the HyperLPR, so that the steps of the method are easy to realize, and the program is simplified.
Example 8:
as shown in fig. 7, an embodiment of the present invention further provides a license plate recognition system 100, configured to operate the license plate recognition method according to any of the above embodiments, where the license plate recognition system 100 includes: the system comprises an information acquisition unit 110, a license plate recognition unit 120, a character recognition unit 130 and an information output unit 140, wherein the information acquisition unit 110 is suitable for acquiring information of a picture to be detected; the license plate recognition unit 120 is in communication connection with the information acquisition unit 110, and is adapted to perform positioning processing on the image information to be detected to acquire the position of the license plate; the character recognition unit 130 is in communication connection with the license plate recognition unit 120 and is suitable for recognizing license plate characters in a license plate position and acquiring license plate information; the information output unit 140 is in communication connection with the character recognition unit 130 and is adapted to output license plate information.
In this embodiment, an Open Source Computer Vision Library (OpenCV) module is used as the information obtaining unit 110, and is configured to read a to-be-detected picture including a license plate. OpenCV is a cross-platform computer vision library that can run on Linux, Windows, Android, and Mac OS operating systems. The method is light and efficient, is composed of a series of C functions and a small number of C + + classes, provides interfaces of languages such as Python, Ruby, MATLAB and the like, and realizes a plurality of general algorithms in the aspects of image processing and computer vision. OpenCV transmits the read picture into a HyperLPR model, and the HyperLPR carries out rough positioning, precise positioning and character recognition on the license plate in sequence in the process of processing the picture to be detected. If the result is returned and the score is larger than 0.7, the result is directly returned. Otherwise, performing a second round of recognition on the picture to be detected, namely, firstly performing position positioning on the license plate in the original picture by using a MobileNet _ SSD detection model, then performing background expansion on the positioned license plate, so that the license plate with the expanded background is transmitted into the precise positioning and character recognition module in the HyperLPR again, and returning the result.
It can be seen that the license plate recognition unit 120 is respectively disposed in the HyperLPR and the MobileNet-SSD, and the character recognition unit 130 and the information output unit 140 are disposed in the HyperLPR, that is, the license plate recognition system 100 can be implemented by combining the HyperLPR and the MobileNet-SSD, and has a simple and reliable structure, and is easy to implement.
Example 9:
as shown in fig. 8, the present embodiment provides a license plate recognition system, which includes the following technical features in addition to the technical features of the above embodiments.
The license plate recognition unit 120 includes: the first license plate recognition unit 122 and the second license plate recognition unit 124, wherein the first license plate recognition unit 122 is adapted to perform a first-level positioning process, a second-level positioning process and a correction process on the picture information to be detected, and the second license plate recognition unit 124 is in communication connection with the first license plate recognition unit and is adapted to perform a third-level positioning process and a background expansion correction on the picture information to be detected.
In this embodiment, the first license plate recognition unit 122 is disposed in the HyperLPR, and the second license plate recognition unit 124 is disposed in the MobileNet-SSD, and is configured to process the to-be-detected picture that cannot recognize the license plate information after being processed by the first license plate recognition unit 122, and then return to the first license plate recognition unit 122 to process the to-be-detected picture until the license plate information in the to-be-detected picture can be acquired. The first license plate recognition unit 122 and the second license plate recognition unit 124 combine the HyperLPR and the MobileNet-SSD for use, so that the accuracy and the efficiency of license plate information recognition are improved.
Example 10:
as shown in fig. 9, the present embodiment provides a license plate recognition device 200, including: a memory 210 and a processor 220, the memory 210 storing computer programs; processor 220 executes computer programs; wherein, the processor 220, when executing the computer program, implements the steps of the license plate recognition method as defined in any one of the above embodiments. Therefore, the technical effects of any of the above embodiments are achieved, and are not described herein again.
Example 11:
the present embodiments also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and the computer program, when executed by the processor, implements the steps of the license plate recognition method defined in any of the above embodiments, so that the method has the technical effects of any of the above embodiments, and is not described herein again.
In this embodiment, the computer program when executed by the processor implements the steps of: acquiring information of a picture to be detected; positioning the picture information to be detected to obtain the position of the license plate; identifying license plate characters in the license plate position to acquire license plate information; and outputting the license plate information. In this embodiment, by acquiring the information of the picture to be detected, some conditions of the image in the picture, such as the color of the vehicle and the surrounding environment of the vehicle, can be obtained. Furthermore, the acquired picture information is positioned, so that the position of the license plate in the picture can be accurately positioned, and the accuracy of positioning the license plate position is improved. After the license plate position is obtained, the license plate characters in the license plate position are identified, and because the license plate characters of each vehicle are different, the license plate information can be obtained by reading the license plate characters, so that the output license plate information is more accurate, and the accuracy of obtaining the license plate information is improved.
In summary, the embodiment of the invention has the following beneficial effects: by acquiring the information of the picture to be detected and accurately positioning the position of the license plate in the picture to be detected, the precision of identifying the license plate information in the license plate can be improved. After the license plate position is obtained, the license plate characters in the license plate position are identified, and because the license plate characters of each vehicle are different, the license plate information can be obtained by reading the license plate characters, so that the output license plate information is more accurate, and the accuracy of obtaining the license plate information is improved.
In the present invention, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance; the term "plurality" means two or more unless expressly limited otherwise. The terms "mounted," "connected," "fixed," and the like are to be construed broadly, and for example, "connected" may be a fixed connection, a removable connection, or an integral connection; "coupled" may be direct or indirect through an intermediary. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the description herein, the description of the terms "one embodiment," "some embodiments," "specific embodiments," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (11)

1. A license plate recognition method is characterized by comprising the following steps:
acquiring information of a picture to be detected;
positioning the to-be-detected picture information to acquire the position of the license plate;
identifying license plate characters in the license plate position to acquire license plate information;
and outputting the license plate information.
2. The license plate recognition method of claim 1, wherein the step of positioning the image information to be detected and obtaining the license plate position specifically comprises the steps of:
performing primary positioning processing on the picture information to be detected to obtain a primary positioning result;
according to the primary positioning result, performing secondary positioning processing on the picture information to be detected to obtain a secondary positioning result;
and according to the secondary positioning result, correcting the information of the picture to be detected to obtain a correction result.
3. The license plate recognition method of claim 2, wherein after the step of performing the correction processing on the to-be-detected picture information according to the secondary positioning result to obtain a correction result, the license plate recognition method further comprises the steps of:
evaluating the availability degree of the correction result to obtain an availability degree evaluation result;
and acquiring the license plate position according to the usability degree evaluation result.
4. The license plate recognition method of claim 3, wherein the obtaining of the license plate position according to the usability degree evaluation result specifically comprises:
judging that the usability evaluation result is greater than a usability evaluation threshold;
and acquiring the position of the license plate according to the correction result.
5. The license plate recognition method of claim 3, wherein the obtaining of the license plate position according to the usability degree evaluation result specifically comprises:
determining that the usability evaluation result is less than or equal to a usability evaluation threshold;
carrying out three-level positioning processing and background amplification correction on the picture information to be detected to obtain a correction result;
and acquiring the position of the license plate according to the correction result.
6. The license plate recognition method of claim 5, wherein the obtaining of the license plate position according to the correction result specifically comprises:
according to the correction result, the secondary positioning processing is carried out on the picture information to be detected again, and the secondary positioning result is obtained again;
correcting the to-be-detected picture information again according to the secondary positioning result, and acquiring a correction result again;
and acquiring the position of the license plate according to the correction result.
7. The license plate recognition method of claim 5, wherein the obtaining of the license plate position according to the correction result specifically comprises:
according to the correction result, the primary positioning processing is carried out on the picture information to be detected again, and a primary positioning result is obtained again;
according to the primary positioning result, performing the secondary positioning processing on the picture information to be detected again, and acquiring a secondary positioning result again;
correcting the to-be-detected picture information again according to the secondary positioning result, and acquiring a correction result again;
and acquiring the position of the license plate according to the correction result.
8. A license plate recognition system that realizes the license plate recognition method according to any one of claims 1 to 7, the license plate recognition system comprising:
the information acquisition unit is suitable for acquiring the information of the picture to be detected;
the license plate recognition unit is in communication connection with the information acquisition unit and is suitable for positioning the to-be-detected picture information to acquire the position of the license plate;
the character recognition unit is in communication connection with the license plate recognition unit and is suitable for recognizing the license plate characters in the license plate position to acquire the license plate information;
and the information output unit is in communication connection with the character recognition unit and is suitable for outputting the license plate information.
9. The license plate recognition system of claim 8, wherein the license plate recognition unit comprises:
the first license plate identification unit is suitable for carrying out primary positioning processing, secondary positioning processing and correction processing on the picture information to be detected;
and the second license plate identification unit is in communication connection with the first license plate identification unit and is suitable for carrying out three-level positioning processing and background amplification correction on the to-be-detected picture information.
10. A license plate recognition device, comprising:
a memory storing a computer program;
a processor executing the computer program;
wherein the processor, when executing the computer program, performs the steps of the license plate recognition method of any of claims 1 to 7.
11. A computer-readable storage medium, comprising:
the computer-readable storage medium stores a computer program which, when executed, implements the steps of the license plate recognition method of any one of claims 1 to 7.
CN202010123472.1A 2020-02-27 2020-02-27 License plate recognition method, system, device and computer readable storage medium Pending CN111368816A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010123472.1A CN111368816A (en) 2020-02-27 2020-02-27 License plate recognition method, system, device and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010123472.1A CN111368816A (en) 2020-02-27 2020-02-27 License plate recognition method, system, device and computer readable storage medium

Publications (1)

Publication Number Publication Date
CN111368816A true CN111368816A (en) 2020-07-03

Family

ID=71212081

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010123472.1A Pending CN111368816A (en) 2020-02-27 2020-02-27 License plate recognition method, system, device and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN111368816A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113781790A (en) * 2021-08-31 2021-12-10 深圳市捷顺科技实业股份有限公司 Method for recognizing license plate and parking lot terminal

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105701487A (en) * 2014-11-26 2016-06-22 王云 Vehicle logo positioning method
CN108701234A (en) * 2018-03-05 2018-10-23 深圳前海达闼云端智能科技有限公司 Licence plate recognition method and cloud system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105701487A (en) * 2014-11-26 2016-06-22 王云 Vehicle logo positioning method
CN108701234A (en) * 2018-03-05 2018-10-23 深圳前海达闼云端智能科技有限公司 Licence plate recognition method and cloud system
WO2019169532A1 (en) * 2018-03-05 2019-09-12 深圳前海达闼云端智能科技有限公司 License plate recognition method and cloud system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113781790A (en) * 2021-08-31 2021-12-10 深圳市捷顺科技实业股份有限公司 Method for recognizing license plate and parking lot terminal
CN113781790B (en) * 2021-08-31 2022-08-12 深圳市捷顺科技实业股份有限公司 Method for recognizing license plate and parking lot terminal

Similar Documents

Publication Publication Date Title
TWI497422B (en) A system and method for recognizing license plate image
CN110197589B (en) Deep learning-based red light violation detection method
CN111488875B (en) Vehicle insurance claim settlement loss checking method and device based on image recognition and electronic equipment
Abdullah et al. YOLO-based three-stage network for Bangla license plate recognition in Dhaka metropolitan city
JP2008130086A (en) Method and apparatus for determining classification boundary
CN111325769A (en) Target object detection method and device
Yonetsu et al. Two-stage YOLOv2 for accurate license-plate detection in complex scenes
US11482012B2 (en) Method for driving assistance and mobile device using the method
CN111860610A (en) License plate overexposure and license plate vehicle identification method and device and computer storage medium
CN112115800A (en) Vehicle combination recognition system and method based on deep learning target detection
Elbamby et al. Real-time automatic multi-style license plate detection in videos
CN112085018A (en) License plate recognition system based on neural network
CN111444843A (en) Multimode driver and vehicle illegal behavior monitoring method and system
CN111368816A (en) License plate recognition method, system, device and computer readable storage medium
CN112101274B (en) License plate recognition method, device, equipment and readable storage medium
TWI603268B (en) Image processing system and method for license plate recognition
CN116778460A (en) Fatigue driving identification method based on image identification
KR102379575B1 (en) Code recognition device, code recognition method, and vehicle number extraction system using the same
CN116468931A (en) Vehicle part detection method, device, terminal and storage medium
CN111402185A (en) Image detection method and device
CN104077566B (en) Bayonet socket picture method for detecting human face based on color difference
CN106920398A (en) A kind of intelligent vehicle license plate recognition system
Vinay et al. Automatic Number Plate Recognition for Different Fonts and Non-Roman Script
Umar et al. Traffic violation detection system using image processing
Nooruddin et al. A bangladeshi license plate detection system based on extracted color features

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20200703