CN111582263A - License plate recognition method and device, electronic equipment and storage medium - Google Patents

License plate recognition method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN111582263A
CN111582263A CN202010397362.4A CN202010397362A CN111582263A CN 111582263 A CN111582263 A CN 111582263A CN 202010397362 A CN202010397362 A CN 202010397362A CN 111582263 A CN111582263 A CN 111582263A
Authority
CN
China
Prior art keywords
license plate
corner
corner detection
recognition
detection frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010397362.4A
Other languages
Chinese (zh)
Inventor
周康明
丁子凡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Eye Control Technology Co Ltd
Original Assignee
Shanghai Eye Control Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Eye Control Technology Co Ltd filed Critical Shanghai Eye Control Technology Co Ltd
Priority to CN202010397362.4A priority Critical patent/CN111582263A/en
Publication of CN111582263A publication Critical patent/CN111582263A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/148—Segmentation of character regions
    • G06V30/153—Segmentation of character regions using recognition of characters or words
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

The embodiment of the application provides a license plate recognition method, a license plate recognition device, electronic equipment and a storage medium. The license plate position recognition network model is used for positioning the position of the license plate in the vehicle appearance image, and then the image of the recognition area where the positioned license plate is located is recognized, so that the license plate recognition accuracy of vehicles such as trucks and engineering vehicles, wherein the vehicles are coated with license plate information on the vehicle body, can be effectively improved.

Description

License plate recognition method and device, electronic equipment and storage medium
Technical Field
The embodiment of the disclosure relates to the field of image recognition processing, and in particular relates to a license plate recognition method and device, electronic equipment and a storage medium.
Background
With the development of image recognition technology, the image recognition technology is widely applied to various industries. In the field of intelligent transportation including automatic driving, the license plate of a vehicle can be identified by using an image identification technology.
In the prior art, the recognition of the license plate is generally based on a text detection technology, for example, a recognition network model capable of realizing character recognition is used to recognize the vehicle image so as to acquire the license plate information therein.
However, such an implementation mode depends on the character definition of the license plate information on the license plate, and for large vehicles such as trucks and engineering vehicles, the license plate is easily blocked due to the large vehicle body, and generally the license plate needs to be painted on the vehicle body to identify the license plate information, while for the license plate information painted on the vehicle body, due to the large size and irregular text, the existing mode easily omits the character information when identifying the license plate information, and the identification accuracy is not good.
Disclosure of Invention
In order to solve the above problems, the present disclosure provides a license plate recognition method, a license plate recognition device, an electronic device, and a storage medium.
In a first aspect, an embodiment of the present disclosure provides a license plate recognition method, including:
obtaining a vehicle appearance image;
calling an angular point detection algorithm and a segmentation algorithm in a license plate position recognition network model to determine a recognition area of a license plate to be recognized in the vehicle appearance image;
and calling a preset identification network model to identify the image corresponding to the identification area, and obtaining the license plate information in the vehicle appearance image.
In an optional embodiment, the step of determining the recognition area of the license plate to be recognized in the vehicle appearance image by calling a corner detection algorithm and a segmentation algorithm in the license plate position recognition network model comprises the following steps:
carrying out feature extraction processing on the vehicle appearance image to obtain image features;
processing the image characteristics by using an angular point detection algorithm and a segmentation algorithm respectively so as to position the license plate position of the license plate to be identified in the vehicle appearance image and obtain a corresponding angular point detection area and a corresponding segmentation area;
and obtaining the identification region of the license plate to be identified in the vehicle appearance image according to the corner detection region and the segmentation region.
In an optional embodiment, the method further comprises:
establishing a license plate position recognition network model to be trained, and collecting license plate recognition samples; the license plate identification sample comprises a sample image and a license plate label of the sample image;
and respectively training an angular point detection algorithm and a segmentation algorithm in the license plate position recognition network model to be trained by using the license plate recognition samples to obtain a license plate position recognition network model for recognizing a recognition area in an appearance image of the vehicle.
In an optional embodiment, the license plate labeling of the sample image comprises a target corner detection frame for labeling four corners of a license plate;
the training of the corner point detection algorithm in the license plate position recognition network model to be trained by using the license plate recognition sample comprises the following steps:
processing the sample images in the license plate recognition samples by using an angular point detection algorithm in a license plate position recognition network model to be trained to obtain a plurality of angular point detection frames of the sample images;
matching each corner detection frame of the sample image with each target corner detection frame, and determining an optimal corner detection frame corresponding to the target corner detection frame of each corner;
establishing a corner loss function according to the target corner detection frame of each corner and the corresponding optimal corner detection frame;
and training the corner detection algorithm to be trained by using the corner loss function to obtain the corner detection algorithm which can be used for positioning the license plate position of the license plate to be identified so as to obtain the corner detection area.
In an optional embodiment, the matching processing of each corner detection frame of the sample image and each target corner detection frame to determine an optimal corner detection frame corresponding to the target corner detection frame of each corner includes:
calculating the intersection and parallel ratio between each angular point detection frame and each target angular point detection frame;
aiming at each target corner detection frame, determining a plurality of candidate corner detection frames corresponding to each target corner detection frame according to the intersection ratio, wherein the intersection ratio of the candidate corner detection frames meets a preset condition;
and aiming at each candidate corner detection frame corresponding to each target corner detection frame, determining an optimal corner detection frame corresponding to each target corner detection frame by using a non-maximum suppression algorithm.
In an optional embodiment, among a plurality of corner detection boxes of the obtained sample image, any two corner detection boxes have the same size;
correspondingly, the calculating the intersection-parallel ratio between each corner detection frame and each target corner detection frame further includes: calculating the distance between the center points of any two corner detection frames in a plurality of corner detection frames of the obtained sample image to obtain the distance between every two corner detection frames;
screening a plurality of corner detection frames of the obtained sample image according to each distance, and taking the corner detection frames remained after screening as the corner detection frames so as to carry out intersection and comparison calculation with the target corner detection frames; and in the corner detection frames reserved after screening, the distance between the center points of any two corner detection frames is greater than a preset distance threshold.
In an optional embodiment, the establishing a corner loss function according to the target corner detection box of each corner and the optimal corner detection box corresponding to the target corner detection box includes:
performing cross entropy processing on a target corner detection area formed by the target corner detection frame of each corner and an optimal corner detection area formed by each optimal corner detection frame to obtain corner classification loss;
carrying out position offset processing on the target corner detection frame of each corner and the optimal corner detection frame corresponding to the target corner detection frame to obtain corner position offset loss;
and establishing a corner loss function by the corner classification loss, the corner position offset loss and a preset loss function.
In a second aspect, an embodiment of the present application provides a license plate recognition device, including:
a communication module for obtaining a vehicle appearance image;
the license plate positioning module is used for calling an angular point detection algorithm and a segmentation algorithm in the license plate position recognition network model to determine a recognition area of a license plate to be recognized in the vehicle appearance image;
and the license plate recognition module is used for calling a preset recognition network model to recognize the image corresponding to the recognition area and obtaining the license plate information in the vehicle appearance image.
In an optional embodiment, the license plate location module is specifically configured to:
carrying out feature extraction processing on the vehicle appearance image to obtain image features;
processing the image characteristics by using an angular point detection algorithm and a segmentation algorithm respectively so as to position the license plate position of the license plate to be identified in the vehicle appearance image and obtain a corresponding angular point detection area and a corresponding segmentation area;
and obtaining the identification region of the license plate to be identified in the vehicle appearance image according to the corner detection region and the segmentation region.
In an optional embodiment, the apparatus further comprises: a training module;
the training module is used for establishing a license plate position recognition network model to be trained and collecting license plate recognition samples; the license plate identification sample comprises a sample image and a license plate label of the sample image;
and respectively training an angular point detection algorithm and a segmentation algorithm in the license plate position recognition network model to be trained by using the license plate recognition samples to obtain a license plate position recognition network model for recognizing a recognition area in an appearance image of the vehicle.
In an optional embodiment, the license plate labeling of the sample image comprises a target corner detection frame for labeling four corners of a license plate;
the training module is specifically configured to:
processing the sample images in the license plate recognition samples by using an angular point detection algorithm in a license plate position recognition network model to be trained to obtain a plurality of angular point detection frames of the sample images;
matching each corner detection frame of the sample image with each target corner detection frame, and determining an optimal corner detection frame corresponding to the target corner detection frame of each corner;
establishing a corner loss function according to the target corner detection frame of each corner and the corresponding optimal corner detection frame;
and training the corner detection algorithm to be trained by using the corner loss function to obtain the corner detection algorithm which can be used for positioning the license plate position of the license plate to be identified so as to obtain the corner detection area.
In an optional embodiment, the training module is specifically configured to:
calculating the intersection and parallel ratio between each angular point detection frame and each target angular point detection frame;
aiming at each target corner detection frame, determining a plurality of candidate corner detection frames corresponding to each target corner detection frame according to the intersection ratio, wherein the intersection ratio of the candidate corner detection frames meets a preset condition;
and aiming at each candidate corner detection frame corresponding to each target corner detection frame, determining an optimal corner detection frame corresponding to each target corner detection frame by using a non-maximum suppression algorithm.
In an optional embodiment, among a plurality of corner detection boxes of the obtained sample image, any two corner detection boxes have the same size;
correspondingly, the training module is further specifically configured to:
calculating the distance between the center points of any two corner detection frames in the plurality of corner detection frames for obtaining the sample image so as to obtain the distance between every two corner detection frames;
screening a plurality of corner detection frames of the obtained sample image according to each distance, and taking the corner detection frames remained after screening as the corner detection frames so as to carry out intersection and comparison calculation with the target corner detection frames; and in the corner detection frames reserved after screening, the distance between the center points of any two corner detection frames is greater than a preset distance threshold.
In an optional embodiment, the training module is specifically configured to: performing cross entropy processing on a target corner detection area formed by the target corner detection frame of each corner and an optimal corner detection area formed by each optimal corner detection frame to obtain corner classification loss; carrying out position offset processing on the target corner detection frame of each corner and the optimal corner detection frame corresponding to the target corner detection frame to obtain corner position offset loss; and establishing a corner loss function by the corner classification loss, the corner position offset loss and a preset loss function.
In a third aspect, an embodiment of the present application provides an electronic device, including: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executes computer-executable instructions stored in the memory, causing the at least one processor to perform the license plate recognition method of any of the first aspects.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the license plate recognition method according to any one of the first aspect is implemented.
The embodiment of the application provides a license plate recognition method, a license plate recognition device, electronic equipment and a storage medium. The license plate position recognition network model is used for positioning the position of the license plate in the vehicle appearance image, and then the image of the recognition area where the positioned license plate is located is recognized, so that the license plate recognition accuracy of vehicles such as trucks and engineering vehicles, wherein the vehicles are coated with license plate information on the vehicle body, can be effectively improved.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present disclosure, and for those skilled in the art, other drawings can be obtained according to the drawings without inventive exercise.
FIG. 1 is a schematic diagram of a network architecture upon which the present disclosure is based;
fig. 2 is a schematic flow chart of a license plate recognition method according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram of an identification area in a license plate identification method provided in the present application;
fig. 4 is a schematic diagram of a license plate labeling in a license plate recognition method provided by the present application;
fig. 5 is a schematic diagram of an architecture of a license plate location identification network model of a license plate identification method according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram of an optimal corner detection box in a license plate recognition method according to an embodiment of the disclosure;
fig. 7 is a block diagram of a license plate recognition device according to an embodiment of the present disclosure;
fig. 8 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present disclosure.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some, but not all embodiments of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
With the development of image recognition technology, the image recognition technology is widely applied to various industries. In the field of intelligent transportation including automatic driving, the license plate of a vehicle can be identified by using an image identification technology.
In the prior art, the recognition of the license plate is generally based on a text detection technology, for example, a recognition network model capable of realizing character recognition is used to recognize the vehicle image so as to acquire the license plate information therein.
However, such an implementation mode depends on the character definition of the license plate information on the license plate, and for large vehicles such as trucks and engineering vehicles, the license plate is easily blocked due to the large vehicle body, and the license plate information on the vehicle body generally needs to be painted for identification, while for the license plate information painted on the vehicle body, due to the large size and irregular text, the existing mode easily omits the character information when identifying the license plate information, and the identification accuracy is not good.
In order to solve the above problems, the present disclosure provides a license plate recognition method, a license plate recognition device, an electronic device, and a storage medium.
Referring to fig. 1, fig. 1 is a schematic diagram of a network architecture on which the present disclosure is based, and as shown in fig. 1, a network architecture on which the present disclosure is based may include a license plate recognition device 1 and a terminal 2.
The license plate recognition device 1 is hardware or software that can interact with the terminal 1 through a network, and can be used to execute the license plate recognition method described in each embodiment described below.
When the license plate recognition device 1 is hardware, the license plate recognition device comprises a cloud server with an operation function. When the license plate recognition device 1 is software, it can be installed in an electronic device with an arithmetic function, wherein the electronic device includes, but is not limited to, a laptop portable computer, a desktop computer, and the like.
In an actual use scene, the license plate recognition device can be integrated in road monitoring equipment (such as a traffic violation monitor, vehicle snapshot equipment and the like), and obtains a vehicle appearance image snapshot by the road monitoring equipment, and sends the license plate recognition device to recognize and record license plate information in the image in the following manner.
In other usage scenarios, the license plate recognition device may be integrated in a server for analyzing license plate information of a vehicle, such as a license plate recognition server, and at this time, the terminal may be a device capable of communicating with the license plate recognition device and performing data interaction with the license plate recognition device through a network, including a smart phone, a tablet computer, a desktop computer, and the like. The terminal can send the vehicle appearance image to be analyzed to the license plate recognition device, so that the license plate recognition device recognizes the license plate information in the image in the following mode and returns the recognized license plate information to the terminal.
The present application provides a method, an apparatus, an electronic device and a storage medium for identifying a side license plate, which will be further described below:
in a first aspect, referring to fig. 2, fig. 2 is a schematic flowchart of a license plate recognition method according to an embodiment of the disclosure. The license plate recognition method provided by the embodiment of the disclosure comprises the following steps:
and step 101, obtaining a vehicle appearance image.
It should be noted that the execution subject of the license plate recognition method provided in this embodiment is the aforementioned license plate recognition device.
As described in the background art, in the prior art, the recognition of the license plate is generally based on a text detection technology, for example, a recognition network model capable of implementing character recognition is used to recognize the vehicle image to obtain the license plate information therein. However, since the license plate of a large vehicle such as a truck or a construction vehicle is small relative to the vehicle body, the vehicle body is often painted with the license plate information in order to facilitate recognition of the license plate information. The license plate recognition mode provided by the embodiment can be well compatible with recognition of the spray type license plate information of the large-sized vehicle.
Specifically, first, the license plate recognition device needs to acquire a vehicle appearance image including the appearance of the vehicle, and generally, the vehicle appearance image needs to include a vehicle body and license plate information or a license plate sprayed on the vehicle body.
And 102, calling a corner detection algorithm and a segmentation algorithm in the license plate position recognition network model to determine a recognition area of the license plate to be recognized in the vehicle appearance image.
Different from the prior art, in the embodiment, the adopted license plate position recognition network model firstly recognizes and positions the position of the license plate to be recognized in the image, so that all license plate information of the vehicle is included in the recognition area, and the problems of character recognition omission and the like in the process of recognizing the image of the recognition area are avoided.
In order to determine the position of the license plate information of the vehicle, the license plate position recognition network model is used in this embodiment to determine the position of the license plate information, and specifically, the license plate position recognition network model is implemented based on a SegNet network architecture, and of course, based on an actual situation, other network architectures such as an FCN network architecture or a U-Net network architecture may also be used, which is not limited in this application.
Furthermore, the license plate position identification network model can comprise two algorithms, one of which is an angular point detection algorithm, namely, the positions of four angular points of the license plate information are determined to determine an identification area corresponding to the license plate information; the second is a segmentation algorithm, namely, a pixel range corresponding to license plate information is determined by analyzing the type of an object to which a pixel in an image belongs, and then a recognition area corresponding to the license plate information is determined.
In an alternative embodiment, the license plate position recognition network model can determine the recognition area by adopting the following modes:
firstly, the license plate recognition device performs feature extraction processing on the vehicle appearance image to obtain image features. And then, the license plate recognition device processes the image characteristics by respectively utilizing an angular point detection algorithm and a segmentation algorithm so as to position the license plate position of the license plate to be recognized in the vehicle appearance image and obtain a corresponding angular point detection area and a corresponding segmentation area. And finally, the license plate recognition device obtains the recognition area of the license plate to be recognized in the vehicle appearance image according to the corner detection area and the segmentation area.
Specifically, in order to increase the operation speed, the license plate position recognition network model provided in this embodiment is a trained network model, and the training process of the license plate position recognition network model is described in the following embodiments, which are not described in detail in this embodiment.
In addition, for the license plate position identification network model, because the license plate position identification network model comprises two different algorithms, in order to reduce the operation amount, the angular point detection algorithm and the segmentation algorithm can adopt a mode of sharing image characteristics to save operation resources, namely, the image characteristics obtained by performing operations such as convolution, sampling and the like on the vehicle appearance image can be synchronously or asynchronously input into the angular point detection algorithm and the segmentation algorithm to realize the determination of the identification area.
After the corner detection algorithm is used for positioning the license plate position of the license plate to be recognized in the vehicle appearance image, a corresponding corner detection area is obtained, and after the segmentation algorithm is used for positioning the license plate position of the license plate to be recognized in the vehicle appearance image, a corresponding segmentation area is obtained.
The license plate recognition device will determine a recognition area based on the corner detection area and the segmentation area. The specific mode of determination can be realized based on an outer contour, fig. 3 is a schematic diagram of an identification region in the license plate identification method provided by the application, and as shown in fig. 3, after an angular point detection region and a segmentation region are obtained, a rectangle containing outer contours of the angular point detection region and the segmentation region is used as the identification region.
And 103, calling a preset identification network model to identify the image corresponding to the identification area, and obtaining license plate information in the vehicle appearance image.
In this embodiment, after the identification area where the license plate is located in the vehicle appearance image is determined, the existing identification network model may be called to perform analysis processing on the image corresponding to the identification area, so as to obtain the license plate information included in the image corresponding to the identification area.
After the image corresponding to the identification area is obtained, the size of the image can be reset, for example, the image is reset to 360 x 60, and then the image is sent to a trained identification network model for identification, so that the information result of the painted license plate in the appearance of the truck can be obtained. Typically, the dimensions, including 360 x 60, are determined based on identifying network model settings.
According to the license plate recognition method provided by the embodiment, a vehicle appearance image is obtained, a license plate position recognition network model is called, an angular point detection algorithm and a segmentation algorithm are adopted to determine a recognition area of a license plate to be recognized in the vehicle appearance image, a preset recognition network model is called to recognize an image corresponding to the recognition area, and license plate information in the vehicle appearance image is obtained. The license plate position recognition network model is used for positioning the position of the license plate in the vehicle appearance image, and then the image of the recognition area where the positioned license plate is located is recognized, so that the license plate recognition accuracy of vehicles such as trucks and engineering vehicles, wherein the vehicles are coated with license plate information on the vehicle body, can be effectively improved.
In order to further ensure the integrity of the license plate information obtained by recognition, for the license plate position recognition network model used for determining the recognition area, the accuracy of the output recognition area is seriously influenced by the training effectiveness of the license plate position recognition network model. On the basis of the above embodiments, the embodiments of the present application further include:
step 201, establishing a license plate position recognition network model to be trained, and collecting license plate recognition samples; the license plate identification sample comprises a sample image and a license plate label of the sample image;
step 202, respectively training an angular point detection algorithm and a segmentation algorithm in the license plate position recognition network model to be trained by using the license plate recognition samples to obtain a license plate position recognition network model for recognizing a recognition area in an appearance image of a vehicle.
Specifically, in the training process, a license plate recognition sample needs to be collected. The license plate identification sample comprises a sample image and a license plate label of the sample image. The license plate labeling can be a target corner detection frame labeled with four corners of the license plate. And marking the identification region in the license plate marking process of the sample image of the license plate identification sample by marking the corner point.
During labeling, developers can use the corners of the four corners of the license plate information as labeling points, so that the license plate recognition device generates target corner detection boxes with the corners of the four corners as the labeling points according to preset settings, and then obtains a labeled target recognition area (the target recognition area is a minimum rectangular area capable of covering all the target corner detection boxes) based on each target corner detection box, wherein the target recognition area can be used as a target corner detection area during subsequent diagonal detection algorithm training and a target separation area during subsequent segmentation algorithm training.
Fig. 4 is a schematic diagram of a license plate labeling in the license plate recognition method provided by the present application, as shown in fig. 4, after a developer labels license plate information, a license plate recognition device generates a target corner detection frame as shown in fig. 4, where the target corner detection frame is a two-dimensional corner coordinate range generated based on 4 clockwise labeling points (upper left corner, upper right corner, lower left corner).
Specifically, the detection frame may adopt a square frame structure, that is, the frame length of the rectangular bounding box is d with the marked point as the center. For example, the target corner detection boxes shown in the upper left corner of FIG. 4, which are generated based on the annotation points with coordinates (x1, y1), should have ranges (vertex coordinates of the detection boxes) of (x1-d, y1+ d), (x1+ d, y1+ d), (x1-d, y1-d), and (x1+ d, y 1-d). By adopting the detection frame mode, the range of the identification area can be determined based on more pixel information when the position of the license plate information is identified, so that the obtained range of the identification area is more accurate.
Fig. 5 is a schematic diagram of an architecture of a license plate position recognition network model of a license plate recognition method according to an embodiment of the present disclosure, and as shown in fig. 5, the license plate position recognition network model includes a feature extraction layer, an angular point detection algorithm processing layer, and a segmentation algorithm processing layer, and a recognition network model for recognizing license plate information.
When the license plate position recognition network model to be trained is established, in order to reduce the operation amount, the angular point detection algorithm and the segmentation algorithm can share the feature extraction layer, so that the angular point detection algorithm processing layer and the segmentation algorithm processing layer respectively obtain the image features output by the feature extraction layer and perform corresponding calculation.
Different training methods can be adopted for different algorithms in the license plate position recognition network model. Particularly, in the embodiment of the present application, a new training mode is provided for the corner detection algorithm, so as to improve the accuracy of the corner detection algorithm for identifying the corner detection area.
And training the corner point detection algorithm in the license plate position recognition network model to be trained by using the license plate recognition sample, wherein the training comprises the following steps:
step 2021, processing the sample image in the license plate recognition sample by using an angular point detection algorithm in the license plate position recognition network model to be trained, and obtaining a plurality of angular point detection frames of the sample image.
Specifically, in step 2021, some corner detection blocks may be framed in a preset size scale for each pixel in the sample image as the plurality of corner detection blocks in step 2021. It should be noted that the corner detection boxes in the embodiments of the present disclosure refer to box-shaped identifiers having a certain size ratio, where the size and/or the central pixel and/or the ratio of each corner detection box are different.
Step 2022, matching each corner detection frame of the sample image with each target corner detection frame, and determining an optimal corner detection frame corresponding to the target corner detection frame of each corner.
Specifically, step 2022 may include calculating an intersection ratio between each corner detection box and each target corner detection box; aiming at each target corner detection frame, determining a plurality of candidate corner detection frames corresponding to each target corner detection frame according to the intersection ratio, wherein the intersection ratio of the candidate corner detection frames meets a preset condition; and aiming at each candidate corner detection frame corresponding to each target corner detection frame, determining an optimal corner detection frame corresponding to each target corner detection frame by using a non-maximum suppression algorithm.
Fig. 6 is a schematic diagram of an optimal corner detection frame in a license plate recognition method provided by an embodiment of the present disclosure, and first, for each target corner detection frame in the four target corner detection frames, an intersection-to-parallel ratio between each corner detection frame obtained in step 2021 and the target corner detection frame is determined, where the intersection-to-parallel ratio is a ratio between an intersection of two detection frames and a union of the two detection frames, and the higher the intersection-to-parallel ratio is, the higher the coincidence degree and the similarity degree of the two detection frames are. Therefore, for each target corner detection frame, a number of corner detection frames that intersect with the target corner detection frame with a cross ratio greater than a preset cross ratio threshold may be first found from the plurality of corner detection frames obtained in step 2021, and used as candidate corner detection frames.
Optionally, the relative position relationship between the four target corner detection boxes is considered, and an optimal corner detection box corresponding to each target corner detection box is determined from the candidate corner detection boxes by using a non-maximum suppression algorithm.
Specifically, for each corner, there are a plurality of candidate corner detection boxes, that is, any one of the candidate corner detection boxes at four corners is selected, and the connection lines of the center points of the four selected corner detection boxes are used to form the final candidate corner identification area, and the number of candidate corner identification areas formed by this method is plural.
In the process, as the license plate information needs to be included in the corner point identification area, the aspect ratio of the general license plate information has certain constraint. Therefore, when the selection of the optimal corner detection frame is considered from the whole, candidate corner identification regions which do not meet the aspect ratio constraint of license plate information can be screened out, so that a plurality of candidate corner identification regions which meet the constraint and candidate corner detection frames which form the corner identification regions are reserved. And finally, finding out a candidate corner identification area with the highest intersection ratio with a target corner identification area in the license plate label from the reserved candidate corner identification areas as an optimal corner identification area, wherein a corresponding corner detection frame forming the optimal corner identification area is used as an optimal corner detection frame.
In other optional embodiments, in order to reduce the computation amount in the foregoing process, correspondingly, the calculating an intersection-to-parallel ratio between each corner detection frame and each target corner detection frame further includes:
calculating the distance between the center points of any two corner detection frames in the plurality of corner detection frames for obtaining the sample image so as to obtain the distance between every two corner detection frames; screening a plurality of corner detection frames of the obtained sample image according to each distance, and taking the corner detection frames remained after screening as the corner detection frames so as to carry out intersection and comparison calculation with the target corner detection frames; wherein, in the corner point detection frames reserved after screening, the distance between the center points of any two corner point detection frames is greater than a preset distance threshold; and calculating the intersection-to-parallel ratio between each reserved corner detection frame and each target corner detection frame.
Step 2023, building a corner loss function according to the optimal corner detection box and the target corner detection box of each corner.
Step 2024, training the corner detection algorithm to be trained by using the corner loss function to obtain a corner detection algorithm which can be used for positioning the license plate position of the license plate to be recognized to obtain a corner detection area.
Specifically, step 2023 is adopted to establish a corresponding corner loss function, which may be specifically implemented based on a softmax function. Specifically, the cross entropy processing may be performed on a target corner detection region formed by the target corner detection box of each corner and an optimal corner detection region formed by each optimal corner detection box to obtain the corner classification loss. And then, carrying out position offset processing on the target corner detection frame of each corner and the optimal corner detection frame corresponding to the target corner detection frame to obtain corner position offset loss. And finally, establishing a corner loss function based on the corner classification loss, the corner position offset loss and a preset loss function. Wherein, the corner classification loss, the corner position offset loss and the preset loss function can be summed to obtain the corner loss function.
In an alternative embodiment, the corner Loss function Loss ═ Ldet + Lseg ═ cross control (yt, et) + smoothL1(yl, pl) + Softmax may be constructed as follows.
Wherein crossEncopy (yt, et) is the corner classification loss, yt represents the target corner detection area, and et represents the optimal corner detection area; smoothL1(yl, pl) is the angular point position offset loss, yl is the offset result obtained by performing position offset processing on the target angular point detection frame, pl is the offset result obtained by performing offset processing on the optimal angular point detection frame, and yl is also associated with a preset regression function and the target angular point detection area; softmax is the existing loss function.
Finally, the corner point loss function is used in step 2024 to train the corner point detection algorithm to be trained, so that the corner point loss function between the optimal corner point detection frame generated by the trained corner point detection algorithm and the target corner point detection frame is in a convergence state, i.e. the deviation is minimum, and at this time, the trained corner point detection algorithm is used as the algorithm in the model.
In other alternative embodiments, existing algorithms may be used for training the segmentation model and the license plate recognition network, which is not limited in this application.
The embodiment of the application provides a license plate recognition method, which comprises the steps of determining a recognition area of a license plate to be recognized in a vehicle appearance image by calling an angular point detection algorithm and a segmentation algorithm in a license plate position recognition network model, calling a preset recognition network model to recognize an image corresponding to the recognition area, and obtaining license plate information in the vehicle appearance image. Particularly, an angular point detection algorithm and a segmentation algorithm are fused by utilizing a license plate position recognition network model, so that the problem that the license plate information sprayed on a vehicle body of a truck, an engineering vehicle and the like cannot be positioned easily due to the fact that the direction and the aspect ratio of the license plate information in an image are indefinite is solved. And finally, the image of the identification area where the positioned license plate is located is identified by using the identification network model, so that the license plate identification accuracy of vehicles such as trucks and engineering vehicles, wherein the vehicles are coated with license plate information on the vehicle body, can be effectively improved. In addition, the algorithm in the license plate position recognition network model adopts a mode of sharing a feature extraction layer, so that the calculation amount can be effectively reduced, and the processing efficiency is improved.
Fig. 7 is a schematic structural diagram of a license plate recognition device provided in an embodiment of the present disclosure, and as shown in fig. 7, an embodiment of the present disclosure provides a license plate recognition device, including:
a communication module 10 for obtaining a vehicle appearance image;
the license plate positioning module 20 is used for calling an angular point detection algorithm and a segmentation algorithm in the license plate position recognition network model to determine a recognition area of a license plate to be recognized in the vehicle appearance image;
and the license plate recognition module 30 is configured to call a preset recognition network model to recognize the image corresponding to the recognition area, and obtain license plate information in the vehicle appearance image.
In an optional embodiment, the license plate location module 20 is specifically configured to:
carrying out feature extraction processing on the vehicle appearance image to obtain image features;
processing the image characteristics by using an angular point detection algorithm and a segmentation algorithm respectively so as to position the license plate position of the license plate to be identified in the vehicle appearance image and obtain a corresponding angular point detection area and a corresponding segmentation area;
and obtaining the identification region of the license plate to be identified in the vehicle appearance image according to the corner detection region and the segmentation region.
In an optional embodiment, the apparatus further comprises: a training module;
the training module is used for establishing a license plate position recognition network model to be trained and collecting license plate recognition samples; the license plate identification sample comprises a sample image and a license plate label of the sample image;
and respectively training an angular point detection algorithm and a segmentation algorithm in the license plate position recognition network model to be trained by using the license plate recognition samples to obtain a license plate position recognition network model for recognizing a recognition area in an appearance image of the vehicle.
In an optional embodiment, the license plate labeling of the sample image comprises a target corner detection frame for labeling four corners of a license plate;
the training module is specifically configured to:
processing the sample images in the license plate recognition samples by using an angular point detection algorithm in a license plate position recognition network model to be trained to obtain a plurality of angular point detection frames of the sample images;
matching each corner detection frame of the sample image with each target corner detection frame, and determining an optimal corner detection frame corresponding to the target corner detection frame of each corner;
establishing a corner loss function according to the target corner detection frame of each corner and the corresponding optimal corner detection frame;
and training the corner detection algorithm to be trained by using the corner loss function to obtain the corner detection algorithm which can be used for positioning the license plate position of the license plate to be identified so as to obtain the corner detection area.
In an optional embodiment, the training module is specifically configured to:
calculating the intersection and parallel ratio between each angular point detection frame and each target angular point detection frame;
aiming at each target corner detection frame, determining a plurality of candidate corner detection frames corresponding to each target corner detection frame according to the intersection ratio, wherein the intersection ratio of the candidate corner detection frames meets a preset condition;
and aiming at each candidate corner detection frame corresponding to each target corner detection frame, determining an optimal corner detection frame corresponding to each target corner detection frame by using a non-maximum suppression algorithm.
In an optional embodiment, among the plurality of corner detection boxes for obtaining the sample image, any two corner detection boxes have the same size;
correspondingly, the training module is further specifically configured to:
calculating the distance between the center points of any two corner detection frames in the plurality of corner detection frames for obtaining the sample image so as to obtain the distance between every two corner detection frames;
screening a plurality of corner detection frames of the obtained sample image according to each distance, and taking the corner detection frames remained after screening as the corner detection frames so as to carry out intersection and comparison calculation with the target corner detection frames; and in the corner detection frames reserved after screening, the distance between the center points of any two corner detection frames is greater than a preset distance threshold.
In an optional embodiment, the training module is specifically configured to: performing cross entropy processing on a target corner detection area formed by the target corner detection frame of each corner and an optimal corner detection area formed by each optimal corner detection frame to obtain corner classification loss; carrying out position offset processing on the target corner detection frame of each corner and the optimal corner detection frame corresponding to the target corner detection frame to obtain corner position offset loss; and establishing a corner loss function by the corner classification loss, the corner position offset loss and a preset loss function.
The embodiment of the application provides a license plate recognition device, which obtains a vehicle appearance image by adopting, calls a license plate position recognition network model, determines a license plate to be recognized in a recognition area in the vehicle appearance image by adopting an angular point detection algorithm and a segmentation algorithm, calls a preset recognition network model to recognize an image corresponding to the recognition area, and obtains license plate information in the vehicle appearance image. The license plate position recognition network model is used for positioning the position of the license plate in the vehicle appearance image, and then the image of the recognition area where the positioned license plate is located is recognized, so that the license plate recognition accuracy of vehicles such as trucks and engineering vehicles, wherein the vehicles are coated with license plate information on the vehicle body, can be effectively improved.
The electronic device provided in this embodiment may be used to implement the technical solutions of the above method embodiments, and the implementation principles and technical effects are similar, which are not described herein again.
Referring to fig. 8, a schematic structural diagram of an electronic device 900 suitable for implementing the embodiment of the present disclosure is shown, where the electronic device 900 may be a terminal device or a server. Among them, the terminal Device may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a Digital broadcast receiver, a Personal Digital Assistant (PDA), a tablet computer (PAD), a Portable Multimedia Player (PMP), a car terminal (e.g., car navigation terminal), etc., and a fixed terminal such as a Digital TV, a desktop computer, etc. The electronic device shown in fig. 8 is only one embodiment, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 8, the electronic device 900 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 901, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 902 or a program loaded from a storage means 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data necessary for the operation of the electronic apparatus 900 are also stored. The processing apparatus 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.
Generally, the following devices may be connected to the I/O interface 905: input devices 906 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 907 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 908 including, for example, magnetic tape, hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic apparatus 900 to perform wireless or wired communication with other apparatuses to exchange data. While fig. 8 illustrates an electronic device 900 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device 909, or installed from the storage device 908, or installed from the ROM 902. The computer program performs the above-described functions defined in the methods of the embodiments of the present disclosure when executed by the processing apparatus 901.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of Network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not in some cases constitute a limitation of the unit itself, for example, the first retrieving unit may also be described as a "unit for retrieving at least two internet protocol addresses".
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific embodiments of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims (10)

1. A license plate recognition method is characterized by comprising the following steps:
obtaining a vehicle appearance image;
calling an angular point detection algorithm and a segmentation algorithm in a license plate position recognition network model to determine a recognition area of a license plate to be recognized in the vehicle appearance image;
and calling a preset identification network model to identify the image corresponding to the identification area, and obtaining the license plate information in the vehicle appearance image.
2. The license plate recognition method of claim 1, wherein the step of calling a corner detection algorithm and a segmentation algorithm in a license plate position recognition network model to determine a recognition area of a license plate to be recognized in the vehicle appearance image comprises the steps of:
carrying out feature extraction processing on the vehicle appearance image to obtain image features;
processing the image characteristics by using an angular point detection algorithm and a segmentation algorithm respectively so as to position the license plate position of the license plate to be identified in the vehicle appearance image and obtain a corresponding angular point detection area and a corresponding segmentation area;
and obtaining the identification region of the license plate to be identified in the vehicle appearance image according to the corner detection region and the segmentation region.
3. The license plate recognition method of claim 1, further comprising:
establishing a license plate position recognition network model to be trained, and collecting license plate recognition samples; the license plate identification sample comprises a sample image and a license plate label of the sample image;
and respectively training an angular point detection algorithm and a segmentation algorithm in the license plate position recognition network model to be trained by using the license plate recognition samples to obtain a license plate position recognition network model for recognizing a recognition area in an appearance image of the vehicle.
4. The license plate recognition method of claim 3, wherein the license plate labeling of the sample image comprises a target corner detection box for labeling four corners of the license plate;
the training of the corner point detection algorithm in the license plate position recognition network model to be trained by using the license plate recognition sample comprises the following steps:
processing the sample images in the license plate recognition samples by using an angular point detection algorithm in a license plate position recognition network model to be trained to obtain a plurality of angular point detection frames of the sample images;
matching each corner detection frame of the sample image with each target corner detection frame, and determining an optimal corner detection frame corresponding to the target corner detection frame of each corner;
establishing a corner loss function according to the target corner detection frame of each corner and the corresponding optimal corner detection frame;
and training the corner detection algorithm to be trained by using the corner loss function to obtain the corner detection algorithm which can be used for positioning the license plate position of the license plate to be identified so as to obtain the corner detection area.
5. The license plate recognition method of claim 4, wherein the matching of each corner detection frame of the sample image with each target corner detection frame to determine an optimal corner detection frame corresponding to the target corner detection frame of each corner comprises:
calculating the intersection and parallel ratio between each angular point detection frame and each target angular point detection frame;
aiming at each target corner detection frame, determining a plurality of candidate corner detection frames corresponding to each target corner detection frame according to the intersection ratio, wherein the intersection ratio of the candidate corner detection frames meets a preset condition;
and aiming at each candidate corner detection frame corresponding to each target corner detection frame, determining an optimal corner detection frame corresponding to each target corner detection frame by using a non-maximum suppression algorithm.
6. The license plate recognition method of claim 5, wherein any two corner detection boxes among the plurality of corner detection boxes of the obtained sample image have the same size;
correspondingly, the calculating the intersection-parallel ratio between each corner detection frame and each target corner detection frame includes:
calculating the distance between the center points of any two corner detection frames in a plurality of corner detection frames of the obtained sample image to obtain the distance between every two corner detection frames;
screening a plurality of corner detection frames of the obtained sample image according to each distance, and taking the corner detection frames remained after screening as the corner detection frames so as to carry out intersection and comparison calculation with the target corner detection frames; and in the corner detection frames reserved after screening, the distance between the center points of any two corner detection frames is greater than a preset distance threshold.
7. The license plate recognition method of claim 5, wherein the establishing a corner loss function according to the target corner detection box of each corner and the optimal corner detection box corresponding thereto comprises:
performing cross entropy processing on a target corner detection area formed by the target corner detection frame of each corner and an optimal corner detection area formed by each optimal corner detection frame to obtain corner classification loss;
carrying out position offset processing on the target corner detection frame of each corner and the optimal corner detection frame corresponding to the target corner detection frame to obtain corner position offset loss;
and establishing a corner loss function according to the corner classification loss, the corner position offset loss and a preset loss function.
8. A license plate recognition device, comprising:
a communication module for obtaining a vehicle appearance image;
the license plate positioning module is used for calling an angular point detection algorithm and a segmentation algorithm in the license plate position recognition network model to determine a recognition area of a license plate to be recognized in the vehicle appearance image;
and the license plate recognition module is used for calling a preset recognition network model to recognize the image corresponding to the recognition area and obtaining the license plate information in the vehicle appearance image.
9. An electronic device, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the license plate recognition method of any of claims 1-7.
10. A computer-readable storage medium having computer-executable instructions stored thereon, which when executed by a processor, implement the license plate recognition method of any one of claims 1-7.
CN202010397362.4A 2020-05-12 2020-05-12 License plate recognition method and device, electronic equipment and storage medium Pending CN111582263A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010397362.4A CN111582263A (en) 2020-05-12 2020-05-12 License plate recognition method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010397362.4A CN111582263A (en) 2020-05-12 2020-05-12 License plate recognition method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN111582263A true CN111582263A (en) 2020-08-25

Family

ID=72110818

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010397362.4A Pending CN111582263A (en) 2020-05-12 2020-05-12 License plate recognition method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN111582263A (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111784569A (en) * 2020-09-04 2020-10-16 平安国际智慧城市科技股份有限公司 Method, device, medium, and apparatus for generating paired license plate images and vehicle images
CN112287875A (en) * 2020-11-16 2021-01-29 北京百度网讯科技有限公司 Abnormal license plate recognition method, device, equipment and readable storage medium
CN112580632A (en) * 2020-12-24 2021-03-30 南方电网深圳数字电网研究院有限公司 Nameplate identification method, nameplate identification system, electronic equipment and computer-readable storage medium
CN112580629A (en) * 2020-12-23 2021-03-30 深圳市捷顺科技实业股份有限公司 License plate character recognition method based on deep learning and related device
CN112883973A (en) * 2021-03-17 2021-06-01 北京市商汤科技开发有限公司 License plate recognition method and device, electronic equipment and computer storage medium
CN112997190A (en) * 2020-12-29 2021-06-18 深圳市锐明技术股份有限公司 License plate recognition method and device and electronic equipment
CN113221894A (en) * 2021-05-31 2021-08-06 中邮信息科技(北京)有限公司 License plate number identification method and device of vehicle, electronic equipment and storage medium
CN113255632A (en) * 2021-07-16 2021-08-13 深圳市赛菲姆科技有限公司 Camera parameter adjusting method, device, equipment and medium based on license plate recognition
CN113408509A (en) * 2021-08-20 2021-09-17 智道网联科技(北京)有限公司 Signboard recognition method and device for automatic driving
CN113743218A (en) * 2021-08-03 2021-12-03 科大讯飞股份有限公司 License plate recognition method, license plate recognition device and computer readable storage medium
CN114241465A (en) * 2021-12-14 2022-03-25 广西云高智慧城市科技有限公司 Method for detecting and matching license plate of vehicle image and license plate identification method
CN114333308A (en) * 2021-12-24 2022-04-12 中国联合网络通信集团有限公司 Traffic abnormality recognition method and device, electronic device and storage medium
CN115690767A (en) * 2022-10-26 2023-02-03 北京远度互联科技有限公司 License plate recognition method and device, unmanned aerial vehicle and storage medium
CN117854055A (en) * 2024-03-07 2024-04-09 河南百合特种光学研究院有限公司 Front vehicle distance judging method for license plate recognition and monocular vision of vehicle-mounted camera

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016095763A (en) * 2014-11-17 2016-05-26 住友電工システムソリューション株式会社 Number plate detection device and number plate detection method
CN107730905A (en) * 2017-06-13 2018-02-23 银江股份有限公司 Multitask fake license plate vehicle vision detection system and method based on depth convolutional neural networks
CN108268865A (en) * 2016-12-30 2018-07-10 航天信息股份有限公司 Licence plate recognition method and system under a kind of natural scene based on concatenated convolutional network
CN108960175A (en) * 2018-07-12 2018-12-07 天津艾思科尔科技有限公司 A kind of licence plate recognition method based on deep learning
CN110378317A (en) * 2019-07-29 2019-10-25 广东泓胜科技股份有限公司 A kind of licence plate recognition method and device
CN110378331A (en) * 2019-06-10 2019-10-25 南京邮电大学 A kind of end-to-end Vehicle License Plate Recognition System and its method based on deep learning
CN110390330A (en) * 2019-07-25 2019-10-29 网链科技集团有限公司 Electric bicycle license plate identifying system and method
CN110390251A (en) * 2019-05-15 2019-10-29 上海海事大学 A kind of pictograph semantic segmentation method based on the processing of multiple neural network Model Fusion
CN110543883A (en) * 2019-08-27 2019-12-06 河海大学 license plate recognition method based on deep learning
CN110633731A (en) * 2019-08-13 2019-12-31 杭州电子科技大学 Single-stage anchor-frame-free target detection method based on staggered sensing convolution
CN110796649A (en) * 2019-10-29 2020-02-14 北京市商汤科技开发有限公司 Target detection method and device, electronic equipment and storage medium
CN110910334A (en) * 2018-09-15 2020-03-24 北京市商汤科技开发有限公司 Instance segmentation method, image processing device and computer readable storage medium
CN111028534A (en) * 2018-10-09 2020-04-17 杭州海康威视数字技术股份有限公司 Parking space detection method and device
US20200134331A1 (en) * 2018-10-31 2020-04-30 Texas Instruments Incorporated Object detection for distorted images

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016095763A (en) * 2014-11-17 2016-05-26 住友電工システムソリューション株式会社 Number plate detection device and number plate detection method
CN108268865A (en) * 2016-12-30 2018-07-10 航天信息股份有限公司 Licence plate recognition method and system under a kind of natural scene based on concatenated convolutional network
CN107730905A (en) * 2017-06-13 2018-02-23 银江股份有限公司 Multitask fake license plate vehicle vision detection system and method based on depth convolutional neural networks
CN108960175A (en) * 2018-07-12 2018-12-07 天津艾思科尔科技有限公司 A kind of licence plate recognition method based on deep learning
CN110910334A (en) * 2018-09-15 2020-03-24 北京市商汤科技开发有限公司 Instance segmentation method, image processing device and computer readable storage medium
CN111028534A (en) * 2018-10-09 2020-04-17 杭州海康威视数字技术股份有限公司 Parking space detection method and device
US20200134331A1 (en) * 2018-10-31 2020-04-30 Texas Instruments Incorporated Object detection for distorted images
CN110390251A (en) * 2019-05-15 2019-10-29 上海海事大学 A kind of pictograph semantic segmentation method based on the processing of multiple neural network Model Fusion
CN110378331A (en) * 2019-06-10 2019-10-25 南京邮电大学 A kind of end-to-end Vehicle License Plate Recognition System and its method based on deep learning
CN110390330A (en) * 2019-07-25 2019-10-29 网链科技集团有限公司 Electric bicycle license plate identifying system and method
CN110378317A (en) * 2019-07-29 2019-10-25 广东泓胜科技股份有限公司 A kind of licence plate recognition method and device
CN110633731A (en) * 2019-08-13 2019-12-31 杭州电子科技大学 Single-stage anchor-frame-free target detection method based on staggered sensing convolution
CN110543883A (en) * 2019-08-27 2019-12-06 河海大学 license plate recognition method based on deep learning
CN110796649A (en) * 2019-10-29 2020-02-14 北京市商汤科技开发有限公司 Target detection method and device, electronic equipment and storage medium

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
HEI LAW 等: "CornerNet: Detecting Objects as Paired Keypoints", INTERNATIONAL JOURNAL OF COMPUTER VISION, 8 August 2019 (2019-08-08), pages 642, XP037051068, DOI: 10.1007/s11263-019-01204-1 *
LINJIE DENG 等: "Detecting multi-oriented text with corner-based region proposals", pages 134 - 142 *
NINGNING SUN 等: "Faster R-CNN Based Table Detection Combining Corner Locating", pages 1314 - 1319 *
PENGYUAN LYU 等: "Multi-oriented Scene Text Detection via Corner Localization and Region Segmentation", pages 1 *

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111784569A (en) * 2020-09-04 2020-10-16 平安国际智慧城市科技股份有限公司 Method, device, medium, and apparatus for generating paired license plate images and vehicle images
CN112287875A (en) * 2020-11-16 2021-01-29 北京百度网讯科技有限公司 Abnormal license plate recognition method, device, equipment and readable storage medium
CN112287875B (en) * 2020-11-16 2023-07-25 北京百度网讯科技有限公司 Abnormal license plate recognition method, device, equipment and readable storage medium
CN112580629A (en) * 2020-12-23 2021-03-30 深圳市捷顺科技实业股份有限公司 License plate character recognition method based on deep learning and related device
CN112580632A (en) * 2020-12-24 2021-03-30 南方电网深圳数字电网研究院有限公司 Nameplate identification method, nameplate identification system, electronic equipment and computer-readable storage medium
CN112997190A (en) * 2020-12-29 2021-06-18 深圳市锐明技术股份有限公司 License plate recognition method and device and electronic equipment
CN112997190B (en) * 2020-12-29 2024-01-12 深圳市锐明技术股份有限公司 License plate recognition method and device and electronic equipment
CN112883973A (en) * 2021-03-17 2021-06-01 北京市商汤科技开发有限公司 License plate recognition method and device, electronic equipment and computer storage medium
CN113221894A (en) * 2021-05-31 2021-08-06 中邮信息科技(北京)有限公司 License plate number identification method and device of vehicle, electronic equipment and storage medium
CN113255632B (en) * 2021-07-16 2023-01-03 深圳市赛菲姆科技有限公司 Camera parameter adjusting method, device, equipment and medium based on license plate recognition
CN113255632A (en) * 2021-07-16 2021-08-13 深圳市赛菲姆科技有限公司 Camera parameter adjusting method, device, equipment and medium based on license plate recognition
CN113743218A (en) * 2021-08-03 2021-12-03 科大讯飞股份有限公司 License plate recognition method, license plate recognition device and computer readable storage medium
CN113743218B (en) * 2021-08-03 2024-05-31 科大讯飞股份有限公司 License plate recognition method, license plate recognition device and computer readable storage medium
CN113408509A (en) * 2021-08-20 2021-09-17 智道网联科技(北京)有限公司 Signboard recognition method and device for automatic driving
CN114241465A (en) * 2021-12-14 2022-03-25 广西云高智慧城市科技有限公司 Method for detecting and matching license plate of vehicle image and license plate identification method
CN114241465B (en) * 2021-12-14 2024-08-02 广西云高智慧城市科技有限公司 Method for detecting and matching license plate of vehicle image and license plate recognition method
CN114333308A (en) * 2021-12-24 2022-04-12 中国联合网络通信集团有限公司 Traffic abnormality recognition method and device, electronic device and storage medium
CN115690767B (en) * 2022-10-26 2023-08-22 北京远度互联科技有限公司 License plate recognition method, license plate recognition device, unmanned aerial vehicle and storage medium
CN115690767A (en) * 2022-10-26 2023-02-03 北京远度互联科技有限公司 License plate recognition method and device, unmanned aerial vehicle and storage medium
CN117854055A (en) * 2024-03-07 2024-04-09 河南百合特种光学研究院有限公司 Front vehicle distance judging method for license plate recognition and monocular vision of vehicle-mounted camera

Similar Documents

Publication Publication Date Title
CN111582263A (en) License plate recognition method and device, electronic equipment and storage medium
CN111598091A (en) Image recognition method and device, electronic equipment and computer readable storage medium
CN111127563A (en) Combined calibration method and device, electronic equipment and storage medium
CN110276345B (en) Convolutional neural network model training method and device and computer readable storage medium
CN111783626B (en) Image recognition method, device, electronic equipment and storage medium
CN112258519B (en) Automatic extraction method and device for way-giving line of road in high-precision map making
CN111368668B (en) Three-dimensional hand recognition method and device, electronic equipment and storage medium
CN115409881A (en) Image processing method, device and equipment
CN111382695A (en) Method and apparatus for detecting boundary points of object
CN112712036A (en) Traffic sign recognition method and device, electronic equipment and computer storage medium
CN116543367A (en) Method, device, equipment and medium for generating parking space information based on fisheye camera
CN115100536B (en) Building identification method and device, electronic equipment and computer readable medium
CN114581890B (en) Method and device for determining lane line, electronic equipment and storage medium
CN114429631B (en) Three-dimensional object detection method, device, equipment and storage medium
CN116012814A (en) Signal lamp identification method, signal lamp identification device, electronic equipment and computer readable storage medium
CN115830065A (en) Image-based speed determination method, device, equipment and storage medium
CN111383337B (en) Method and device for identifying objects
CN111681267B (en) Track anti-intrusion method based on image recognition
CN111765892B (en) Positioning method, positioning device, electronic equipment and computer readable storage medium
CN111340813A (en) Image instance segmentation method and device, electronic equipment and storage medium
CN114155545A (en) Form identification method and device, readable medium and electronic equipment
CN112434644A (en) Vehicle image processing method and device, electronic equipment and storage medium
CN113936271A (en) Text recognition method and device, readable medium and electronic equipment
CN114627400A (en) Lane congestion detection method and device, electronic equipment and storage medium
CN111860209B (en) Hand recognition method, device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
AD01 Patent right deemed abandoned

Effective date of abandoning: 20240927

AD01 Patent right deemed abandoned