WO2022141073A1 - 车牌识别方法、装置及电子设备 - Google Patents

车牌识别方法、装置及电子设备 Download PDF

Info

Publication number
WO2022141073A1
WO2022141073A1 PCT/CN2020/140930 CN2020140930W WO2022141073A1 WO 2022141073 A1 WO2022141073 A1 WO 2022141073A1 CN 2020140930 W CN2020140930 W CN 2020140930W WO 2022141073 A1 WO2022141073 A1 WO 2022141073A1
Authority
WO
WIPO (PCT)
Prior art keywords
license plate
image frame
detection
vertex
frame
Prior art date
Application number
PCT/CN2020/140930
Other languages
English (en)
French (fr)
Inventor
林旭南
叶开
王睿
Original Assignee
深圳市锐明技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市锐明技术股份有限公司 filed Critical 深圳市锐明技术股份有限公司
Priority to CN202080003842.6A priority Critical patent/CN112997190B/zh
Priority to PCT/CN2020/140930 priority patent/WO2022141073A1/zh
Priority to US17/555,835 priority patent/US20220207889A1/en
Publication of WO2022141073A1 publication Critical patent/WO2022141073A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/148—Segmentation of character regions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/18—Extraction of features or characteristics of the image
    • G06V30/18162—Extraction of features or characteristics of the image related to a structural representation of the pattern
    • G06V30/18181—Graphical representation, e.g. directed attributed graph
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G06V30/19007—Matching; Proximity measures
    • G06V30/19013—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the present application belongs to the technical field of vehicle information recognition, and in particular, relates to a license plate recognition method, device, electronic device, and computer-readable storage medium.
  • a license plate recognition device is usually installed at the entrance and exit of the parking lot to automatically recognize the license plate of the vehicle entering and leaving.
  • the embodiment of the present application provides a license plate recognition method, which can obtain a more accurate license plate recognition result.
  • an embodiment of the present application provides a license plate recognition method, including:
  • first license plate detection result is used to indicate whether there is a first license plate in the Nth image frame, if there is the first license plate license plate, indicating the position of the first license plate in the Nth image frame, where N is an integer, and N is greater than or equal to 1;
  • the first license plate detection result indicates that there is a first license plate in the Nth image frame, then according to the position of the first license plate in the Nth image frame and the content information of the first license plate, from the Segment the text area and the number area in the first license plate;
  • an embodiment of the present application provides a license plate recognition device, including:
  • the first license plate detection unit is used to perform license plate detection on the Nth image frame in the video stream to obtain a first license plate detection result, and the first license plate detection result is used to indicate whether the Nth image frame has the first license plate detection result.
  • a license plate if the first license plate exists, it indicates the position of the first license plate in the Nth image frame, where N is an integer, and N is greater than or equal to 1;
  • a first license plate content segmentation unit configured to, if the first license plate detection result indicates that a first license plate exists in the Nth image frame, according to the position of the first license plate in the Nth image frame and the The content information of the first license plate, and the text area and the numerical area are divided from the first license plate;
  • the first license plate content recognition unit is configured to recognize the text area and the number area segmented from the first license plate to obtain a first recognition result of the first license plate.
  • an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program A method as described in the first aspect is implemented.
  • an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to the first aspect is implemented.
  • an embodiment of the present application provides a computer program product that, when the computer program product runs on an electronic device, causes the electronic device to execute the method described in the first aspect.
  • the license plate since the license plate is segmented according to the content information of the license plate itself, more accurate text areas and numerical areas of the license plate can be obtained, and after these more accurate text areas and numerical areas are identified, more accurate text areas and numerical areas can be obtained. license plate recognition results.
  • FIG. 1 is a schematic flowchart of a license plate recognition method provided in Embodiment 1 of the present application;
  • FIG. 2 is a schematic flowchart of another license plate recognition method provided in Embodiment 1 of the present application.
  • FIG. 3 is a schematic diagram of performing license plate recognition on a Middle East license plate according to Embodiment 1 of the present application;
  • FIG. 4 is a schematic structural diagram of a license plate recognition device provided in Embodiment 2 of the present application.
  • FIG. 5 is a schematic structural diagram of an electronic device provided in Embodiment 3 of the present application.
  • references in this specification to "one embodiment” or “some embodiments” and the like mean that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the present application.
  • appearances of the phrases “in one embodiment,” “in some embodiments,” “in other embodiments,” “in other embodiments,” etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean “one or more but not all embodiments” unless specifically emphasized otherwise.
  • the inventors of the present application know through analysis that the existing license plate recognition method can only recognize the license plate with a fixed format, which needs to divide the character information on the license plate into a single regular character area, and use a fixed window to identify each character.
  • the units are recognized separately, that is, once the format of the license plate is changed, serious recognition errors will occur.
  • the license plate in the Middle East has a variety of characters, including Arabic numerals, Arabic, etc., and the character layout of the license plate also has a variety of styles. , there is a lot of redundant information, and the characters are small.
  • the embodiment of the present application provides a new license plate recognition method.
  • the license plate recognition method the license plate is segmented according to the content information of the license plate itself to obtain the text area and the numerical area of the license plate, and then The text area and the number area are recognized, and the final license plate recognition result is obtained. Since the license plate is segmented according to the content information of the license plate itself, more accurate text areas and numerical areas of the license plate can be obtained, and after these more accurate text areas and numerical areas are identified, more accurate license plate recognition results can be obtained.
  • Fig. 1 shows a flowchart of a license plate recognition method provided by an embodiment of the present application.
  • first and second in the first license plate and the second license plate are only used to distinguish different
  • the license plate of the image frame has no special meaning, and the rest include “first” and “second” names similar to this, and will not be repeated in the following:
  • Step S11 performing license plate detection on the Nth image frame in the video stream to obtain a first license plate detection result, which is used to indicate whether there is a first license plate in the Nth image frame, and if there is the first license plate detection result.
  • license plate it indicates the position of the first license plate in the Nth image frame, where N is an integer, and N is greater than or equal to 1.
  • the video stream includes multiple image frames, and the Nth image frame in this step is any image frame in the video stream.
  • the Nth image frame represents the video The first image frame in the stream
  • the Nth image frame represents the second image frame in the video stream.
  • the maximum value of N is equal to the number of image frames included in the video stream itself. For example, if the number of image frames included in the video stream is 30 frames, the above-mentioned maximum value of N is 30.
  • the above step S11 specifically includes: performing license plate detection on the Nth image frame in the video stream by using the first target detection model to obtain a first license plate detection result.
  • the position of the first license plate in the Nth image frame and the corresponding confidence level are obtained.
  • the confidence is large, such as greater than a preset confidence threshold, it indicates that the first license plate detection result for the first license plate output by the first target detection model has a high degree of confidence, and the corresponding position is transmitted to the next algorithm.
  • the position of the first license plate in the Nth image frame can be represented by the detection frame of the rectangle positioned by the coordinates of the upper left (x1, y1) and lower right (x2, y2) points of the license plate, or by four corner points The coordinates are represented by a polygonal box.
  • the first target detection model may be a one-stage target detection model, and the one-stage target detection model includes but is not limited to target detection models formed by target detection algorithms such as YOLO and SSD.
  • the above-mentioned first target detection model is a model obtained by training the second target detection model, and the first target detection model is a model with a neural network.
  • the second target detection model is trained in the following manner:
  • the image captured by the camera is acquired, the coordinates of the license plate in the image are manually marked to obtain the corresponding training label, and the image captured by the camera with the training label is used to train the second target detection model to obtain the first target detection model.
  • the countries to which the license plates that can be recognized by the obtained first target detection model belong are also different. For example, when the country to which the license plate belongs is China, the country to which the license plate can be identified by the obtained first target detection model is China.
  • the regions to which the license plates belong are all North America, the region to which the license plates that can be identified by the obtained first target detection model belong is North America.
  • the obtained first target detection model can also include multiple countries to which the license plate belongs. That is, the second target detection model is trained by using the license plates mixed with different countries, so that the obtained The first target detection model can identify license plates corresponding to different countries.
  • Step S12 if the first license plate detection result indicates that there is a first license plate in the Nth image frame, then according to the position of the first license plate in the Nth image frame and the content information of the first license plate, from the first license plate In the license plate, the text area and the number area are divided.
  • the content information of the first license plate refers to the text information and numerical information contained in the first license plate, and the location of the text information and the numerical information in the first license plate.
  • a license plate in addition to the digital information, there is also text information. Since the recognition of the digital information and the recognition of the text information are different, in the embodiment of the present application, it is necessary to separate the text area and the number from the first license plate respectively. area, so that an accurate recognition result can be obtained by recognizing the characters in the text area and the numbers in the number area subsequently.
  • the first license plate detection result indicates that the Nth image frame does not have the first license plate, then continue to perform license plate detection on the next image frame of the Nth image frame.
  • Step S13 identifying the text area and the number area segmented from the first license plate to obtain a first recognition result of the first license plate.
  • the first recognition result of the first license plate is obtained by recognizing the segmented text area and number area, and the first recognition result includes city information and license plate number information of the first license plate.
  • the license plate since the license plate is segmented according to the content information of the license plate itself, more accurate text areas and numerical areas of the license plate can be obtained, and after these more accurate text areas and numerical areas are identified, more accurate text areas and numerical areas can be obtained. license plate recognition results.
  • the above step S13 specifically includes: using a first license plate recognition model to recognize the text area and the number area segmented from the first license plate to obtain a first recognition result of the first license plate.
  • the above-mentioned first license plate recognition model is a model obtained after training the second license plate recognition model
  • the first license plate recognition model is a model with a neural network.
  • the second license plate recognition model is trained in the following manner:
  • the segmented license plate image input to the second license plate recognition model and obtain the corresponding training label by manually labeling or synthesizing the license plate content string.
  • the above-mentioned license plate content string includes characters and numbers.
  • the second license plate recognition model is trained by using the segmented license plate images and training labels to obtain the first license plate recognition model.
  • FIG. 2 shows a flowchart of another license plate recognition method provided by an embodiment of the present application.
  • step S21, Steps S22 and S23 are the same as the above-mentioned steps S11, S12, and S13, and will not be repeated here:
  • Step S21 performing license plate detection on the Nth image frame in the video stream to obtain a first license plate detection result, which is used to indicate whether there is a first license plate in the Nth image frame, and if there is the first license plate detection result.
  • license plate it indicates the position of the first license plate in the Nth image frame, where N is an integer, and N is greater than or equal to 1.
  • Step S22 if the first license plate detection result indicates that there is a first license plate in the Nth image frame, then according to the position of the first license plate in the Nth image frame and the content information of the first license plate, from the first license plate In the license plate, the text area and the number area are divided.
  • Step S23 Identify the text area and the number area segmented from the first license plate to obtain a first recognition result of the first license plate.
  • Step S24 respectively perform license plate detection on the M image frames in the video stream, and obtain M second license plate detection results, and the second license plate detection results are used to indicate whether the image frames for license plate detection in the M image frames exist
  • the second license plate if the second license plate exists, it indicates the position of the second license plate in the image frame where the license plate is detected in the M image frames, and the M image frames are all image frames after the Nth image frame , M is greater than or equal to 1.
  • the license plate detection when M is equal to 2, the license plate detection will be performed on the 2 image frames (such as the N+1 th image frame and the N+2 th image frame), and the license plate detection and recognition will be performed on each image frame.
  • the process is similar to the process of license plate detection and recognition for the Nth image frame, and will not be repeated here.
  • Step S25 if there is at least one target license plate detection result in the M second license plate detection results, then perform license plate detection in the M image frames according to at least one second license plate indicated by the at least one target license plate detection result.
  • the position of the frame and the content information of the at least one second license plate are respectively divided into a text area and a numerical area from the at least one second license plate, wherein the target license plate detection result means that the second license plate is included in the M.
  • the obtained second license plate detection result is M1
  • the M1 indicates that the N+1 th image frame has a second license plate m1.
  • the obtained second license plate detection result is M2, which indicates that there is no second license plate in the N+1 th image frame
  • M1 is the target license plate detection result. According to the position of m1 in the N+1 th image frame and the content information of the m1, the text area and the number area are segmented from m1.
  • Step S26 Recognize the text area and the number area respectively segmented from the at least one second license plate to obtain at least one second recognition result of the at least one second license plate.
  • the text area and the numerical area of two second license plates (the second license plate m1 and the second license plate m2) need to be recognized, then the text area of the second license plate m1 and the second license plate m1 are first identified. Then, the text area of m2 of the second license plate and the digital area of the second license plate m2 are recognized to obtain another second recognition result.
  • Step S27 according to the position of the image frame of the at least one second license plate in the M image frames for license plate detection and the position of the first license plate in the Nth image frame, respectively determine the at least one second license plate and the Whether a license plate matches.
  • the first license plate and the second license plate match, it means that the first license plate and the second license plate are the same license plate; if the first license plate and the second license plate do not match, it means that the first license plate and the second license plate are the same.
  • the license plates are not the same license plate. Specifically, the position of the first license plate and the position of the second license plate can be compared, and if the position changes of the two in adjacent image frames are small, it is determined that the first license plate and the second license plate match, otherwise, it is determined that the first license plate matches the second license plate. The first license plate and the second license plate do not match.
  • step S27 may also be performed after step S24 or after step S25, which is not limited here.
  • Step S28 Determine the output license plate recognition result according to the first recognition result of the first license plate and the target recognition result, wherein the target recognition result refers to the second recognition result corresponding to the second license plate matching the first license plate.
  • the output license plate recognition result can be determined according to the confidence of the position of the first license plate in the first recognition result and the confidence of the position of the second license plate in the target recognition result. Or, by combining the information in the first recognition result and the target recognition result, for example, selecting part of the information in the first recognition result, and selecting part of the information in the target recognition result, and then combining the selected 2 parts of information , to determine the output license plate recognition result.
  • the output license plate recognition result is determined according to the recognition results of the same license plate in the adjacent image frames. That is, by adding the recognition results of the same license plate in other image frames to determine the final license plate recognition result, the accuracy of the obtained license plate recognition result can be improved.
  • this step S28 includes:
  • A1 Split the first recognition result according to a preset output format to obtain first split content, where the first split content includes at least two split sub-contents, each of which corresponds to a confidence level .
  • the second split contents include at least two split sub-contents, each split sub-content corresponds to a confidence level.
  • A3. Accumulate the confidence of the first split content and the at least two second split contents with the same split sub-content respectively, and select the split sub-content with high confidence after accumulation according to the preset output format, respectively, And form the output license plate recognition result.
  • the first recognition result and the target recognition result are split according to the preset output format, so that different types of license plates are summarized under the same structural framework, and then the split sub-content and Corresponding confidence level, determine the accumulated confidence level corresponding to the same split sub-content, when the preset output format is at the same position, the higher the accumulated confidence level corresponding to the corresponding split sub-content, it indicates that the position is The higher the accumulated confidence, the higher the probability of splitting the sub-content. Therefore, according to the preset output format, the split sub-content with high confidence after accumulation is selected, and the output license plate recognition result corresponding to the accurate The higher the degree is (that is, the license plate recognition result is output according to the voting mechanism).
  • the split sub-content corresponding to the first recognition result is “DUBAI” + “I 55555”
  • the corresponding confidence levels are “0.6", “0.7”
  • the split sub-content corresponding to target recognition result 1 is “DUBAI” + “I 55556”
  • the corresponding confidence levels are "0.7", “0.6”
  • the split sub-content corresponding to target recognition result 2 is "DUBAL” respectively +”I 55555”
  • the corresponding confidence levels are "0.5” and "0.5” respectively
  • the corresponding accumulated confidence level is "0.5".
  • step S27 includes:
  • M image frame teams are selected from the N th image frame to the N+M th image frame, and one of the image frame teams is two adjacent image frames.
  • license plate detection is also performed on the M image frames following the Nth image frame, that is, the Nth image frame to the Nth image frame is also subjected to license plate detection.
  • +M image frames for license plate detection From the (M+1) image frames, two adjacent image frames are divided into one image frame team for dividing M image frame teams.
  • the above-mentioned B2 is repeatedly executed until the first license plate and the second license plate in any image frame team in the M image frame teams have been matched.
  • the first image frame and the second image frame in the video stream are divided into an image frame team (assumed to be image frame team 1), and the second image frame and The third image frame is divided into an image frame team (assuming it is image frame team 2), according to the position of the second license plate in the second image frame and the position of the first license plate in the first image frame to determine the first license plate and Whether the second license plate matches. After that, it is judged whether the first license plate and the second license plate match according to the position of the second license plate in the third image frame and the position of the first license plate in the second image frame.
  • the second license plate above refers to the image frame located at the back of the video stream in the image frame team.
  • the second image frame is the image frame located at the back of the video stream.
  • the second image frame becomes the image frame located in front of the video stream.
  • the probability that the first license plate and the second license plate are the same license plate is high.
  • determining whether the first license plate and the second license plate match according to the position of the second license plate in the image frame team in the image frame and the position of the first license plate in the image frame in the B2, including:
  • the detection frame R i is any element in the sequence S 1
  • the detection frame R j is any element in the sequence S 2
  • the sequence S 1 is composed of the detection frame of the first license plate
  • the sequence S 2 is composed of the The detection frame of the second license plate is formed, and the position of the first license plate in the Nth image frame and the position of the second license plate in the N+1th image frame are represented by corresponding detection frames.
  • two sets formed by detection frames of two adjacent image frames are initialized, one set is used to store the detection frame of the first license plate, and the other set is used to store the detection frame of the second license plate.
  • the detection frame R i and the detection frame R j are repeatedly taken out from the two sequences S 1 and S 2 , and the intersection over union ratio (Intersection over Union, IoU) of the detection frame R i and the detection frame R j is calculated, and the IoU is two The ratio of intersection and union of detection boxes, where:
  • intersection is compared as the weight of the edge connecting the detection frame R1 and the detection frame R2.
  • the above-mentioned first preset value may be a value less than 0.5, for example, 0.
  • the corresponding edges are matched for each vertex in the sequence S 1 one by one. Since the intersection ratio is used as the weight of the edge connecting the detection frame R1 and the detection frame R2 , the greater the weight of the edge connecting the detection frame R1 and the detection frame R2, it indicates that the detection frame R1 and the detection frame R2 have a higher weight.
  • the Hungarian algorithm or the KM algorithm can be used to perform multi-target (the target is the first license plate and the second license plate) in each image frame. Data association to form an optimal match. Further, a unique identification (Identity document, ID) of the corresponding license plate is established for the detection frames of different image frames obtained by matching.
  • the first license plates corresponding to the vertices in the sequence S1 may be matched in the order of each detection frame in the sequence S1 (eg, selecting the vertex X from front to back, or from back to front).
  • the first license plate corresponding to the vertex X in the sequence S1 fails to match, including:
  • the second preset value is greater than 0. Since the second preset value is greater than 0, after subtracting the second preset value from the weight value of the vertex X, the remaining weight value will be smaller than the original weight value.
  • the matching principle is: only match the edge with the same weight as the weight of the left vertex (that is, the above initialization is assigned to the left vertex), and if no matching edge is found, this path corresponds to the left vertex
  • the value of is minus d
  • the value of the right vertex is plus d
  • the matching of the next vertex in the left sequence is continued.
  • the weights of the vertices corresponding to the detection frame of the first license plate are reduced, and the vertices with the reduced weights continue to be matched subsequently.
  • stop the vertex after the weight reduction when the vertex X fails to match, it means that the detection of the first license plate corresponding to the vertex X originally appeared in the image frame
  • the box no longer appears in subsequent image frames, indicating that the first license plate corresponding to this vertex X may have been removed from view). That is, by gradually reducing the weights of the vertices, the probability of finding a matching edge can be increased.
  • this step S13 includes:
  • the text area and number area in the license plate are combined into information to be identified in a fixed format.
  • the corresponding license plate types are very rich, and the structure and layout of the license plate are different: there are single-layer license plates, as well as double-layer and multi-layer license plates.
  • the distribution positions of the text parts in the license plate are also different, which makes it difficult to recognize the license plate.
  • the segmented parts of the license plate are spliced according to a fixed format, for example, according to the fixed format of the letters on the left and the numbers on the right, to ensure that different license plates are all single-layer structures before they are input into the first license plate recognition model.
  • This can further unify the input data type and simplify the problem, so that the first license plate recognition model can obtain higher accuracy, and can also obtain better transferability in license plate recognition in different countries.
  • step S12 according to the position of the first license plate in the Nth image frame and the content information of the first license plate, segment the text area and the number from the first license plate area, including:
  • C1. Determine the format of the first license plate according to the content information of the first license plate.
  • the format of the first license plate is used to respectively indicate the position of the text area and the numerical area of the first license plate in the area of the first license plate.
  • the correspondence between the content information of different license plates and the format of the license plate is pre-stored, and after the content information of the license plate is obtained, the format of the license plate corresponding to the content information of the license plate is determined according to the stored correspondence.
  • the subsequent The format can quickly extract the text area and number area of the license plate.
  • step S12 according to the position of the first license plate in the Nth image frame and the content information of the first license plate, segment the text area and the number from the first license plate area, including:
  • the first license plate image is an image corresponding to the region of the first license plate intercepted from the Nth image frame
  • the number of pixels in the first license plate image will be much smaller than the Nth image.
  • the number of pixels in the frame reduces the number of pixels that need to be processed subsequently, thereby saving the resources of the electronic device.
  • D2. Perform at least one of the following processing on the first license plate image: correction processing, image enhancement processing, denoising processing, deblurring processing and normalization processing, wherein the correction processing is used to The license plate image is corrected into a flat first license plate image, and the normalization process is used to process the pixel value range of the first license plate image into a standardized distribution.
  • the corrected image can increase the effective pixel area.
  • image enhancement processing refers to adding some information or transforming data to the original image by certain means, selectively highlighting interesting features in the original image or suppressing (masking) some unwanted features in the original image, so that the processed image can be enhanced. Images are matched to visual response properties.
  • the existing image enhancement algorithm can be used to realize the image enhancement processing.
  • deblurring can reduce ghosting caused by motion blur and make the license plate clearer.
  • the normalization process can make the pixel value range of the license plate to be distributed in a standardized manner, which meets the processing requirements of the neural network.
  • the processed first license plate image can make the license plate easier to identify, the corresponding text area and number area can be more accurately segmented from the processed first license plate image.
  • the D3 includes:
  • the pixel-level text area and number area are segmented from the processed first license plate image.
  • the semantic segmentation model is used to segment different regions of the processed first license plate image, distinguish which positions correspond to province information, need to identify text, which positions correspond to license plate number information, and need to identify numbers, etc.
  • the semantic segmentation model is a neural network model, which needs to be trained with tens of millions of data before application.
  • the training data is the position of the license plate detected by the first target detection model in the image frame, and the training labels are different areas after the license plate is manually segmented, and these different areas include text areas and numbers.
  • the trained semantic segmentation model is able to segment pixel-level text regions and digit regions from license plate images.
  • the semantic segmentation model is used to perform pixel-level classification, prediction, and inference labels for city information (such as Arabic city information) to achieve fine-grained reasoning, so that each pixel is labeled as the category of its enclosed area, and the learning
  • city information such as Arabic city information
  • the obtained recognition features are semantically projected onto the pixel space (high resolution) to obtain dense classification and output the final city information result.
  • FIG. 3 shows a schematic diagram of performing license plate recognition by using the license plate recognition provided by an embodiment of the present application.
  • the first target detection model adopts the One-stage target detection model, and the multi-target detection algorithm, such as the Hungarian algorithm or the KM algorithm, is specifically used when matching the first license plate and the second license plate of the image frame team.
  • the semantic segmentation model is used to segment the text area and the number area from the first license plate, and, from the second license plate, 2 text areas (i.e. city information) and a number area (that is, the specific license plate number information), wherein the two text areas are: the first text area and the second text area, and the information in the first text area is the English word "DUBAI" of "Dubai” , the second text area is the Arabic equivalent of Dubai.
  • the first text area, the second text area and the number area are spliced together according to the format of the characters on the left and the numbers on the right, and then the above-mentioned spliced information is identified through the end-to-end recognition model (ie, the first license plate recognition model above), to obtain License plate recognition results and output.
  • the end-to-end recognition model ie, the first license plate recognition model above
  • Embodiment 2 is a diagrammatic representation of Embodiment 1:
  • FIG. 4 shows a structural block diagram of the license plate recognition device provided by the embodiment of the present application.
  • the license plate recognition device can be applied to electronic equipment, and the electronic equipment can be a server or a terminal device. , for the convenience of description, only the parts related to the embodiments of the present application are shown.
  • the license plate recognition device 4 includes: a first license plate detection unit 41 , a first license plate detection unit 42 , and a first license plate content recognition unit 43 . in:
  • the first license plate detection unit 41 is configured to perform license plate detection on the Nth image frame in the video stream to obtain a first license plate detection result, which is used to indicate whether the Nth image frame has a first license plate , if the first license plate exists, it indicates the position of the first license plate in the Nth image frame, where N is an integer, and N is greater than or equal to 1.
  • the first license plate content segmentation unit 42 is used for if the first license plate detection result indicates that the Nth image frame has a first license plate, then according to the position of the first license plate in the Nth image frame and the first license plate. Content information, segment the text area and the number area from the first license plate.
  • the first license plate content recognition unit 43 is configured to recognize the text area and the number area segmented from the first license plate to obtain a first recognition result of the first license plate.
  • the license plate since the license plate is segmented according to the content information of the license plate itself, more accurate text areas and numerical areas of the license plate can be obtained, and after these more accurate text areas and numerical areas are identified, more accurate text areas and numerical areas can be obtained. license plate recognition results.
  • the license plate recognition device 4 further includes:
  • the second license plate detection unit is configured to perform license plate detection on the M image frames in the video stream, respectively, to obtain M second license plate detection results, and the second license plate detection results are used to instruct the M image frames to perform license plate detection. Whether there is a second license plate in the image frame of Image frame after frame, M is greater than or equal to 1.
  • the second license plate content segmentation unit is configured to: if there is at least one target license plate detection result in the M second license plate detection results, then according to the at least one second license plate indicated by the at least one target license plate detection result in the M image frames
  • the position of the image frame in which the license plate is detected and the content information of the at least one second license plate are respectively divided into a text area and a numerical area from the at least one second license plate, wherein the target license plate detection result means that the target license plate detection result includes an indication of the The license plate detection result of the position of the image frame where the license plate detection is performed for the second license plate in the M image frames.
  • the second license plate content recognition unit is configured to recognize the text area and the numerical area respectively segmented from the at least one second license plate, and obtain at least one second recognition result of the at least one second license plate.
  • the license plate matching unit is used to judge the at least one second license plate according to the position of the image frame of the at least one second license plate in the M image frames for license plate detection and the position of the first license plate in the Nth image frame. Whether it matches the first license plate.
  • the license plate recognition result determination unit is used to determine the output license plate recognition result according to the first recognition result of the first license plate and the target recognition result, wherein the target recognition result refers to the corresponding second license plate matching the first license plate.
  • the second recognition result refers to the corresponding second license plate matching the first license plate.
  • the license plate recognition result determination unit includes:
  • a first identification result splitting module configured to split the first identification result according to a preset output format to obtain first split content, where the first split content includes at least two split sub-contents, each of which is Splitting subcontent corresponds to a confidence level.
  • a target recognition result splitting module configured to split at least two of the target recognition results respectively according to a preset output format to obtain at least two second split contents, where the second split contents include at least two split subs content, each split sub-content corresponds to a confidence level.
  • a confidence accumulation module used for accumulating the confidence of the first split content and the at least two second split contents with the same split sub-content, respectively selecting the accumulated confidence according to the preset output format
  • the split sub-content of and compose the output license plate recognition result.
  • the license plate matching unit includes:
  • the image frame team determining module is configured to select M image frame teams from the N th image frame to the N+M th image frame, and one of the image frame teams is two adjacent image frames.
  • the license plate matching module of the image frame team is used for any image frame team in the M image frame teams, according to the position of the second license plate in the image frame team in the image frame and the position of the first license plate in the image frame.
  • the location determines whether the first license plate and the second license plate match.
  • the execution of any image frame team in the M image frame teams is repeated until the first license plate and the second license plate in any image frame team in the M image frame teams have been matched.
  • the license plate matching module of the image frame team determines the first license plate and the second license plate according to the position of the second license plate in the image frame in the image frame and the position of the first license plate in the image frame. When the license plate matches, it is used for:
  • the detection frame R i is any element in the sequence S 1
  • the detection frame R j is any element in the sequence S 2
  • the sequence S 1 is composed of the detection frame of the first license plate
  • the sequence S 2 is composed of the The detection frame of the second license plate is formed, and the position of the first license plate in the Nth image frame and the position of the second license plate in the N+1th image frame are represented by corresponding detection frames.
  • the intersection is compared as the weight of the edge connecting the detection frame R1 and the detection frame R2.
  • the weight of each vertex in sequence S1 is the edge connected to its corresponding detection frame
  • the maximum weight of , the weight of each vertex in the sequence S2 is a first preset value, and the first preset value is less than 0.5.
  • For the vertex X in the sequence S1 find the edge with the same weight as the vertex X from the sequence S2. If the edge with the same weight as the vertex X is found, then the sequence S1 is determined. The first license plate corresponding to the vertex X in the sequence S 1 is successfully matched. If no edge with the same weight as the vertex X is found, it is determined that the first license plate corresponding to the vertex X in the sequence S 1 fails to match, wherein , and vertex X is any vertex in the sequence S1.
  • the license plate matching module of the image frame team when the license plate matching module of the image frame team does not find an edge with the same weight as the vertex X, it is determined that the first license plate corresponding to the vertex X in the sequence S1 fails to match. , specifically for:
  • the first license plate content identification unit 43 is specifically used for:
  • the first license plate content segmentation unit 42 divides the text area and numbers from the first license plate according to the position of the first license plate in the Nth image frame and the content information of the first license plate area, specifically for:
  • the format of the first license plate is determined according to the content information of the first license plate, and the format of the first license plate is used to respectively indicate the position of the text area and the numerical area of the first license plate in the area of the first license plate. According to the position of the first license plate in the Nth image frame and the format of the first license plate, a text area and a number area are segmented from the first license plate.
  • the first license plate content segmentation unit 42 divides the text area and numbers from the first license plate according to the position of the first license plate in the Nth image frame and the content information of the first license plate area, specifically for:
  • the first license plate image is obtained by intercepting the area of the first license plate from the Nth image frame according to the position of the first license plate in the Nth image frame. At least one of the following processing is performed on the first license plate image: correction processing, image enhancement processing, denoising processing, deblurring processing and normalization processing, wherein the correction processing is used to transform the first license plate image with angular deflection Corrected into a flat first license plate image, the normalization process is used to process the pixel value range of the first license plate image into a standardized distribution. A text area and a number area are segmented from the processed first license plate image.
  • the first license plate content segmentation unit 42 is specifically configured to: when segmenting the text area and the number area from the processed first license plate image:
  • the pixel-level text area and number area are segmented from the processed first license plate image.
  • FIG. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
  • the electronic device may be a server or a terminal device.
  • the electronic device 5 in this embodiment includes: at least one processor 50 (only one processor is shown in FIG. 5 ), a memory 51 , and a storage device stored in the memory 51
  • a computer program 52 that can be run on the at least one processor 50, and the processor 50 implements the steps in any of the above method embodiments when the processor 50 executes the computer program 52:
  • the first license plate detection result is used to indicate whether there is a first license plate in the Nth image frame. If the first license plate exists, then Indicates the position of the first license plate in the Nth image frame, where N is an integer, and N is greater than or equal to 1.
  • the first license plate detection result indicates that there is a first license plate in the Nth image frame
  • the license plate recognition method further includes:
  • M image frames are the image frames after the Nth image frame, and M is greater than or equal to 1.
  • this target license plate detection result means to include and indicate that this second license plate is in this M image frame The result of the license plate detection at the position of the image frame where the license plate detection is performed.
  • the position of the at least one second license plate in the image frame where the license plate is detected in the M image frames and the position of the first license plate in the Nth image frame it is respectively determined whether the at least one second license plate and the first license plate are not match.
  • the output license plate recognition result is determined according to the first recognition result of the first license plate and the target recognition result, wherein the target recognition result refers to the second recognition result corresponding to the second license plate matching the first license plate.
  • the output license plate recognition result determined according to the first recognition result of the first license plate and the target recognition result including:
  • the first identification result is split according to a preset output format to obtain first split content, where the first split content includes at least two split sub-contents, and each of the split sub-contents corresponds to a confidence level.
  • the second split contents include at least two split sub-contents, each split sub-content and a Corresponding confidence.
  • the position of the image frame for license plate detection in the M image frames according to the at least one second license plate and the first license plate Determine whether the at least one second license plate matches the first license plate at the position of the Nth image frame, including:
  • M image frame teams are selected from the N th image frame to the N+M th image frame, and one of the image frame teams is two adjacent image frames.
  • the first license plate and the second license plate determine the first license plate and the second license plate according to the position of the second license plate in the image frame and the position of the first license plate in the image frame. Whether the license plate matches.
  • determining whether the first license plate and the second license plate match according to the position of the second license plate in the image frame in the image frame and the position of the first license plate in the image frame including:
  • the detection frame R i is any element in the sequence S 1
  • the detection frame R j is any element in the sequence S 2
  • the sequence S 1 is composed of the detection frame of the first license plate
  • the sequence S 2 is composed of the The detection frame of the second license plate is formed, and the position of the first license plate in the Nth image frame and the position of the second license plate in the N+1th image frame are represented by corresponding detection frames.
  • intersection is compared as the weight of the edge connecting the detection frame R1 and the detection frame R2.
  • the weight of each vertex in sequence S1 is the edge connected to its corresponding detection frame
  • the maximum weight of , the weight of each vertex in the sequence S2 is a first preset value, and the first preset value is less than 0.5.
  • the vertex X in the sequence S1 For the vertex X in the sequence S1 , find the edge with the same weight as the vertex X from the sequence S2. If the edge with the same weight as the vertex X is found, then the sequence S1 is determined. The first license plate corresponding to the vertex X in the sequence S 1 is successfully matched. If no edge with the same weight as the vertex X is found, it is determined that the first license plate corresponding to the vertex X in the sequence S 1 fails to match, wherein , and vertex X is any vertex in the sequence S1.
  • the edge with the same weight as that of the vertex X is not found, it is determined that the first license plate corresponding to the vertex X in the sequence S1 fails to match, including:
  • the text area and the digital area segmented from the first license plate are identified to obtain a first recognition result of the first license plate, including:
  • segment a text area and a numerical area from the first license plate including:
  • the format of the first license plate is determined according to the content information of the first license plate, and the format of the first license plate is used to respectively indicate the position of the text area and the numerical area of the first license plate in the area of the first license plate.
  • a text area and a number area are segmented from the first license plate.
  • segment a text area and a numerical area from the first license plate including:
  • the first license plate image is obtained by intercepting the area of the first license plate from the Nth image frame according to the position of the first license plate in the Nth image frame.
  • At least one of the following processing is performed on the first license plate image: correction processing, image enhancement processing, denoising processing, deblurring processing and normalization processing, wherein the correction processing is used to transform the first license plate image with angular deflection Corrected into a flat first license plate image, the normalization process is used to process the pixel value range of the first license plate image into a standardized distribution.
  • a text area and a number area are segmented from the processed first license plate image.
  • segmenting a text area and a number area from the processed first license plate image including:
  • the pixel-level text area and number area are segmented from the processed first license plate image.
  • the electronic device 5 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server.
  • the electronic device may include, but is not limited to, a processor 50 and a memory 51 .
  • FIG. 5 is only an example of the electronic device 5, and does not constitute a limitation on the electronic device 5. It may include more or less components than the one shown, or combine some components, or different components , for example, may also include input and output devices, network access devices, and the like.
  • the so-called processor 50 may be a central processing unit (Central Processing Unit, CPU), and the processor 50 may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuits) , ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the memory 51 may be an internal storage unit of the electronic device 5 in some embodiments, such as a hard disk or a memory of the electronic device 5 .
  • the memory 51 may also be an external storage device of the electronic device 5, such as a plug-in hard disk equipped on the electronic device 5, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc.
  • the memory 51 may also include both an internal storage unit of the electronic device 5 and an external storage device.
  • the memory 51 is used to store an operating system, an application program, a boot loader (Boot Loader), data, and other programs, such as program codes of the computer program, and the like.
  • the memory 51 can also be used to temporarily store data that has been output or will be output.
  • An embodiment of the present application also provides a network device, the network device includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, the processor executing The computer program implements the steps in any of the foregoing method embodiments.
  • Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented.
  • the embodiments of the present application provide a computer program product, when the computer program product runs on a mobile terminal, the steps in the foregoing method embodiments can be implemented when the mobile terminal executes the computer program product.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium.
  • all or part of the processes in the methods of the above embodiments can be implemented by a computer program to instruct the relevant hardware.
  • the computer program can be stored in a computer-readable storage medium, and the computer program When executed by a processor, the steps of each of the above method embodiments can be implemented.
  • the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form, and the like.
  • the computer-readable medium may include at least: any entity or device capable of carrying the computer program code to the photographing device/electronic device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signals, telecommunication signals, and software distribution media.
  • ROM read-only memory
  • RAM random access memory
  • electrical carrier signals telecommunication signals
  • software distribution media For example, U disk, mobile hard disk, disk or CD, etc.
  • computer readable media may not be electrical carrier signals and telecommunications signals.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Traffic Control Systems (AREA)
  • Character Input (AREA)
  • Character Discrimination (AREA)
  • Image Analysis (AREA)

Abstract

本申请适用于车辆信息识别技术领域，提供了车牌识别方法、装置及电子设备，包括：对视频流中的第N个图像帧进行车牌检测，得到第一车牌检测结果，所述第一车牌检测结果用于指示所述第N个图像帧是否存在第一车牌，若存在所述第一车牌，则指示所述第一车牌在所述第N个图像帧的位置，其中，N为整数，N大于或等于1；若所述第一车牌检测结果指示所述第N个图像帧存在第一车牌，则根据所述第一车牌在所述第N个图像帧的位置以及所述第一车牌的内容信息，从所述第一车牌中分割出文字区域和数字区域；对从所述第一车牌中分割出的所述文字区域和所述数字区域进行识别，得到所述第一车牌的第一识别结果。通过上述方法，能够得到准确的车牌识别结果。

Description

车牌识别方法、装置及电子设备 技术领域
本申请属于车辆信息识别技术领域，尤其涉及车牌识别方法、装置、电子设备及计算机可读存储介质。
背景技术
为了便于用户快速出行，比如，为了便于用户快速进、出停车场，通常会在停车场的出入口设置车牌识别装置，以自动识别进出车辆的车牌。
现有的车牌识别方法在进行车牌识别时，有时候得到的是错误的识别结果。
技术解决方案
本申请实施例提供了车牌识别方法，能够得到更准确的车牌识别结果。
为解决上述技术问题，本申请实施例采用的技术方案是：
第一方面，本申请实施例提供了一种车牌识别方法，包括：
对视频流中的第N个图像帧进行车牌检测，得到第一车牌检测结果，所述第一车牌检测结果用于指示所述第N个图像帧是否存在第一车牌，若存在所述第一车牌，则指示所述第一车牌在所述第N个图像帧的位置，其中，N为整数，N大于或等于1；
若所述第一车牌检测结果指示所述第N个图像帧存在第一车牌，则根据所述第一车牌在所述第N个图像帧的位置以及所述第一车牌的内容信息，从所述第一车牌中分割出文字区域和数字区域；
对从所述第一车牌中分割出的所述文字区域和所述数字区域进行识别，得到所述第一车牌的第一识别结果。
第二方面，本申请实施例提供了一种车牌识别装置，包括：
第一车牌检测单元，用于对视频流中的第N个图像帧进行车牌检测，得到第一车牌检测结果，所述第一车牌检测结果用于指示所述第N个图像帧是否存在第一车牌，若存在所述第一车牌，则指示所述第一车牌在所述第N个图像帧的位置，其中，N为整数，N大于或等于1；
第一车牌内容分割单元，用于若所述第一车牌检测结果指示所述第N个图像帧存在第一车牌，则根据所述第一车牌在所述第N个图像帧的位置以及所述第一车牌的内容信息，从所述第一车牌中分割出文字区域和数字区域；
第一车牌内容识别单元，用于对从所述第一车牌中分割出的所述文字区域和所述数字区域进行识别，得到所述第一车牌的第一识别结果。
第三方面，本申请实施例提供了一种电子设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如第一方面所述的方法。
第四方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现如第一方面所述的方法。
第五方面，本申请实施例提供了一种计算机程序产品，当计算机程序产品在电子设备上运行时，使得电子设备执行上述第一方面所述的方法。
有益效果
本申请实施例中，由于根据车牌本身内容信息对车牌进行分割，因此能够得到该车牌更准确的文字区域和数字区域，进而对这些更准确的文字区域和数字区域进行识别后，能够得到更准确的车牌识别结果。
可以理解的是，上述第二方面至第五方面的有益效果可以参见上述第一方面中的相关 描述，在此不再赘述。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍。
图1是本申请实施例一提供的一种车牌识别方法的流程示意图；
图2是本申请实施例一提供的另一种车牌识别方法的流程示意图；
图3是本申请实施例一提供的一种对中东车牌进行车牌识别的示意图；
图4是本申请实施例二提供的一种车牌识别装置的结构示意图；
图5是本申请实施例三提供的电子设备的结构示意图。
本发明的实施方式
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、技术之类的具体细节，以便透彻理解本申请实施例。然而，本领域的技术人员应当清楚，在没有这些具体细节的其它实施例中也可以实现本申请。在其它情况中，省略对众所周知的系统、装置、电路以及方法的详细说明，以免不必要的细节妨碍本申请的描述。
应当理解，当在本申请说明书和所附权利要求书中使用时，术语“包括”指示所描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。
还应当理解，在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。
在本申请说明书中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此，在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例，而是意味着“一个或多个但不是所有的实施例”，除非是以其他方式另外特别强调。
实施例一：
目前的车牌识别方法中，有可能得到错误的识别结果。本申请发明人通过分析可知，现有的车牌识别方法只能对有固定格式的车牌进行识别，其需要将车牌上的字符信息划分为单个规整的字符区域，并以固定的窗口对每一个字符单元分别进行识别，也即，一旦车牌的格式发生改变，将会出现严重的识别错误，比如中东车牌上具有多种字符，包括阿拉伯数字，阿拉伯语等，其车牌的字符布局也具有繁多的样式，冗余信息多，字符小，若采用现有的车牌识别方法，将会出现严重的分割错误和识别错误，单个分割区域的识别准确率下降会造成车牌整体的准确率大幅下降，进而导致得到的识别结果的准确率较低。为了解决上述技术问题，本申请实施例提供了一种新的车牌识别方法，在该车牌识别方法中，根据车牌本身的内容信息对车牌进行分割，得到该车牌的文字区域和数字区域，再对该文字区域和数字区域进行识别，得到最终的车牌识别结果。由于根据车牌本身内容信息对车牌进行分割，因此能够得到该车牌更准确的文字区域和数字区域，进而对这些更准确的文字区域和数字区域进行识别后，能够得到更准确的车牌识别结果。
下面结合附图对本申请实施例提供的车牌识别方法进行描述：
图1示出了本申请实施例提供的一种车牌识别方法的流程图，在本申请实施例中，第一车牌、第二车牌中的“第一”和“第二”仅用于区分不同图像帧的车牌，并无特殊含义，其余包含“第一”，“第二”的命名与此类似，后续不再赘述：
步骤S11，对视频流中的第N个图像帧进行车牌检测，得到第一车牌检测结果，该第一车牌检测结果用于指示该第N个图像帧是否存在第一车牌，若存在该第一车牌，则指示该第一车牌在该第N个图像帧的位置，其中，N为整数，N大于或等于1。
本申请实施例中，视频流包括多个图像帧，该步骤的第N个图像帧为该视频流中的任一个图像帧，例如，当N等于1时，该第N个图像帧表示该视频流中的第一个图像帧，当N等于2时，该第N个图像帧表示该视频流中的第二个图像帧。在本实施例中，N的最大值与视频流本身包含的图像帧的个数相等，例如，若视频流包括的图像帧的个数为30帧，那么上述的N的最大值为30。
在一些实施例中，上述步骤S11具体包括：通过第一目标检测模型对视频流中的第N个图像帧进行车牌检测，得到第一车牌检测结果。
本实施例中，采用第一目标检测模型进行车牌检测后，得到第一车牌在第N个图像帧的位置以及对应的置信度。当置信度较大，比如大于预设的置信度阈值，则表明该第一目标检测模型输出的针对第一车牌的第一车牌检测结果具有较高的把握，并将对应的位置传送到下一个算法。其中，第一车牌在第N个图像帧的位置可用车牌左上(x1，y1)和右下(x2，y2)两个点的坐标所定位的矩形的检测框表示，或者，用4个角点坐标所构成的多边形框表示。其中，该第一目标检测模型可以为One-stage目标检测模型，该One-stage目标检测模型包括但不限于YOLO，SSD等目标检测算法所构成的目标检测模型。上述的第一目标检测模型为对第二目标检测模型进行训练后得到的模型，该第一目标检测模型为具有神经网络的模型。具体地，通过以下方式对第二目标检测模型进行训练：
获取摄像头捕获的图像，通过人工对图像中的车牌的坐标进行标注，得到对应的训练标签，采用具有训练标签的摄像头捕获的图像对第二目标检测模型进行训练，得到第一目标检测模型。需要指出的是，当上述摄像头捕获的图像包含的车牌所属的国家不同时，得到的第一目标检测模型能够识别的车牌所属的国家也不同。例如，当车牌所属的国家均为中国时，得到的第一目标检测模型能够识别的车牌所属的国家为中国。当车牌所属的地区均为北美地区时，得到的第一目标检测模型能够识别的车牌所属的地区为北美地区。当车牌所属的国家包括多个时，得到的第一目标检测模型能够识别的车牌所属的国家也包含多个，即通过采用混合了不同国家的车牌对第二目标检测模型进行训练，使得得到的第一目标检测模型能够识别对应不同国家的车牌。
步骤S12，若该第一车牌检测结果指示该第N个图像帧存在第一车牌，则根据该第一车牌在该第N个图像帧的位置以及该第一车牌的内容信息，从该第一车牌中分割出文字区域和数字区域。
本申请实施例中，第一车牌的内容信息是指该第一车牌中所包含的文字信息和数字信息，以及该文字信息和该数字信息在第一车牌的区域位置。
在一个车牌中，除了数字信息还有文字信息，由于对数字信息的识别和对文字信息的识别是不同的，因此，在本申请实施例中，需要分别从第一车牌分离出文字区域和数字区域，以便后续通过对文字区域中的文字和对数字区域中的数字进行识别后，得到准确的识别结果。
当然，若第一车牌检测结果指示第N个图像帧不存在第一车牌，则继续对该第N个图像帧的下一个图像帧进行车牌检测。
步骤S13，对从该第一车牌中分割出的该文字区域和该数字区域进行识别，得到该第一车牌的第一识别结果。
本实施例中，通过分别识别分割出的文字区域和数字区域，得到第一车牌的第一识别结果，该第一识别结果包含该第一车牌的城市信息和车牌号信息等。
本申请实施例中，由于根据车牌本身内容信息对车牌进行分割，因此能够得到该车牌更准确的文字区域和数字区域，进而对这些更准确的文字区域和数字区域进行识别后，能够得到更准确的车牌识别结果。
在一些实施例中，上述步骤S13具体包括：通过第一车牌识别模型对从该第一车牌中分割出的该文字区域和该数字区域进行识别，得到该第一车牌的第一识别结果。其中，上述的第一车牌识别模型为对第二车牌识别模型进行训练后得到的模型，该第一车牌识别模 型为具有神经网络的模型。具体地，通过以下方式对第二车牌识别模型进行训练：
获取输入该第二车牌识别模型的分割好的车牌图像，通过人工标注或合成车牌内容字符串，得到对应的训练标签。上述的车牌内容字符串包括文字和数字。采用上述分割好的车牌图像和训练标签对第二车牌识别模型进行训练，得到第一车牌识别模型。
图2示出了本申请实施例提供的另一种车牌识别方法的流程图，本实施例中，为了提高输出的车牌识别结果的准确性，除了对当前图像帧(第N个图像帧进行车牌检测)，还对该当前图像帧的下一个图像帧(第N+1个图像帧进行车牌检测)，最后结合相邻图像帧的检测结果，得到最终输出的车牌识别结果，其中，步骤S21、步骤S22、步骤S23与上述的步骤S11、步骤S12、步骤S13相同，此处不再赘述：
步骤S21，对视频流中的第N个图像帧进行车牌检测，得到第一车牌检测结果，该第一车牌检测结果用于指示该第N个图像帧是否存在第一车牌，若存在该第一车牌，则指示该第一车牌在该第N个图像帧的位置，其中，N为整数，N大于或等于1。
步骤S22，若该第一车牌检测结果指示该第N个图像帧存在第一车牌，则根据该第一车牌在该第N个图像帧的位置以及该第一车牌的内容信息，从该第一车牌中分割出文字区域和数字区域。
步骤S23，对从该第一车牌中分割出的该文字区域和该数字区域进行识别，得到该第一车牌的第一识别结果。
步骤S24，分别对该视频流中的M个图像帧进行车牌检测，得到M个第二车牌检测结果，该第二车牌检测结果用于指示该M个图像帧中进行车牌检测的图像帧是否存在第二车牌，若存在该第二车牌，则指示该第二车牌在该M个图像帧中进行车牌检测的图像帧的位置，该M个图像帧均为该第N个图像帧之后的图像帧，M大于或等于1。
本申请实施例中，当M等于2时，将分别对该2个图像帧(比如第N+1图像帧和第N+2图像帧)进行车牌检测，对每一个图像帧进行车牌检测以及识别的过程均与对第N个图像帧进行车牌检测以及识别的过程相似，此处不再赘述。
步骤S25，若该M个第二车牌检测结果中存在至少一个目标车牌检测结果，则根据该至少一个目标车牌检测结果所指示的至少一个第二车牌在该M个图像帧中进行车牌检测的图像帧的位置以及该至少一个第二车牌的内容信息，分别从该至少一个第二车牌中分割出文字区域和数字区域，其中，该目标车牌检测结果是指包括了指示该第二车牌在该M个图像帧中进行车牌检测的图像帧的位置的车牌检测结果。
假设对第N+1个图像帧进行车牌检测后，得到的第二车牌检测结果为M1，该M1指示该第N+1个图像帧存在第二车牌m1。对第N+2个图像帧进行车牌检测后，得到的第二车牌检测结果为M2，该M2指示该第N+1个图像帧没有存在第二车牌，M1为目标车牌检测结果。根据m1在第N+1个图像帧的位置以及该m1的内容信息，从m1中分割出文字区域和数字区域。
步骤S26，对分别从该至少一个第二车牌中分割出的该文字区域和该数字区域进行识别，得到该至少一个第二车牌的至少一个第二识别结果。
本申请实施例中，假设需要对2个第二车牌(第二车牌m1和第二车牌m2)的文字区域和数字区域识别，则先识别出第二车牌m1的文字区域和该第二车牌m1的数字区域，得到一个第二识别结果，之后，再识别第二车牌的m2的文字区域和该第二车牌m2的数字区域，得到另一个第二识别结果。
步骤S27，根据该至少一个第二车牌在该M个图像帧中进行车牌检测的图像帧的位置以及该第一车牌在该第N个图像帧的位置分别判断该至少一个第二车牌与该第一车牌是否匹配。
本实施例中，若第一车牌和第二车牌匹配，则表明该第一车牌和第二车牌是相同的车牌，若第一车牌和第二车牌不匹配，则表明该第一车牌和第二车牌不是相同的车牌。具体地，可将该第一车牌的位置和该第二车牌的位置比较，若两者在相邻图像帧中的位置变化 很小，则判定第一车牌和第二车牌匹配，否则，判定第一车牌和第二车牌不匹配。
需要指出的是，上述步骤S27也可以在步骤S24之后或步骤S25之后执行，此处不作限定。
步骤S28，根据该第一车牌的第一识别结果和目标识别结果确定输出的车牌识别结果，其中，该目标识别结果是指与该第一车牌匹配的第二车牌所对应的第二识别结果。
本实施例中，可根据第一识别结果中第一车牌的位置的置信度，和目标识别结果中第二车牌的位置的置信度确定输出的车牌识别结果。或者，通过对第一识别结果和目标识别结果中的信息进行组合，比如，选取第一识别结果中的部分信息，以及，选取目标识别结果中的部分信息，再将选取的2部分信息进行组合，确定输出的车牌识别结果。
本申请实施例中，判断相邻图像帧中是否存在相同的车牌，并在存在相同的车牌时，根据该相同的车牌分别在相邻图像帧中识别结果确定输出的车牌识别结果。也即，通过增加其他图像帧队同一车牌的识别结果来确定最终的车牌识别结果，从而能够提高得到的车牌识别结果的准确性。
在一些实施例中，若分别从第一识别结果和目标识别结果选取部分内容组成最终的车牌识别结果，且上述的目标识别结果的个数大于或等于2，则该步骤S28，包括：
A1、根据预设的输出格式对该第一识别结果进行拆分，得到第一拆分内容，该第一拆分内容包括至少2个拆分子内容，每一个该拆分子内容与一个置信度对应。
A2、根据预设的输出格式分别对至少2个该目标识别结果进行拆分，得到至少2个第二拆分内容，该第二拆分内容包括至少2个拆分子内容，每一个拆分子内容与一个置信度对应。
A3、分别累加该第一拆分内容和该至少2个该第二拆分内容中具有相同拆分子内容的置信度，按照该预设的输出格式分别选取累加后置信度高的拆分子内容，并组成输出的车牌识别结果。
本实施例中，首先将第一识别结果和目标识别结果均按照预设的输出格式进行拆分，使得不同种类的车牌均被概括在同一结构框架下，再根据拆分得到的拆分子内容及对应的置信度，确定具有同一拆分子内容对应的累加后的置信度，当预设的输出格式中的同一位置，其对应的拆分子内容对应的累加后的置信度越高，表明该位置为越高的累加后的置信度所对应的拆分子内容的概率越高，因此，按照预设的输出格式分别选取累加后置信度高的拆分子内容，并组成输出的车牌识别结果所对应的准确度越高(即根据投票机制输出车牌识别结果)。例如，假设预设的输出格式为“城市”+“车牌号码”，第一识别结果对应的拆分子内容分别为“DUBAI”+“I 55555”，对应的置信度分别为“0.6”，“0.7”，目标识别结果1对应的拆分子内容分别为“DUBAI”+“I 55556”，对应的置信度分别为“0.7”，“0.6”，目标识别结果2对应的拆分子内容分别为“DUBAL”+“I 55555”，对应的置信度分别为“0.5”，“0.5”，则拆分子内容“粤B”对应的累加后的置信度为“0.6+0.7＝1.3”，拆分子内容“DUBAL”对应的累加后的置信度为“0.5”。拆分子内容“I 55555”对应的累加后的置信度为“0.7+0.5＝1.2”，拆分子内容“I 55556”对应的置信度为“0.6”，由于1.3大于0.5,1.2大于0.6，故最后得到的车牌识别结果为“DUBALI 55555”。
在一些实施例中，若该第一车牌和该第二车牌的个数均大于1，则上述步骤S27，包括：
B1、从该第N个图像帧至第N+M个图像帧中选取出M个图像帧队，一个该图像帧队为两个相邻的图像帧。
本申请实施例中，由于在对第N个图像帧进行车牌检测之后，还对该第N个图像帧之后的M个图像帧进行车牌检测，也即，是对第N个图像帧至第N+M个图像帧进行车牌检测。从这(M+1)个图像帧中，将相邻的两个图像帧划为一个图像帧队，供划分出M个图像帧队。
B2、对该M个图像帧队中的任一图像帧队，根据该图像帧队中的该第二车牌在图像帧的位置以及该第一车牌在图像帧的位置判断该第一车牌和该第二车牌是否匹配。
重复执行上述B2，直到该M个图像帧队中的任一图像帧队中的第一车牌和第二车牌均已匹配结束。
例如，假设M等于2，N＝1，则将视频流中的第1个图像帧和第2个图像帧划分为一个图像帧队(假设为图像帧队1)，将第2个图像帧和第3个图像帧划分为一个图像帧队(假设为图像帧队2)，则根据第二车牌在第2个图像帧的位置以及第一车牌在第1个图像帧的位置判断第一车牌和第二车牌是否匹配。之后，再根据第二车牌在第三个图像帧的位置以及第一车牌在第2个图像帧的位置判断第一车牌和第二车牌是否匹配。需要指出的是，上面的第二车牌均是指在图像帧队中位于视频流的后面的图像帧，例如在图像帧队1中，第2个图像帧为位于视频流的后面的图像帧，而在图像帧队2中，第2个图像帧则变为位于视频流的前面的图像帧。
本申请实施例中，考虑到第一车牌所在的图像帧与第二车牌所在的图像帧为相邻图像帧时，该第一车牌和该第二车牌为同一车牌的概率较大，因此，通过匹配分别处于相邻的两个图像帧的第一车牌和第二车牌，能够更快查找到匹配的两个车牌。
在一些实施例中，该B2中根据该图像帧队中的该第二车牌在图像帧的位置以及该第一车牌在图像帧的位置判断该第一车牌和该第二车牌是否匹配，包括：
B21、计算检测框R i和检测框R j的交并比，得到序列S 1和序列S 2中各个元素的交并比。其中，检测框R i为序列S 1中的任一元素，检测框R j为序列S 2中的任一元素，该序列S 1由该第一车牌的检测框组成，该序列S 2由该第二车牌的检测框组成，该第一车牌在该第N个图像帧的位置和该第二车牌在该第N+1个图像帧的位置均用对应的检测框表示。
本申请实施例中，首先初始化两个相邻图像帧的检测框所构成的两个集合，一个集合用于存储第一车牌的检测框，另一个集合用于存储第二车牌的检测框。将两个集合按照空间位置关系排列为二分图的左右两个序列S 1和S 2。例如，将第N图像帧的所有检测框按照空间位置规则(例如检测框的中心坐标到原点坐标的欧几里得距离)排序构成左序列，同理，将第N+1图像帧的所有检测框构成右序列。
重复地从两个序列S 1和S 2中取出检测框R i和检测框R j，计算检测框R i和检测框R j的交并比(Intersection over Union，IoU)，该IoU是两个检测框的交集与并集的比值，其中：
Figure PCTCN2020140930-appb-000001
B22、将该交并比作为连接该检测框R 1和该检测框R 2的边的权值。
B23、将序列S 1和序列S 2中的每个检测框作为二分图的顶点，并初始化该二分图的顶点的权值：序列S 1中每个顶点的权值为与其对应的检测框连接的边的最大的权值，序列S 2中每个顶点的权值为第一预设值。
其中，上述第一预设值可为小于0.5的数值，例如，为0。
B24、对序列S 1中的顶点X，从序列S 2中查找权值与该顶点X的权值相同的边，若查找到权值与该顶点X的权值相同的边，则判定序列S 1中的顶点X所对应的第一车牌匹配成功，若没有查找到权值与该顶点X的权值相同的边，则判定序列S 1中的顶点X所对应的第一车牌匹配失败，其中，顶点X为序列S 1中的任一个顶点。
本申请实施例中，逐个为序列S 1中的各个顶点匹配对应的边。由于交并比作为连接检测框R 1和该检测框R 2的边的权值，因此，连接检测框R 1和该检测框R 2的边的权值越大，表明检测框R 1和检测框R 2存在的相同区域越大，并且，由于序列S 1中每个顶点的权值为与其对应的检测框连接的边的最大的权值，因此，通过判断序列S 2是否存在与顶点X的权值相同的边来判断该顶点X对应的检测框所对应的第一车牌是否与第二车牌匹配，能够提高匹 配结果的准确性。
在一些实施例中，分别对多个第一车牌和多个第二车牌匹配时，可通过匈牙利算法或KM算法对每个图像帧的多目标(该目标为第一车牌和第二车牌)进行数据关联，形成最优匹配。进一步地，对匹配得到的不同图像帧的检测框建立对应车牌的唯一标识(Identity document，ID)。通过上述处理，便于后续连续跟踪画面中的每一个车牌，特别是当画面中检测出多个车牌时，可以确保前后图像帧的车牌之间的匹配关系。例如，在根据预设的输出格式对第一识别结果和第二识别结果进行拆分时，可根据预设的输出格式对对应同样ID的第一识别结果和第二识别结果进行拆分。
在一些实施例中，可按序列S 1中各个检测框的顺序(如从前往后，或从后往前选取顶点X)，对序列S 1中的顶点所对应的第一车牌进行匹配。
在一些实施例中，上述B24中若没有查找到权值与该顶点X的权值相同的边，则判定序列S 1中的顶点X所对应的第一车牌匹配失败，包括：
B241、若没有查找到权值与该顶点X的权值相同的边，则将该顶点X的权值减去第二预设值，且将与该顶点X对应的检测框连接的检测框所对应的顶点的权值增加该第二预设值。
其中，第二预设值大于0。由于第二预设值大于0，因此，顶点X的权值减去第二预设值后，剩下的权值将小于原有的权值。
B242、将该顶点X的下一个顶点作为新的顶点X，并返回该对序列S 1中的顶点X，从序列S 2中查找权值与该顶点X的权值相同的边(即B24)的步骤以及后续步骤，直到该顶点X的权值变为0，则判定该序列S 1中的顶点X所对应的第一车牌匹配失败。
具体地，匹配的原则是：只和权值与左边顶点的权值(即上述初始化为左顶点所赋值)相同的边进行匹配，若找不到匹配的边，则将此条路径对应左边顶点的值减d，右边顶点的值加d，继续进行左边序列的下一个顶点的匹配。
本申请实施例中，当第一车牌的检测框所对应的顶点匹配失败后，使第一车牌的检测框所对应的顶点的权值减小，后续继续对权值减小后的顶点匹配，直到该权值减小后的顶点的权值为0才停止对该权值减小后的顶点(当顶点X匹配失败，意味着原先出现在图像帧中的顶点X对应的第一车牌的检测框在后续的图像帧中不再出现，表明该顶点X对应的第一车牌可能已经移除视野)。也即，通过逐步降低顶点的权值，能够提高查找到匹配的边的概率。
在一些实施例中，该步骤S13(或步骤S23)，包括：
将从该第一车牌中分割出的该文字区域和该数字区域组合成固定格式的第一待识别信息，对该第一待识别信息进行识别，得到该第一车牌的第一识别结果。
本申请实施例中，考虑到对不同格式的信息进行识别将增加识别难度，因此，为了降低识别难度，提高识别准确度，则将车牌中的文字区域和数字区域组合成固定格式的待识别信息。例如，以中东车牌为例，其对应的车牌种类十分丰富，车牌结构布局各有千秋：有单层车牌，还有双层和多层车牌。车牌中的文字部分的分布位置也不尽相同，这给车牌识别造成困难。为了能够提高识别准确度，将车牌分割后的部分按照固定格式拼接，例如，按照左边文字右边数字的固定格式拼接起来，确保不同车牌在输入到第一车牌识别模型之前都是单层的结构。这样可以进一步统一输入数据类型并简化问题，使第一车牌识别模型获得更高的准确率，还可以在不同国家的车牌识别上获取更好的可迁移性。
需要指出的是，对第二车牌中分割出的文字区域和数字区域同样执行上述步骤，此处不再赘述。
在一些实施例中，该步骤S12(或步骤S22)中根据该第一车牌在该第N个图像帧的位置以及该第一车牌的内容信息，从该第一车牌中分割出文字区域和数字区域，包括：
C1、根据该第一车牌的内容信息确定该第一车牌的格式，该第一车牌的格式用于分别指示该第一车牌的文字区域和数字区域在该第一车牌的区域位置。
本申请实施例中，预先存储不同车牌的内容信息与车牌的格式的对应关系，当获取到车牌的内容信息后，根据存储的对应关系确定与该车牌的内容信息对应的车牌的格式。
C2、根据该第一车牌在该第N个图像帧的位置以及该第一车牌的格式，从该第一车牌中分割出文字区域和数字区域。
本申请实施例中，由于预存了不同车牌的内容信息与车牌的格式的对应关系，而车牌的格式用于分别指示车牌的文字区域和数字区域在上述车牌的区域位置，因此，后续根据车牌的格式能够快速提取到该车牌的文字区域和数字区域。
在一些实施例中，该步骤S12(或步骤S22)中根据该第一车牌在该第N个图像帧的位置以及该第一车牌的内容信息，从该第一车牌中分割出文字区域和数字区域，包括：
D1、根据该第一车牌在该第N个图像帧的位置从该第N个图像帧中截取该第一车牌的区域，得到第一车牌图像。
本申请实施例中，由于第一车牌图像是从第N个图像帧中截取的第一车牌的区域所对应的图像，因此，该第一车牌图像的像素点数量将远小于该第N个图像帧的像素点数量，即减少了后续需要处理的像素点数量，从而节约了电子设备的资源。
D2、对该第一车牌图像执行以下至少一种处理：矫正处理、图像增强处理、去噪处理、去模糊处理和归一化处理，其中，该矫正处理用于将具有角度偏曲的第一车牌图像矫正为平整的第一车牌图像，该归一化处理用于将该第一车牌图像的像素值域处理为标准化分布。
其中，矫正处理后的图像能够增大有效的像素面积。
其中，图像增强处理是指通过一定手段对原图像附加一些信息或变换数据，有选择地突出原图像中感兴趣的特征或者抑制(掩盖)原图像中某些不需要的特征,使处理后的图像与视觉响应特性相匹配。本实施例中的可采用现有的图像增强算法实现图像增强处理。
其中，去模糊处理可以减轻运动模糊造成的重影，使车牌更清晰。
其中，归一化处理能够使得车牌的像素值域呈标准化分布，满足神经网络的处理要求。
D3、从处理后的该第一车牌图像中分割出文字区域和数字区域。
由于处理后的第一车牌图像能够使得车牌更容易识别，因此，能够从处理后的该第一车牌图像中更准确地分割出对应的文字区域和数字区域。
在一些实施例中，该D3，包括：
通过语义分割模型，从处理后的该第一车牌图像中分割出像素级的文字区域和数字区域。
其中，语义分割模型用于将处理后的第一车牌图像的不同区域分割出来，区分哪些位置对应省份信息，需要识别文字，哪些位置对应车牌号信息，需要识别数字等。
本申请实施例中，语义分割模型为神经网络模型，其在应用之前需要进行通过千万级数据对其进行训练。具体地，训练的数据为通过第一目标检测模型检测到的车牌在图像帧的位置，训练的标签为人工对车牌进行分割后的不同区域，这些不同区域包含文字的区域，也包含数字的区域。训练后的语义分割模型能够从车牌图像中分割出像素级的文字区域和数字区域。由于采用语义分割模型对城市信息(如阿拉伯语的城市信息)进行像素级分类、预测、推断标签来实现细粒度的推理，从而使每个像素都被标记为其封闭区域的类别，进而将学习到的识别特征语义投影到像素空间(高分辨率)上，得到密集的分类，输出最终的城市信息结果。
图3示出了采用本申请实施例提供的车牌识别进行车牌识别的示意图。
在图3中，第一目标检测模型采用One-stage目标检测模型，对图像帧队的第一车牌和第二车牌进行匹配时具体采用多目标检测算法，如匈牙利算法或KM算法。当从多目标检测算法确定出匹配的第一车牌和第二车牌后，通过语义分割模型从第一车牌分割出文字区域和数字区域，以及，从第二车牌分割出2个文字区域(即城市信息)和1个数字区域(即具体的车牌号码信息)，其中，2个文字区域分别为：第一文字区域、第二文字区域，该第一文字区域的信息为“迪拜”的英文单词“DUBAI”，第二文字区域是迪拜的阿拉伯语对应 的单词。将第一文字区域、第二文字区域以及数字区域按照左边文字右边数字的格式拼接起来，再通过端到端识别模型(即上文中的第一车牌识别模型)对上述拼接后的信息进行识别，得到车牌识别结果并输出。
应理解，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
实施例二：
对应于上文实施例一该的车牌识别方法，图4示出了本申请实施例提供的车牌识别装置的结构框图，该车牌识别装置可应用于电子设备，该电子设备可为服务器或终端设备，为了便于说明，仅示出了与本申请实施例相关的部分。
参照图4，该车牌识别装置4包括：第一车牌检测单元41、第一车牌检测单元42、第一车牌内容识别单元43。其中：
第一车牌检测单元41，用于对视频流中的第N个图像帧进行车牌检测，得到第一车牌检测结果，该第一车牌检测结果用于指示该第N个图像帧是否存在第一车牌，若存在该第一车牌，则指示该第一车牌在该第N个图像帧的位置，其中，N为整数，N大于或等于1。
第一车牌内容分割单元42，用于若该第一车牌检测结果指示该第N个图像帧存在第一车牌，则根据该第一车牌在该第N个图像帧的位置以及该第一车牌的内容信息，从该第一车牌中分割出文字区域和数字区域。
第一车牌内容识别单元43，用于对从该第一车牌中分割出的该文字区域和该数字区域进行识别，得到该第一车牌的第一识别结果。
本申请实施例中，由于根据车牌本身内容信息对车牌进行分割，因此能够得到该车牌更准确的文字区域和数字区域，进而对这些更准确的文字区域和数字区域进行识别后，能够得到更准确的车牌识别结果。
在一些实施例中，该车牌识别装置4还包括：
第二车牌检测单元，用于分别对该视频流中的M个图像帧进行车牌检测，得到M个第二车牌检测结果，该第二车牌检测结果用于指示该M个图像帧中进行车牌检测的图像帧是否存在第二车牌，若存在该第二车牌，则指示该第二车牌在该M个图像帧中进行车牌检测的图像帧的位置，该M个图像帧均为该第N个图像帧之后的图像帧，M大于或等于1。
第二车牌内容分割单元，用于若该M个第二车牌检测结果中存在至少一个目标车牌检测结果，则根据该至少一个目标车牌检测结果所指示的至少一个第二车牌在该M个图像帧中进行车牌检测的图像帧的位置以及该至少一个第二车牌的内容信息，分别从该至少一个第二车牌中分割出文字区域和数字区域，其中，该目标车牌检测结果是指包括了指示该第二车牌在该M个图像帧中进行车牌检测的图像帧的位置的车牌检测结果。
第二车牌内容识别单元，用于对分别从该至少一个第二车牌中分割出的该文字区域和该数字区域进行识别，得到该至少一个第二车牌的至少一个第二识别结果。
车牌匹配单元，用于根据该至少一个第二车牌在该M个图像帧中进行车牌检测的图像帧的位置以及该第一车牌在该第N个图像帧的位置分别判断该至少一个第二车牌与该第一车牌是否匹配。
车牌识别结果确定单元，用于根据该第一车牌的第一识别结果和目标识别结果确定输出的车牌识别结果，其中，该目标识别结果是指与该第一车牌匹配的第二车牌所对应的第二识别结果。
在一些实施例中，若该目标识别结果的个数大于或等于2，该车牌识别结果确定单元，包括：
第一识别结果拆分模块，用于根据预设的输出格式对该第一识别结果进行拆分，得到第一拆分内容，该第一拆分内容包括至少2个拆分子内容，每一个该拆分子内容与一个置 信度对应。
目标识别结果拆分模块，用于根据预设的输出格式分别对至少2个该目标识别结果进行拆分，得到至少2个第二拆分内容，该第二拆分内容包括至少2个拆分子内容，每一个拆分子内容与一个置信度对应。
置信度累加模块，用于分别累加该第一拆分内容和该至少2个该第二拆分内容中具有相同拆分子内容的置信度，按照该预设的输出格式分别选取累加后置信度高的拆分子内容，并组成输出的车牌识别结果。
在一些实施例中，若该第一车牌和该第二车牌的个数均大于1，则该车牌匹配单元，包括：
图像帧队确定模块，用于从该第N个图像帧至第N+M个图像帧中选取出M个图像帧队，一个该图像帧队为两个相邻的图像帧。
图像帧队的车牌匹配模块，用于对该M个图像帧队中的任一图像帧队，根据该图像帧队中的该第二车牌在图像帧的位置以及该第一车牌在图像帧的位置判断该第一车牌和该第二车牌是否匹配。重复执行该对该M个图像帧队中的任一图像帧队，直到该M个图像帧队中的任一图像帧队中的第一车牌和第二车牌均已匹配结束。
在一些实施例中，该图像帧队的车牌匹配模块在根据该图像帧队中的该第二车牌在图像帧的位置以及该第一车牌在图像帧的位置判断该第一车牌和该第二车牌是否匹配时，具体用于：
计算检测框R i和检测框R j的交并比，得到序列S 1和序列S 2中各个元素的交并比。其中，检测框R i为序列S 1中的任一元素，检测框R j为序列S 2中的任一元素，该序列S 1由该第一车牌的检测框组成，该序列S 2由该第二车牌的检测框组成，该第一车牌在该第N个图像帧的位置和该第二车牌在该第N+1个图像帧的位置均用对应的检测框表示。将该交并比作为连接该检测框R 1和该检测框R 2的边的权值。将序列S 1和序列S 2中的每个检测框作为二分图的顶点，并初始化该二分图的顶点的权值：序列S 1中每个顶点的权值为与其对应的检测框连接的边的最大的权值，序列S 2中每个顶点的权值为第一预设值，该第一预设值小于0.5。对序列S 1中的顶点X，从序列S 2中查找权值与该顶点X的权值相同的边，若查找到权值与该顶点X的权值相同的边，则判定该序列S 1中的顶点X所对应的第一车牌匹配成功，若没有查找到权值与该顶点X的权值相同的边，则判定该序列S 1中的顶点X所对应的第一车牌匹配失败，其中，顶点X为序列S 1中的任一个顶点。
在一些实施例中，该图像帧队的车牌匹配模块在没有查找到权值与该顶点X的权值相同的边，则判定该序列S 1中的顶点X所对应的第一车牌匹配失败时，具体用于：
若没有查找到权值与该顶点X的权值相同的边，则将该顶点X的权值减去第二预设值，且将与该顶点X对应的检测框连接的检测框所对应的顶点的权值增加该第二预设值，该第二预设值大于0。将该顶点X的下一个顶点作为新的顶点X，并返回该对序列S 1中的顶点X，从序列S 2中查找权值与该顶点X的权值相同的边的步骤以及后续步骤，直到该顶点X的权值变为0，则判定该序列S 1中的顶点X所对应的第一车牌匹配失败。
在一些实施例中，该第一车牌内容识别单元43，具体用于：
将从该第一车牌中分割出的该文字区域和该数字区域组合成固定格式的第一待识别信息，对该第一待识别信息进行识别，得到该第一车牌的第一识别结果。
在一些实施例中，该第一车牌内容分割单元42在根据该第一车牌在该第N个图像帧的位置以及该第一车牌的内容信息，从该第一车牌中分割出文字区域和数字区域时，具体用于：
根据该第一车牌的内容信息确定该第一车牌的格式，该第一车牌的格式用于分别指示该第一车牌的文字区域和数字区域在该第一车牌的区域位置。根据该第一车牌在该第N个 图像帧的位置以及该第一车牌的格式，从该第一车牌中分割出文字区域和数字区域。
在一些实施例中，该第一车牌内容分割单元42在根据该第一车牌在该第N个图像帧的位置以及该第一车牌的内容信息，从该第一车牌中分割出文字区域和数字区域时，具体用于：
根据该第一车牌在该第N个图像帧的位置从该第N个图像帧中截取该第一车牌的区域，得到第一车牌图像。对该第一车牌图像执行以下至少一种处理：矫正处理、图像增强处理、去噪处理、去模糊处理和归一化处理，其中，该矫正处理用于将具有角度偏曲的第一车牌图像矫正为平整的第一车牌图像，该归一化处理用于将该第一车牌图像的像素值域处理为标准化分布。从处理后的该第一车牌图像中分割出文字区域和数字区域。
在一些实施例中，该第一车牌内容分割单元42在从处理后的该第一车牌图像中分割出文字区域和数字区域时，具体用于：
通过语义分割模型，从处理后的该第一车牌图像中分割出像素级的文字区域和数字区域。
需要说明的是，上述装置/单元之间的信息交互、执行过程等内容，由于与本申请方法实施例基于同一构思，其具体功能及带来的技术效果，具体可参见方法实施例部分，此处不再赘述。
实施例三：
图5为本申请一实施例提供的电子设备的结构示意图。该电子设备可以为服务器或终端设备，如图5所示，该实施例的电子设备5包括：至少一个处理器50(图5中仅示出一个处理器)、存储器51以及存储在该存储器51中并可在该至少一个处理器50上运行的计算机程序52，该处理器50执行该计算机程序52时实现上述任意各个方法实施例中的步骤：
对视频流中的第N个图像帧进行车牌检测，得到第一车牌检测结果，该第一车牌检测结果用于指示该第N个图像帧是否存在第一车牌，若存在该第一车牌，则指示该第一车牌在该第N个图像帧的位置，其中，N为整数，N大于或等于1。
若该第一车牌检测结果指示该第N个图像帧存在第一车牌，则根据该第一车牌在该第N个图像帧的位置以及该第一车牌的内容信息，从该第一车牌中分割出文字区域和数字区域。
对从该第一车牌中分割出的该文字区域和该数字区域进行识别，得到该第一车牌的第一识别结果。
可选地，该车牌识别方法还包括：
分别对该视频流中的M个图像帧进行车牌检测，得到M个第二车牌检测结果，该第二车牌检测结果用于指示该M个图像帧中进行车牌检测的图像帧是否存在第二车牌，若存在该第二车牌，则指示该第二车牌在该M个图像帧中进行车牌检测的图像帧的位置，该M个图像帧均为该第N个图像帧之后的图像帧，M大于或等于1。
若该M个第二车牌检测结果中存在至少一个目标车牌检测结果，则根据该至少一个目标车牌检测结果所指示的至少一个第二车牌在该M个图像帧中进行车牌检测的图像帧的位置以及该至少一个第二车牌的内容信息，分别从该至少一个第二车牌中分割出文字区域和数字区域，其中，该目标车牌检测结果是指包括了指示该第二车牌在该M个图像帧中进行车牌检测的图像帧的位置的车牌检测结果。
对分别从该至少一个第二车牌中分割出的该文字区域和该数字区域进行识别，得到该至少一个第二车牌的至少一个第二识别结果。
根据该至少一个第二车牌在该M个图像帧中进行车牌检测的图像帧的位置以及该第一车牌在该第N个图像帧的位置分别判断该至少一个第二车牌与该第一车牌是否匹配。
根据该第一车牌的第一识别结果和目标识别结果确定输出的车牌识别结果，其中，该目标识别结果是指与该第一车牌匹配的第二车牌所对应的第二识别结果。
可选地，若该目标识别结果的个数大于或等于2，该根据该第一车牌的第一识别结果和目标识别结果确定输出的车牌识别结果，包括：
根据预设的输出格式对该第一识别结果进行拆分，得到第一拆分内容，该第一拆分内容包括至少2个拆分子内容，每一个该拆分子内容与一个置信度对应。
根据预设的输出格式分别对至少2个该目标识别结果进行拆分，得到至少2个第二拆分内容，该第二拆分内容包括至少2个拆分子内容，每一个拆分子内容与一个置信度对应。
分别累加该第一拆分内容和该至少2个该第二拆分内容中具有相同拆分子内容的置信度，按照该预设的输出格式分别选取累加后置信度高的拆分子内容，并组成输出的车牌识别结果。
可选地，若该第一车牌和该第二车牌的个数均大于1，则该根据该至少一个第二车牌在该M个图像帧中进行车牌检测的图像帧的位置以及该第一车牌在该第N个图像帧的位置分别判断该至少一个第二车牌与该第一车牌是否匹配，包括：
从该第N个图像帧至第N+M个图像帧中选取出M个图像帧队，一个该图像帧队为两个相邻的图像帧。
对该M个图像帧队中的任一图像帧队，根据该图像帧队中的该第二车牌在图像帧的位置以及该第一车牌在图像帧的位置判断该第一车牌和该第二车牌是否匹配。
重复执行该对该M个图像帧队中的任一图像帧队，直到该M个图像帧队中的任一图像帧队中的第一车牌和第二车牌均已匹配结束。
可选地，该根据该图像帧队中的该第二车牌在图像帧的位置以及该第一车牌在图像帧的位置判断该第一车牌和该第二车牌是否匹配，包括：
计算检测框R i和检测框R j的交并比，得到序列S 1和序列S 2中各个元素的交并比。其中，检测框R i为序列S 1中的任一元素，检测框R j为序列S 2中的任一元素，该序列S 1由该第一车牌的检测框组成，该序列S 2由该第二车牌的检测框组成，该第一车牌在该第N个图像帧的位置和该第二车牌在该第N+1个图像帧的位置均用对应的检测框表示。
将该交并比作为连接该检测框R 1和该检测框R 2的边的权值。
将序列S 1和序列S 2中的每个检测框作为二分图的顶点，并初始化该二分图的顶点的权值：序列S 1中每个顶点的权值为与其对应的检测框连接的边的最大的权值，序列S 2中每个顶点的权值为第一预设值，该第一预设值小于0.5。
对序列S 1中的顶点X，从序列S 2中查找权值与该顶点X的权值相同的边，若查找到权值与该顶点X的权值相同的边，则判定该序列S 1中的顶点X所对应的第一车牌匹配成功，若没有查找到权值与该顶点X的权值相同的边，则判定该序列S 1中的顶点X所对应的第一车牌匹配失败，其中，顶点X为序列S 1中的任一个顶点。
可选地，该若没有查找到权值与该顶点X的权值相同的边，则判定该序列S 1中的顶点X所对应的第一车牌匹配失败，包括：
若没有查找到权值与该顶点X的权值相同的边，则将该顶点X的权值减去第二预设值，且将与该顶点X对应的检测框连接的检测框所对应的顶点的权值增加该第二预设值，该第二预设值大于0。
将该顶点X的下一个顶点作为新的顶点X，并返回该对序列S 1中的顶点X，从序列S 2中查找权值与该顶点X的权值相同的边的步骤以及后续步骤，直到该顶点X的权值变为0，则判定该序列S 1中的顶点X所对应的第一车牌匹配失败。
可选地，该对从该第一车牌中分割出的该文字区域和该数字区域进行识别，得到该第一车牌的第一识别结果，包括：
将从该第一车牌中分割出的该文字区域和该数字区域组合成固定格式的第一待识别信息，对该第一待识别信息进行识别，得到该第一车牌的第一识别结果。
可选地，该根据该第一车牌在该第N个图像帧的位置以及该第一车牌的内容信息，从该第一车牌中分割出文字区域和数字区域，包括：
根据该第一车牌的内容信息确定该第一车牌的格式，该第一车牌的格式用于分别指示该第一车牌的文字区域和数字区域在该第一车牌的区域位置。
根据该第一车牌在该第N个图像帧的位置以及该第一车牌的格式，从该第一车牌中分割出文字区域和数字区域。
可选地，该根据该第一车牌在该第N个图像帧的位置以及该第一车牌的内容信息，从该第一车牌中分割出文字区域和数字区域，包括：
根据该第一车牌在该第N个图像帧的位置从该第N个图像帧中截取该第一车牌的区域，得到第一车牌图像。
对该第一车牌图像执行以下至少一种处理：矫正处理、图像增强处理、去噪处理、去模糊处理和归一化处理，其中，该矫正处理用于将具有角度偏曲的第一车牌图像矫正为平整的第一车牌图像，该归一化处理用于将该第一车牌图像的像素值域处理为标准化分布。
从处理后的该第一车牌图像中分割出文字区域和数字区域。
可选地，该从处理后的该第一车牌图像中分割出文字区域和数字区域，包括：
通过语义分割模型，从处理后的该第一车牌图像中分割出像素级的文字区域和数字区域。
该电子设备5可以是桌上型计算机、笔记本、掌上电脑及云端服务器等计算设备。该电子设备可包括，但不仅限于，处理器50、存储器51。本领域技术人员可以理解，图5仅仅是电子设备5的举例，并不构成对电子设备5的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如还可以包括输入输出设备、网络接入设备等。
所称处理器50可以是中央处理单元(Central Processing Unit，CPU)，该处理器50还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
所述存储器51在一些实施例中可以是所述电子设备5的内部存储单元，例如电子设备5的硬盘或内存。所述存储器51在另一些实施例中也可以是所述电子设备5的外部存储设备，例如所述电子设备5上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。进一步地，所述存储器51还可以既包括所述电子设备5的内部存储单元也包括外部存储设备。所述存储器51用于存储操作系统、应用程序、引导装载程序(BootLoader)、数据以及其他程序等，例如所述计算机程序的程序代码等。所述存储器51还可以用于暂时地存储已经输出或者将要输出的数据。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
本申请实施例还提供了一种网络设备，该网络设备包括：至少一个处理器、存储器以及存储在所述存储器中并可在所述至少一个处理器上运行的计算机程序，所述处理器执行 所述计算机程序时实现上述任意各个方法实施例中的步骤。
本申请实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现可实现上述各个方法实施例中的步骤。
本申请实施例提供了一种计算机程序产品，当计算机程序产品在移动终端上运行时，使得移动终端执行时实现可实现上述各个方法实施例中的步骤。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质至少可以包括：能够将计算机程序代码携带到拍照装置/电子设备的任何实体或装置、记录介质、计算机存储器、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、电载波信号、电信信号以及软件分发介质。例如U盘、移动硬盘、磁碟或者光盘等。在某些司法管辖区，根据立法和专利实践，计算机可读介质不可以是电载波信号和电信信号。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。

Claims (13)

  1. 一种车牌识别方法，其特征在于，包括：
    对视频流中的第N个图像帧进行车牌检测，得到第一车牌检测结果，所述第一车牌检测结果用于指示所述第N个图像帧是否存在第一车牌，若存在所述第一车牌，则指示所述第一车牌在所述第N个图像帧的位置，其中，N为整数，N大于或等于1；
    若所述第一车牌检测结果指示所述第N个图像帧存在第一车牌，则根据所述第一车牌在所述第N个图像帧的位置以及所述第一车牌的内容信息，从所述第一车牌中分割出文字区域和数字区域；
    对从所述第一车牌中分割出的所述文字区域和所述数字区域进行识别，得到所述第一车牌的第一识别结果。
  2. 如权利要求1所述的车牌识别方法，其特征在于，所述车牌识别方法还包括：
    分别对所述视频流中的M个图像帧进行车牌检测，得到M个第二车牌检测结果，所述第二车牌检测结果用于指示所述M个图像帧中进行车牌检测的图像帧是否存在第二车牌，若存在所述第二车牌，则指示所述第二车牌在所述M个图像帧中进行车牌检测的图像帧的位置，所述M个图像帧均为所述第N个图像帧之后的图像帧，M大于或等于1；
    若所述M个第二车牌检测结果中存在至少一个目标车牌检测结果，则根据所述至少一个目标车牌检测结果所指示的至少一个第二车牌在所述M个图像帧中进行车牌检测的图像帧的位置以及所述至少一个第二车牌的内容信息，分别从所述至少一个第二车牌中分割出文字区域和数字区域，其中，所述目标车牌检测结果是指包括了指示所述第二车牌在所述M个图像帧中进行车牌检测的图像帧的位置的车牌检测结果；
    对分别从所述至少一个第二车牌中分割出的所述文字区域和所述数字区域进行识别，得到所述至少一个第二车牌的至少一个第二识别结果；
    根据所述至少一个第二车牌在所述M个图像帧中进行车牌检测的图像帧的位置以及所述第一车牌在所述第N个图像帧的位置，分别判断所述至少一个第二车牌与所述第一车牌是否匹配；
    根据所述第一车牌的第一识别结果和目标识别结果确定输出的车牌识别结果，其中，所述目标识别结果是指与所述第一车牌匹配的第二车牌所对应的第二识别结果。
  3. 如权利要求2所述的车牌识别方法，其特征在于，若所述目标识别结果的个数大于或等于2，所述根据所述第一车牌的第一识别结果和目标识别结果确定输出的车牌识别结果，包括：
    根据预设的输出格式对所述第一识别结果进行拆分，得到第一拆分内容，所述第一拆分内容包括至少2个拆分子内容，每一个所述拆分子内容与一个置信度对应；
    根据预设的输出格式分别对至少2个所述目标识别结果进行拆分，得到至少2个第二拆分内容，所述第二拆分内容包括至少2个拆分子内容，每一个拆分子内容与一个置信度对应；
    分别累加所述第一拆分内容和所述至少2个所述第二拆分内容中具有相同拆分子内容的置信度，按照所述预设的输出格式分别选取累加后置信度高的拆分子内容，并组成输出的车牌识别结果。
  4. 如权利要求2所述的车牌识别方法，其特征在于，若所述第一车牌和所述第二车牌的个数均大于1，则所述根据所述至少一个第二车牌在所述M个图像帧中进行车牌检测的图像帧的位置以及所述第一车牌在所述第N个图像帧的位置分别判断所述至少一个第二车牌与所述第一车牌是否匹配，包括：
    从所述第N个图像帧至第N+M个图像帧中选取出M个图像帧队，一个所述图像帧队为两个相邻的图像帧；
    对所述M个图像帧队中的任一图像帧队，根据所述图像帧队中的所述第二车牌在图 像帧的位置以及所述第一车牌在图像帧的位置判断所述第一车牌和所述第二车牌是否匹配；
    重复执行所述对所述M个图像帧队中的任一图像帧队，直到所述M个图像帧队中的任一图像帧队中的第一车牌和第二车牌均已匹配结束。
  5. 如权利要求4所述的车牌识别方法，其特征在于，所述根据所述图像帧队中的所述第二车牌在图像帧的位置以及所述第一车牌在图像帧的位置判断所述第一车牌和所述第二车牌是否匹配，包括：
    计算检测框R i和检测框R j的交并比，得到序列S 1和序列S 2中各个元素的交并比；其中，检测框R i为序列S 1中的任一元素，检测框R j为序列S 2中的任一元素，所述序列S 1由所述第一车牌的检测框组成，所述序列S 2由所述第二车牌的检测框组成，所述第一车牌在所述第N个图像帧的位置和所述第二车牌在所述第N+1个图像帧的位置均用对应的检测框表示；
    将所述交并比作为连接所述检测框R 1和所述检测框R 2的边的权值；
    将序列S 1和序列S 2中的每个检测框作为二分图的顶点，并初始化所述二分图的顶点的权值：序列S 1中每个顶点的权值为与其对应的检测框连接的边的最大的权值，序列S 2中每个顶点的权值为第一预设值，所述第一预设值小于0.5；
    对序列S 1中的顶点X，从序列S 2中查找权值与所述顶点X的权值相同的边，若查找到权值与所述顶点X的权值相同的边，则判定所述序列S 1中的顶点X所对应的第一车牌匹配成功，若没有查找到权值与所述顶点X的权值相同的边，则判定所述序列S 1中的顶点X所对应的第一车牌匹配失败，其中，顶点X为序列S 1中的任一个顶点。
  6. 如权利要求5所述的车牌识别方法，其特征在于，所述若没有查找到权值与所述顶点X的权值相同的边，则判定所述序列S 1中的顶点X所对应的第一车牌匹配失败，包括：
    若没有查找到权值与所述顶点X的权值相同的边，则将所述顶点X的权值减去第二预设值，且将与所述顶点X对应的检测框连接的检测框所对应的顶点的权值增加所述第二预设值，所述第二预设值大于0；
    将所述顶点X的下一个顶点作为新的顶点X，并返回所述对序列S 1中的顶点X，从序列S 2中查找权值与所述顶点X的权值相同的边的步骤以及后续步骤，直到重新从序列S 2中查找权值与所述顶点X的权值相同的边，且所述顶点X的权值变为0，则判定所述序列S 1中的顶点X所对应的第一车牌匹配失败。
  7. 如权利要求1所述的车牌识别方法，其特征在于，所述对从所述第一车牌中分割出的所述文字区域和所述数字区域进行识别，得到所述第一车牌的第一识别结果，包括：
    将从所述第一车牌中分割出的所述文字区域和所述数字区域组合成固定格式的第一待识别信息，对所述第一待识别信息进行识别，得到所述第一车牌的第一识别结果。
  8. 如权利要求1所述的车牌识别方法，其特征在于，所述根据所述第一车牌在所述第N个图像帧的位置以及所述第一车牌的内容信息，从所述第一车牌中分割出文字区域和数字区域，包括：
    根据所述第一车牌的内容信息确定所述第一车牌的格式，所述第一车牌的格式用于分别指示所述第一车牌的文字区域和数字区域在所述第一车牌的区域位置；
    根据所述第一车牌在所述第N个图像帧的位置以及所述第一车牌的格式，从所述第一车牌中分割出文字区域和数字区域。
  9. 如权利要求1至8任一项所述的车牌识别方法，其特征在于，所述根据所述第一车牌在所述第N个图像帧的位置以及所述第一车牌的内容信息，从所述第一车牌中分割出文字区域和数字区域，包括：
    根据所述第一车牌在所述第N个图像帧的位置从所述第N个图像帧中截取所述第一车牌的区域，得到第一车牌图像；
    对所述第一车牌图像执行以下至少一种处理：矫正处理、图像增强处理、去噪处理、 去模糊处理和归一化处理，其中，所述矫正处理用于将具有角度偏曲的第一车牌图像矫正为平整的第一车牌图像，所述归一化处理用于将所述第一车牌图像的像素值域处理为标准化分布；
    从处理后的所述第一车牌图像中分割出文字区域和数字区域。
  10. 如权利要求9所述的车牌识别方法，其特征在于，所述从处理后的所述第一车牌图像中分割出文字区域和数字区域，包括：
    通过语义分割模型，从处理后的所述第一车牌图像中分割出像素级的文字区域和数字区域。
  11. 一种车牌识别装置，其特征在于，包括：
    第一车牌检测单元，用于对视频流中的第N个图像帧进行车牌检测，得到第一车牌检测结果，所述第一车牌检测结果用于指示所述第N个图像帧是否存在第一车牌，若存在所述第一车牌，则指示所述第一车牌在所述第N个图像帧的位置，其中，N为整数，N大于或等于1；
    第一车牌内容分割单元，用于若所述第一车牌检测结果指示所述第N个图像帧存在第一车牌，则根据所述第一车牌在所述第N个图像帧的位置以及所述第一车牌的内容信息，从所述第一车牌中分割出文字区域和数字区域；
    第一车牌内容识别单元，用于对从所述第一车牌中分割出的所述文字区域和所述数字区域进行识别，得到所述第一车牌的第一识别结果。
  12. 一种电子设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至10任一项所述的方法。
  13. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至10任一项所述的方法。
PCT/CN2020/140930 2020-12-29 2020-12-29 车牌识别方法、装置及电子设备 WO2022141073A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202080003842.6A CN112997190B (zh) 2020-12-29 2020-12-29 车牌识别方法、装置及电子设备
PCT/CN2020/140930 WO2022141073A1 (zh) 2020-12-29 2020-12-29 车牌识别方法、装置及电子设备
US17/555,835 US20220207889A1 (en) 2020-12-29 2021-12-20 Method for recognizing vehicle license plate, electronic device and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/140930 WO2022141073A1 (zh) 2020-12-29 2020-12-29 车牌识别方法、装置及电子设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/555,835 Continuation-In-Part US20220207889A1 (en) 2020-12-29 2021-12-20 Method for recognizing vehicle license plate, electronic device and computer readable storage medium

Publications (1)

Publication Number Publication Date
WO2022141073A1 true WO2022141073A1 (zh) 2022-07-07

Family

ID=76344771

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/140930 WO2022141073A1 (zh) 2020-12-29 2020-12-29 车牌识别方法、装置及电子设备

Country Status (3)

Country Link
US (1) US20220207889A1 (zh)
CN (1) CN112997190B (zh)
WO (1) WO2022141073A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116311215A (zh) * 2023-05-22 2023-06-23 成都运荔枝科技有限公司 一种车牌识别方法

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112560856B (zh) * 2020-12-18 2024-04-12 深圳赛安特技术服务有限公司 车牌检测识别方法、装置、设备及存储介质
US11978267B2 (en) * 2022-04-22 2024-05-07 Verkada Inc. Automatic multi-plate recognition
US11557133B1 (en) 2022-04-22 2023-01-17 Verkada Inc. Automatic license plate recognition
CN117437625A (zh) * 2022-07-12 2024-01-23 青岛云天励飞科技有限公司 车牌识别方法及相关设备
CN115082832A (zh) * 2022-07-13 2022-09-20 北京京东乾石科技有限公司 信息识别方法、装置及存储介质
CN117373259B (zh) * 2023-12-07 2024-03-01 四川北斗云联科技有限公司 高速公路车辆逃费行为识别方法、装置、设备和存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101118601A (zh) * 2007-09-06 2008-02-06 西东控制集团（沈阳）有限公司 一种用于远距离车辆识别系统的车载器
CN101159039A (zh) * 2007-11-14 2008-04-09 华中科技大学 超高频车辆识别卡及其识别装置
CN109447074A (zh) * 2018-09-03 2019-03-08 中国平安人寿保险股份有限公司 一种车牌识别方法及终端设备

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9025825B2 (en) * 2013-05-10 2015-05-05 Palo Alto Research Center Incorporated System and method for visual motion based object segmentation and tracking
CN104298976B (zh) * 2014-10-16 2017-09-26 电子科技大学 基于卷积神经网络的车牌检测方法
JP6720694B2 (ja) * 2016-05-20 2020-07-08 富士通株式会社 画像処理プログラム、画像処理方法および画像処理装置
US9838643B1 (en) * 2016-08-04 2017-12-05 Interra Systems, Inc. Method and system for detection of inherent noise present within a video source prior to digital video compression
CN108108734B (zh) * 2016-11-24 2021-09-24 杭州海康威视数字技术股份有限公司 一种车牌识别方法及装置
US10839257B2 (en) * 2017-08-30 2020-11-17 Qualcomm Incorporated Prioritizing objects for object recognition
WO2019241224A1 (en) * 2018-06-11 2019-12-19 Raytheon Company Architectures for vehicle tolling
CN111832337A (zh) * 2019-04-16 2020-10-27 高新兴科技集团股份有限公司 一种车牌识别方法及装置
CN110674821B (zh) * 2019-09-24 2022-05-03 浙江工商大学 一种非机动车车牌识别方法
CN111368830B (zh) * 2020-03-03 2024-02-27 西北工业大学 基于多视频帧信息和核相关滤波算法的车牌检测识别方法
CN111582263A (zh) * 2020-05-12 2020-08-25 上海眼控科技股份有限公司 车牌识别方法、装置、电子设备及存储介质
KR20220049864A (ko) * 2020-10-15 2022-04-22 에스케이텔레콤 주식회사 인식한 번호판의 각도에 기초하여 차량의 번호를 인식하는 방법

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101118601A (zh) * 2007-09-06 2008-02-06 西东控制集团（沈阳）有限公司 一种用于远距离车辆识别系统的车载器
CN101159039A (zh) * 2007-11-14 2008-04-09 华中科技大学 超高频车辆识别卡及其识别装置
CN109447074A (zh) * 2018-09-03 2019-03-08 中国平安人寿保险股份有限公司 一种车牌识别方法及终端设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116311215A (zh) * 2023-05-22 2023-06-23 成都运荔枝科技有限公司 一种车牌识别方法
CN116311215B (zh) * 2023-05-22 2023-11-17 成都运荔枝科技有限公司 一种车牌识别方法

Also Published As

Publication number Publication date
CN112997190B (zh) 2024-01-12
US20220207889A1 (en) 2022-06-30
CN112997190A (zh) 2021-06-18

Similar Documents

Publication Publication Date Title
WO2022141073A1 (zh) 车牌识别方法、装置及电子设备
CN108009543B (zh) 一种车牌识别方法及装置
US11861888B2 (en) Logo recognition in images and videos
US9076056B2 (en) Text detection in natural images
US9171204B2 (en) Method of perspective correction for devanagari text
CN110276342B (zh) 车牌辨识方法以及其系统
JP5775225B2 (ja) マルチレイヤ連結成分をヒストグラムと共に用いるテキスト検出
CN111145214A (zh) 目标跟踪方法、装置、终端设备及介质
US20210150182A1 (en) Cloud detection from satellite imagery
CN111931864B (zh) 基于顶点距离与交并比多重优化目标检测器的方法与系统
WO2013112746A1 (en) Detecting and correcting skew in regions of text in natural images
CN111209920B (zh) 一种复杂动态背景下飞机检测方法
WO2024130857A1 (zh) 一种物品陈列检测方法、装置、设备及可读存储介质
Asgarian Dehkordi et al. Vehicle type recognition based on dimension estimation and bag of word classification
US10217020B1 (en) Method and system for identifying multiple strings in an image based upon positions of model strings relative to one another
CN111556362A (zh) 一种车身广告植入方法、装置、电子设备及存储介质
US11087122B1 (en) Method and system for processing candidate strings detected in an image to identify a match of a model string in the image
Park et al. Multi-style license plate recognition system using k-nearest neighbors
CN112580516A (zh) 一种道路场景识别方法、装置、设备及存储介质
CN109934185B (zh) 数据处理方法及装置、介质和计算设备
CN116580230A (zh) 一种目标检测方法及一种分类模型的训练方法
CN111488776A (zh) 对象检测方法、对象检测装置和电子设备
CN113158976B (zh) 地面箭头的识别方法、系统、终端及计算机可读存储介质
CN114419531A (zh) 目标检测方法、目标检测系统以及计算机可读存储介质
CN113762027A (zh) 一种异常行为的识别方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20967421

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20967421

Country of ref document: EP

Kind code of ref document: A1