WO2021103897A1 - 车牌号码识别方法、装置、电子设备及存储介质 - Google Patents

车牌号码识别方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2021103897A1
WO2021103897A1 PCT/CN2020/124085 CN2020124085W WO2021103897A1 WO 2021103897 A1 WO2021103897 A1 WO 2021103897A1 CN 2020124085 W CN2020124085 W CN 2020124085W WO 2021103897 A1 WO2021103897 A1 WO 2021103897A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature
license plate
verification
plate number
result
Prior art date
Application number
PCT/CN2020/124085
Other languages
English (en)
French (fr)
Inventor
曾卓熙
胡文泽
王孝宇
Original Assignee
深圳云天励飞技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳云天励飞技术股份有限公司 filed Critical 深圳云天励飞技术股份有限公司
Priority to US17/623,903 priority Critical patent/US11645857B2/en
Publication of WO2021103897A1 publication Critical patent/WO2021103897A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/148—Segmentation of character regions
    • G06V30/153—Segmentation of character regions using recognition of characters or words
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/32—Normalisation of the pattern dimensions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/12—Detection or correction of errors, e.g. by rescanning the pattern
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/146—Aligning or centring of the image pick-up or image-field
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/18—Extraction of features or characteristics of the image
    • G06V30/18105—Extraction of features or characteristics of the image related to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
    • G06V30/1916—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G06V10/247—Aligning, centring, orientation detection or correction of the image by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids

Definitions

  • the present invention relates to the field of artificial intelligence technology, and in particular to a method, device, electronic equipment and storage medium for recognizing a person's license plate number.
  • Image recognition is currently one of the commonly used technologies in traffic, community or parking lot management. For example, using image recognition-based license plate number recognition to identify the vehicle's license plate number.
  • traditional license plate number recognition is generally divided into multiple independent steps, such as: 1.
  • Image normalization the license plate image is programmed into a "formal map" through computer vision methods (such as homography matrix homography, etc.).
  • Image preprocessing processing the occlusion, dirt, light, etc. of the image here (such as binary distribution binarized, etc.) 3.
  • Character segmentation character segmentation through computer vision methods (such as edge detection edge detection, etc.) 4.
  • Character recognition Recognize the segmented characters (such as random forest random forest, support vector machine svm, logistic regression logistic machine learning or deep learning methods such as regression). As a result, the errors in each step may accumulate, resulting in a poor final recognition effect, and it is not easy to locate where the problem occurs.
  • traditional license plate recognition has relatively high requirements for input images, and has strict requirements on angle and clarity. The various limitations of traditional license plate recognition result in strict requirements for the installation of cameras and monitoring scenes, and the recognition rate is easily affected by weather and light. Therefore, the traditional license plate number recognition error will output the wrong recognition result, the output error rate is high, there is no verification mechanism, and the verification of the recognition result usually requires manual verification.
  • the embodiment of the present invention provides a method for recognizing a license plate number, which can reduce the output error rate of the license plate number recognition.
  • an embodiment of the present invention provides a method for recognizing a license plate number, including:
  • the extraction of the license plate number features of the image to be recognized through a pre-trained convolutional neural network includes:
  • the license plate number feature is obtained.
  • the first convolutional network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a first downsampling layer, a second downsampling layer, a third downsampling layer, and a fourth downsampling layer.
  • a downsampling layer, the second convolutional network includes a fourth convolutional layer, wherein the inputs of the first downsampling layer and the second downsampling layer are respectively connected to the output of the first convolutional layer, and the A convolutional network and a second convolutional network perform feature extraction on the corrected image to be recognized, and correspondingly obtain the first feature and the second feature of the same size in sequence, including:
  • the target size is the size of the second feature.
  • the obtaining the license plate number feature based on the first feature and the second feature includes:
  • the first feature, the second feature, the third feature, and the fourth feature are stacked in the channel dimension to obtain the license plate number feature.
  • the intermediate convolution result includes at least one of the first convolution result, the second convolution result, the third convolution result, and the fourth convolution result.
  • the process of extracting the intermediate convolution result, and extracting the first verification feature and/or the second verification feature according to the intermediate convolution result including:
  • the second verification feature is extracted.
  • the first verification feature is a color feature
  • the verification of the license plate number feature according to the first verification feature includes:
  • the image to be recognized includes license plate aspect ratio information
  • the second verification feature is a size feature
  • the verification of the license plate number feature according to the second verification feature includes:
  • an embodiment of the present invention provides a license plate number recognition device, including:
  • the first extraction module is configured to extract the license plate number features of the image to be recognized through a pre-trained convolutional neural network, where the image to be recognized includes the license plate number;
  • the second extraction module is configured to extract an intermediate convolution result in the process of extracting the license plate number feature, and extract the first verification feature and/or the second verification feature according to the intermediate convolution result;
  • a verification module configured to verify whether the license plate number feature is correct according to the first verification feature and/or the second verification feature
  • the output module is used for outputting the result of the license plate number predicted according to the characteristics of the license plate number if it is correct.
  • an embodiment of the present invention provides an electronic device including: a memory, a processor, and a computer program stored on the memory and capable of running on the processor, and when the processor executes the computer program The steps in the license plate number recognition method provided by the embodiment of the present invention are realized.
  • an embodiment of the present invention provides a computer-readable storage medium having a computer program stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for recognizing a license plate number provided by the embodiment of the present invention is implemented A step of.
  • the license plate number feature of the image to be recognized is extracted through a pre-trained convolutional neural network, and the image to be recognized includes the license plate number; in the process of extracting the license plate number feature, extraction is performed according to the intermediate convolution result The first verification feature and/or the second verification feature; verify whether the license plate number feature is correct according to the first verification feature and/or the second verification feature; if it is correct, output the license plate predicted based on the license plate number feature Number result.
  • the intermediate feature is extracted as a verification feature to verify whether the extracted license plate number feature is correct. Only when the verification is passed, the license plate number result is output, which reduces the license plate number recognition The output error rate of the result.
  • FIG. 1 is a flowchart of a method for recognizing a license plate number provided by an embodiment of the present invention
  • FIG. 2 is a flowchart of another method for recognizing a license plate number provided by an embodiment of the present invention
  • FIG. 3 is a flowchart of another method for recognizing a license plate number provided by an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of the structure of a license plate number recognition device provided by an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of another vehicle license plate number recognition device provided by an embodiment of the present invention.
  • Fig. 6 is a schematic structural diagram of another vehicle license plate number recognition device provided by an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of another vehicle license plate number recognition device provided by an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of another vehicle license plate number recognition device provided by an embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of another vehicle license plate number recognition device provided by an embodiment of the present invention.
  • FIG. 10 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.
  • FIG. 1 is a flowchart of a method for recognizing a license plate number according to an embodiment of the present invention. As shown in FIG. 1, it includes the following steps:
  • the image to be recognized may include the license plate number and license plate parameters.
  • the above-mentioned image to be recognized can be a static image of a vehicle license plate or an image frame of a dynamic video uploaded by the user, or a static image or a static image of the vehicle license plate obtained by cameras deployed on traffic roads, entrances and exits of communities, and entrances and exits of parking lots.
  • the aforementioned license plate parameters include the height and width parameters of the license plate.
  • the above-mentioned license plate numbers in the image to be recognized may be one or more, that is, there are one or more license plate numbers to be recognized in an image to be recognized.
  • the above-mentioned pre-trained convolutional neural network packs convolutional layers (conv block), downsampling layers (downsample), fully connected layers (FC), and so on.
  • the above-mentioned convolutional layer includes input layer (input), activation layer (relu), batch normalization layer (batch norm), convolution calculation layer (conv), etc.
  • a convolution calculation layer the output of the convolution calculation is completed. It will be added to the input of the convolutional layer to make the image information redundant and improve the convergence speed of network training.
  • the next convolution calculation layer can choose whether to convolve the content or the unconvolved content as the input Calculation.
  • the above-mentioned down-sampling layer reduces the image size by sampling pixels to obtain a larger receptive field.
  • the down-sampling layer may also be called a pooling layer.
  • the above-mentioned fully connected layer connects each channel obtained by convolution to obtain the overall feature characterization.
  • the image to be recognized is input into the convolutional neural network, and convolution calculation is performed on the image to be recognized through each convolutional layer in the convolutional neural network in order to extract the characteristics of the license plate number in the image to be recognized.
  • the image before inputting the image to be recognized into the convolutional layer, the image can be preprocessed.
  • the preprocessing described above includes image normalization, image size conversion, image correction, and so on.
  • the aforementioned convolutional neural network is preceded by a spatial transformation network, such as STN (Spatial Transform Network) spatial transformation network, which performs spatial transformation on the image to be recognized so that the image to be recognized meets the input expectations of the convolutional layer.
  • a spatial transformation network such as STN (Spatial Transform Network) spatial transformation network, which performs spatial transformation on the image to be recognized so that the image to be recognized meets the input expectations of the convolutional layer.
  • the above process of extracting the features of the license plate number can be understood as the convolution process of the image to be recognized in the convolutional neural network.
  • each convolutional layer corresponds to a convolution result.
  • each convolution calculation layer corresponds to a convolution result.
  • the convolution calculation process is the process of calculating the input matrix through the convolution kernel.
  • the above input matrix can be understood as the pixel matrix of the image to be recognized, or the feature point matrix in the feature image, the feature image It is the image obtained by convolution of the image to be recognized.
  • the same number of channels as the convolution kernel will be generated, and the channels are added to obtain the corresponding features
  • Images, in one way of understanding, channels can also be called feature images.
  • the foregoing first verification feature and the second verification feature are different verification features, and the foregoing verification feature may be a license plate color, a license plate size, a character color, and other features based on the structure of the license plate itself. Since the structural parameters of the license plate itself are standardized, these standardized structural parameters can be used as a priori information to verify the license plate number recognition.
  • the aforementioned intermediate convolution result is the convolution result of the output of each convolution layer when feature extraction is performed on the image to be recognized in the convolutional neural network. It should be noted that, since there are multiple convolutional layers in the convolutional neural network, the first verification feature and the second convolution feature may be the convolution result of any one of the multiple convolutional layers.
  • the aforementioned first verification feature and the second verification feature are different verification features, and the aforementioned verification features may be license plate color, license plate size, character color, and other features based on the structure of the license plate itself.
  • the license plate number can be verified based on the above-mentioned characteristics of the license plate color, license plate size, character color, etc., as the prior information for the license plate number verification.
  • the aforementioned first verification feature is a color feature.
  • a branch network can be used to perform feature recognition on the extracted intermediate convolution results to identify the first verification feature and determine the color category of the first verification feature; predict the characteristics of the license plate number to determine the ownership of the license plate number License plate type; determine the color type of the license plate number according to the license plate type of the license plate number; determine whether the color type of the first verification feature is the same as the color type of the license plate number; if they are the same, the verification result is passed ; If it is different, the verification result is not passed.
  • the license plate includes the license plate identification characters and the color of the license plate.
  • the license plate identification characters include: the Chinese characters for the abbreviation of the province, autonomous region, and municipality directly under the Central Government, the capital letter identification of the licensing agency code, the capital letter/number identification of the serial number; in addition, it can also include the vehicle to which the vehicle belongs Characteristic Chinese character identification, for example, the sequence of a police car will have the character "Police", and the sequence of a driving school training car will have the character of "Learning”, etc.
  • the color of the license plate mentioned above refers to the background color of the license plate, including: blue (general household car license plate), yellow (large vehicle or agricultural vehicle license plate and trainer vehicle license plate), white (special vehicle license plate, such as military vehicle police car license plate and racing license plate) ), black (the license plate of a car used by foreign companies and foreign companies or a foreign car), for example, the common domestic civilian license plate, the color of the license plate is blue.
  • blue general household car license plate
  • yellow large vehicle or agricultural vehicle license plate and trainer vehicle license plate
  • white special vehicle license plate, such as military vehicle police car license plate and racing license plate
  • black the license plate of a car used by foreign companies and foreign companies or a foreign car
  • the color of the license plate is blue.
  • the corresponding license plate number is predicted, and the license plate color corresponding to the number is searched in the license plate number database, and the recognized license plate color is compared with the searched license plate color to determine the recognized license plate number.
  • the color of the license plate is the same as the color of the searched license plate, if they are the same, both are blue, it means that there is no error in the recognition of the color, and the verification can be considered as passed.
  • the color of the searched license plate is not blue, it means There is an error in the color recognition. Since it is the color extracted during the recognition process, it can be considered that there is an error in the recognition process, and then the verification is considered to be failed.
  • the image to be recognized includes the aspect ratio of the license plate
  • the second verification feature is a size feature
  • the verification of whether the license plate number feature is correct according to the second verification feature may be: recognizing the second verification feature to determine the second verification The corresponding size parameter of the feature, and the verification aspect ratio of the second verification feature is calculated according to the size parameter; determine whether the verification aspect ratio of the second verification feature is the same as the aspect ratio of the image to be recognized including the license plate; if they are the same, then The verification result is passed; if it is different, the verification result is not passed.
  • the above-mentioned license plate aspect ratio can be preset.
  • the size feature corresponding to the height and width is calculated to obtain the corresponding recognition aspect ratio, and the recognition aspect ratio is compared with the preset license plate height Compare the aspect ratios to determine whether they are the same.
  • the aspect ratio of the license plate may also be calculated according to the pixel coordinates of the license plate in the image to be recognized after the image to be recognized is input, and then compared with the recognized aspect ratio to determine whether it is the same.
  • first verification feature and second verification feature can verify the license plate number feature alone, or can be combined to verify the license plate number feature.
  • the license plate number feature may be verified by combining the first verification feature with the second verification feature, where the first verification feature may be the base color of the license plate, and the second verification feature may be the color of the character.
  • the background colors of the license plate are blue, yellow, white, and black. Among them: blue is the license plate for small vehicles (including small-tonnage trucks); yellow is the license plate for large vehicles or agricultural vehicles and trainer vehicle license plates, as well as new products for finalized test vehicles, motorcycles, etc.; white is special vehicle license plates (such as military vehicle police vehicle license plate and racing license plate); black is the license plate of foreign-owned vehicles and foreign-owned enterprises.
  • the background and character colors of the license plate are: large civilian cars: black on a yellow background; small civilian cars: white on a blue background; special vehicles for armed police: red "WJ" on a white background with black letters; other foreign cars: white on a black background; Foreign cars in the consulate: black background with white letters and a hollow "Shi" sign; test license: white background with red letters, with a "test” sign before the numbers; temporary license plates: white background with red letters, with the word “temporary” before the numbers; car supplement License plate: black on white. Therefore, the characteristics of the license plate number can be verified by combining the background color of the license plate and the color of the characters.
  • the color of the recognized license plate is obtained, and by recognizing the second verification feature, the color of the recognized characters is obtained, and the characteristics of the license plate number are predicted to obtain the license plate number.
  • the license plate number is searched in the license plate number database.
  • the license plate color and character color corresponding to the license plate number are compared with the recognized license plate color and the recognized character color, and the comparison result is judged to determine whether the verification is passed.
  • the result of the license plate number and the corresponding abnormal prompt may be output.
  • the abnormal prompt includes a verification record.
  • the prompt may be: "The color of the recognition result does not match, please pay attention.”
  • multiple parallel convolutional neural networks can be deployed to respectively recognize multiple images to be recognized.
  • multiple parallel convolutional neural networks are separately trained through the same data set. If the verification fails, transfer the image to be recognized to another parallel convolutional neural network for secondary recognition. If the result of the license plate number obtained by the two recognitions is the same, then output; if the license plate number obtained by the two recognitions is different , Then output the result of the license plate number verified by the first verification feature and/or the second verification feature; if the license plate numbers obtained by the two recognitions are different, and the verification of the first verification feature and/or the second verification feature fails, then output two The result of the secondary license plate number and the corresponding abnormal prompt, the prompt includes the verification record during the two identification processes.
  • the license plate number feature of the image to be recognized is extracted through a pre-trained convolutional neural network, and the image to be recognized includes the license plate number; in the process of extracting the license plate number feature, extraction is performed according to the intermediate convolution result The first verification feature and/or the second verification feature; verify whether the license plate number feature is correct according to the first verification feature and/or the second verification feature; if it is correct, output the license plate predicted based on the license plate number feature Number result.
  • the intermediate feature is extracted as a verification feature to verify whether the extracted license plate number feature is correct. Only when the verification is passed, the license plate number result is output, which reduces the license plate number recognition The output error rate of the result.
  • license plate number recognition method provided by the embodiment of the present invention can be applied to mobile phones, monitors, computers, servers and other devices that need to recognize license plate numbers.
  • FIG. 2 is a flowchart of another method for recognizing a license plate number provided by an embodiment of the present invention.
  • the difference from the embodiment in FIG. 1 is that the pre-trained convolutional neural network includes the first volume
  • the product network and the second convolution network, as shown in Figure 2 include the following steps:
  • the aforementioned pre-trained spatial transformation network can be STN (Spatial Transform Network) spatial transformation network.
  • the aforementioned spatial transformation network may be set before the first convolutional network and the second convolutional network to correct the image to be recognized, so that the image to be recognized is corrected to the desired input of the convolutional layer.
  • a spatial transformation network can also be set up between the first convolutional network and the second convolutional network to correct the input of the next convolutional network so that the output of the previous convolutional network is corrected to the next convolutional network. The expected input of the network.
  • first convolutional network and the second convolutional network include multiple convolutional layers
  • a spatial transformation network can also be set between the convolutional layers to correct the input of the next convolutional layer to make the previous convolutional layer
  • the output of the build-up layer is corrected to the expected input of the next convolutional layer.
  • the above correction can be understood as spatial transformation and alignment of the image to be recognized, and may include translation, scaling, and rotation of the image to be recognized. In this way, the deployment requirements for image capture equipment can be reduced.
  • the above-mentioned first feature and the second feature can be understood as feature images calculated by convolution, and the above-mentioned same size refers to the size of the feature image.
  • the above-mentioned feature images have the same size, which facilitates the addition of each feature image.
  • the above-mentioned feature extraction is to perform convolution calculation on the image to be recognized according to the trained convolution kernel, so as to extract the feature information that meets the expectations.
  • the convolution result of the first convolutional network down-sampling is performed when necessary to obtain the first feature, and according to the convolution result of the second convolutional network, the second feature is obtained.
  • the size of the feature image extracted by the first convolutional network may be larger than the size of the feature image extracted by the second convolutional network.
  • the feature image extracted by the first convolution network can be down-sampled to obtain the feature image corresponding to the first feature, and the feature image extracted by the first convolution network can be down-sampled to the feature extracted by the second convolution.
  • the first feature and the second feature can be added to obtain the license plate number feature.
  • the addition of the first feature and the second feature can be understood as the addition of corresponding feature maps, that is, the addition of corresponding feature points on the feature image.
  • the first feature and the second feature are added in the channel dimension to obtain the license plate number feature.
  • the first convolutional network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a first downsampling layer, a second downsampling layer, a third downsampling layer, and a fourth downsampling layer
  • the second convolutional network includes a fourth convolutional layer, wherein the inputs of the first downsampling layer and the second downsampling layer are respectively connected to the output of the first convolution area.
  • feature extraction is performed on the corrected image to be recognized through the first convolution layer, and the first convolution result is down-sampled through the first down-sampling layer to obtain the first feature that meets the target size through sampling;
  • the first convolution result of the four downsampling layers is down-sampled, and then the fourth down-sampling result is extracted through the second convolution layer for feature extraction, and the second convolution result is down-sampled through the second down-sampling layer to obtain a sample
  • the third feature that meets the target size; down-sampling is performed through the second convolution result of the third down-sampling layer to obtain the third down-sampling result that meets the target size by sampling, and then the third down-sampling result is performed through the third convolution layer
  • Feature extraction obtain the fourth feature based on the third convolution result; perform feature extraction on the fourth feature through a four-convolution network, and obtain the second feature based on the fourth convolution result; wherein, the target size is the second feature size of.
  • the first feature, the second feature, the third feature, and the fourth feature are added in the channel dimension to obtain the license plate number feature.
  • the first verification network can be understood as a branch network of the convolutional network. Specifically, the first verification network is parallel to the second convolutional network, and the convolution result of the first convolutional network is input to the second convolutional network at the same time. Jaeger Network and First Verification Network.
  • the first convolutional network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a first downsampling layer, a second downsampling layer, a third downsampling layer, and a fourth downsampling layer
  • the second convolutional network includes a fourth convolutional layer, wherein the inputs of the first downsampling layer and the second downsampling layer are respectively connected to the output of the first convolution area.
  • first convolutional network second convolutional network
  • first verification network belong to the same convolutional neural network, and can be trained through the same data set.
  • the above-mentioned convolutional neural network passes CTC loss (Connectionist temporal classification loss, connected to the temporal classification loss) for training, specifically, the first convolutional network and the second convolutional network pass CTC Loss is trained, and the first verification network is trained through cross entropy.
  • CTC loss Connectionist temporal classification loss, connected to the temporal classification loss
  • the first convolutional network and the second convolutional network pass CTC Loss is trained, and the first verification network is trained through cross entropy.
  • the first convolutional network and the second convolutional network can learn the time series classification to recognize the license plate number according to the learned time series, so that the first verification network can learn the classification of the verification feature.
  • step 204 the second verification feature is extracted according to the fourth convolution result.
  • the convolution result of the second convolution network is input to the second verification network, and the second verification feature is extracted.
  • the second verification network can be understood as a branch network of the convolutional network.
  • the second verification network is parallel to the second convolutional network, and the convolution result of the first convolutional network is simultaneously input to the second convolutional network and The second verification network.
  • the above-mentioned first convolutional network, second convolutional network, and first verification network belong to the same convolutional neural network, and can be trained through the same data set.
  • This step is similar to step 103 in the embodiment of FIG. 1, and will not be repeated here.
  • the verification feature is extracted from the convolution results obtained from different network depths in the convolutional neural network, so that the verification feature and the license plate number feature can be extracted from the convolution result of a convolutional neural network, and only The same data set is needed to train the convolutional neural network, which reduces the difficulty of training the convolutional neural network and the resources required for deployment.
  • FIG. 3 is a flowchart of another method for recognizing a license plate number according to an embodiment of the present invention. As shown in FIG. 3, the method includes the following:
  • the above-mentioned image to be recognized will be processed as a (198, 48, 3) RGB image.
  • the image to be recognized is corrected at the spatial transformation network layer.
  • the corrected image to be recognized will be input to the convolutional layer to be convolved through a 3 ⁇ 3 convolution kernel, and the obtained convolution result will be down-sampled to obtain a (96, 48, 64) feature image; for the (96, 48) , 64)
  • the feature image is subjected to convolution calculation to obtain the corresponding convolution result.
  • the feature image of (24, 24, 64) is obtained as the added feature.
  • the down-sampling can be an average pool Change, that is, take the average of the feature values extracted from the feature image by the pooling kernel as the new feature value.
  • the convolution result will be down-sampled once to obtain (48, 48 , 128) feature image as the input of the next convolutional layer; two convolution calculations are performed on the feature image of (48, 48, 128) through two convolution layers in turn, and the corresponding convolution result is obtained.
  • the convolution result is averaged and pooled, and the feature image of (24, 24, 128) is obtained as the addition feature, and the convolution result is down-sampled to obtain the feature image of (24, 24, 256) as the next convolution
  • the convolution result is (24, 24, 256) Feature image, the feature image of (24, 24, 256) is used as the addition feature, and at the same time as the input of the next convolutional layer; the feature image of (24, 24, 256) is sequentially input to the horizontal convolutional layer , The convolution calculation is performed in the vertical convolution layer, and the feature image of (24, 24, 256) is obtained as the added feature.
  • the above-mentioned horizontal convolutional layer and vertical convolutional layer can extract the size parameter of the license plate, and the size parameter can be used as the verification feature to verify the license plate number feature.
  • the convolution kernel in the horizontal convolution layer is 11 ⁇ 1
  • the convolution kernel in the vertical convolution layer is 1 ⁇ 3
  • the shape of the convolution kernel of the horizontal convolution layer and the vertical convolution layer is related to the shape of the license plate.
  • the above-obtained additive features (24, 24, 64), (24, 24, 128), (24, 24, 256), (24, 24, 256) are stacked in channel dimensions to obtain (24, 24, 704) license plate number feature, the channel dimension of the license plate number feature is 704.
  • the number feature is input into the convolutional layer to perform convolution calculation through a 3 ⁇ 3 convolution kernel, and the license plate number feature of (24, 24, 16) is obtained.
  • the channel dimension of the license plate number feature is 16.
  • the license plate number features are connected through the fully connected layer to obtain a feature vector.
  • the fully connected layer functions to map the learned "distributed feature representation" to the sample label space, that is, to map a 16-dimensional vector to a 70 Make predictions in the sample label space of dimensions.
  • a (10, 1) license plate number result is output through the output layer, where 10 includes characters, start and end characters, and alignment characters, and 1 is the number of results.
  • the branch network can be color
  • the classification network performs color classification on the (24, 24, 256) feature image.
  • the branch network outputs a type of color classification result as the color category to which the verification feature belongs.
  • the color category to which the verification feature belongs is used to verify the license plate number feature.
  • the corresponding license plate number is predicted according to the characteristics of the license plate number, and the license plate color corresponding to the license plate number is searched in the license plate number database as the color category of the license plate number, so that the color category of the feature and the color of the license plate number can be verified by judgment Whether the categories are the same to determine whether the license plate number features pass the verification.
  • the verification feature is extracted from the convolution results obtained from different network depths in the convolutional neural network, so that the verification feature and the license plate number feature can be extracted from the convolution result of a convolutional neural network, and only The same data set is needed to train the convolutional neural network, which reduces the difficulty of training the convolutional neural network and the resources required for deployment.
  • the convolution process through multiple downsampling operations, the receptive field of the convolutional neural network is larger, and the robustness of the neural network can be improved.
  • license plate number recognition method provided in the embodiments of the present invention can be applied to mobile phones, monitors, computers, servers and other devices that need to perform license plate number recognition.
  • FIG. 4 is a schematic structural diagram of a license plate number recognition device provided by an embodiment of the present invention. As shown in FIG. 4, the device includes:
  • the first extraction module 401 is configured to extract the license plate number features of the image to be recognized through a pre-trained convolutional neural network, where the image to be recognized includes the license plate number;
  • the second extraction module 402 is configured to extract an intermediate convolution result in the process of extracting the license plate number feature, and extract the first verification feature and/or the second verification feature according to the intermediate convolution result;
  • the verification module 403 is configured to verify whether the license plate number feature is correct according to the first verification feature and/or the second verification feature;
  • the output module 404 is configured to output the result of the license plate number predicted according to the characteristics of the license plate number if it is correct.
  • the pre-trained convolutional neural network includes a first convolutional network and a second convolutional network
  • the first extraction module 401 includes:
  • the preprocessing unit 4011 is configured to correct the image to be recognized through the spatial transformation network to obtain the corrected image to be recognized;
  • the first feature extraction unit 4012 is configured to perform feature extraction on the corrected image to be recognized through the first convolutional network and the second convolutional network in sequence, and correspondingly obtain the first feature and the second feature of the same size in sequence;
  • the processing unit 4013 is configured to obtain the license plate number feature based on the first feature and the second feature.
  • the first convolutional network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a first downsampling layer, a second downsampling layer, and a third convolutional layer.
  • a downsampling layer and a fourth downsampling layer, the second convolutional network includes a fourth convolutional layer, wherein the inputs of the first downsampling layer and the second downsampling layer are respectively connected to the output of the first convolution area ,
  • the first feature extraction unit 4012 includes:
  • the first calculation sub-unit 40121 is configured to extract features of the corrected image to be recognized through the first convolution layer, and down-sample the first convolution result through the first down-sampling layer to obtain a sample that meets the target The first feature of size;
  • the second calculation sub-unit 40122 is configured to perform down-sampling through the first convolution result of the fourth down-sampling layer, and then perform feature extraction on the fourth down-sampling result through the second convolution layer, and combine the second convolution result Down-sampling is performed through the second down-sampling layer to obtain a third feature that meets the target size through sampling;
  • the third calculation subunit 40123 is configured to down-sample the second convolution result of the third down-sampling layer to obtain a third down-sampling result that meets the target size by sampling, and then pass the third down-sampling result through the third volume Multilayer performs feature extraction, and obtains the fourth feature based on the third convolution result;
  • the fourth calculation subunit 40124 is configured to perform feature extraction on the fourth feature through a four-convolution network, and obtain the second feature based on the fourth convolution result;
  • the target size is the size of the second feature.
  • the processing unit 4013 is further configured to stack the first feature, the second feature, the third feature, and the fourth feature in the channel dimension to obtain the license plate number feature.
  • the intermediate convolution result includes at least one of the first convolution result, the second convolution result, the third convolution result, and the fourth convolution result.
  • the second extraction module 402 includes:
  • the second feature extraction unit 4021 is configured to input any one of the first convolution result, the second convolution result, and the third convolution result into the first verification network to extract the first verification feature; and/or
  • the third feature extraction unit 4022 is configured to extract and obtain a second verification feature according to the fourth convolution result.
  • the first verification feature is a color feature
  • the verification module 403 includes:
  • the first determining unit 4031 is configured to identify the first verification feature and determine the color category of the first verification feature
  • the second determining unit 4032 is configured to predict the characteristics of the license plate number, and determine the license plate category to which the license plate number belongs;
  • the third determining unit 4033 is configured to determine the color category of the license plate number according to the license plate category of the license plate number;
  • the first determining unit 4034 is configured to determine whether the color category of the first verification feature is the same as the color category of the license plate number; if they are the same, the verification result is passed; if they are different, the verification result is not passed.
  • the image to be recognized includes license plate aspect ratio information
  • the second verification feature is a size feature
  • the verification module 403 includes:
  • the fourth determining unit 4035 is configured to identify the second verification feature, determine the corresponding size parameter of the second verification feature, and calculate the verification height and width of the second verification feature according to the size parameter ratio;
  • the second judging unit 4036 is used to judge whether the verification aspect ratio of the second verification feature is the same as the aspect ratio of the image to be recognized including the license plate; if they are the same, the verification result is passed; if they are different, the verification result is Fail.
  • license plate number recognition device provided in the embodiment of the present invention can be applied to mobile phones, monitors, computers, servers and other devices that need to perform license plate number recognition.
  • the license plate number recognition device provided by the embodiment of the present invention can realize each process realized by the license plate number recognition method in the foregoing method embodiment, and can achieve the same beneficial effects. To avoid repetition, I won’t repeat them here.
  • FIG. 10 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As shown in FIG. 10, it includes: a memory 1002, a processor 1001, and a memory 1002 stored in the memory 1002 and available on the processor. A computer program running on 1001, where:
  • the processor 1001 is configured to call a computer program stored in the memory 1002, and execute the following steps:
  • the result of the license plate number predicted according to the characteristics of the license plate number is output.
  • the pre-trained convolutional neural network includes a first convolutional network and a second convolutional network.
  • the processor 1001 extracts the license plate number of the image to be recognized through the pre-trained convolutional neural network.
  • Features include:
  • the license plate number feature is obtained.
  • the first convolutional network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a first downsampling layer, a second downsampling layer, a third downsampling layer, and a fourth downsampling layer.
  • a downsampling layer, the second convolutional network includes a fourth convolutional layer, wherein the inputs of the first downsampling layer and the second downsampling layer are respectively connected to the output of the first convolution area, the processor 1001
  • the execution of the feature extraction of the corrected image to be recognized through the first convolutional network and the second convolutional network in sequence, and correspondingly obtaining the first feature and the second feature of the same size in sequence includes:
  • the target size is the size of the second feature.
  • the execution of the processor 1001 to obtain the license plate number feature based on the first feature and the second feature includes:
  • the first feature, the second feature, the third feature, and the fourth feature are stacked in the channel dimension to obtain the license plate number feature.
  • the intermediate convolution result includes at least one of the first convolution result, the second convolution result, the third convolution result, and the fourth convolution result.
  • the process of extracting an intermediate convolution result in the process of the license plate number feature, and extracting the first verification feature and/or the second verification feature according to the intermediate convolution result includes:
  • the second verification feature is extracted.
  • the first verification feature is a color feature
  • the verification of the license plate number feature according to the first verification feature performed by the processor 1001 includes:
  • the image to be recognized includes license plate aspect ratio information
  • the second verification feature is a size feature
  • the processor 1001 verifies whether the license plate number feature is correct according to the second verification feature, including :
  • the above-mentioned electronic device may be applied to devices such as mobile phones, monitors, computers, servers, etc., that require license plate number recognition.
  • the electronic device provided in the embodiment of the present invention can implement each process implemented by the license plate number recognition method in the foregoing method embodiment, and can achieve the same beneficial effects. To avoid repetition, details are not described herein again.
  • the embodiment of the present invention also provides a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium.
  • a computer program is executed by a processor, each process of the license plate number recognition method provided by the embodiment of the present invention is implemented, and can To achieve the same technical effect, in order to avoid repetition, I will not repeat them here.
  • the program can be stored in a computer readable storage medium, and the program can be stored in a computer readable storage medium. During execution, it may include the procedures of the above-mentioned method embodiments.
  • the storage medium can be a magnetic disk, an optical disk, a read-only storage memory (Read-Only Memory, ROM) or random access memory (Random Access Memory, RAM for short), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Image Analysis (AREA)
  • Character Discrimination (AREA)

Abstract

一种车牌号码识别方法、装置、电子设备及存储介质，所述方法包括：通过预先训练好的卷积神经网络提取待识别图像的车牌号码特征，所述待识别图像包括车牌号码（101）；在提取所述车牌号码特征的过程中提取中间卷积结果，根据所述中间卷积结果提取第一验证特征和/或第二验证特征（102）；根据所述第一验证特征和/或第二验证特征验证所述车牌号码特征是否正确（103）；若正确，则输出根据所述车牌号码特征预测得到的车牌号码结果（104）。在进行车牌号码特征的特征提取过程，提取中间特征作为验证特征，用来验证提取到的车牌号码特征是否正确，在验证通过的情况下，才进行车牌号码结果的输出，降低了车牌号码识别结果的输出错误率。

Description

车牌号码识别方法、装置、电子设备及存储介质 技术领域
本发明涉及人工智能技术领域，尤其涉及一种人车牌号码识别方法、装置、电子设备及存储介质。
 背景技术
图像识别是当前交通、小区或停车场管理常用的技术之一，例如：使用基于图像识别的车牌号码识别，识别到车辆的车牌号码。目前传统车牌号码识别一般被分为多个独立的步骤，比如：1.图像归一化：把车牌图片通过计算机视觉方法（如单应性矩阵homography等）编程“正式图”。2.图像预处理：在这里对于图像的遮挡，污垢，光照等情况进行处理（如二值分布binarized等）3.字符分割：通过计算机视觉方法进行字符分割（如边缘检测edge detection等）4.字符识别：对分割好的字符进行识别（如随机森林random forest，支持向量机svm，逻辑回归logistic regression等机器学习或深度学习方法）。这样导致了每个步骤中出现的错误可能会累加，从而造成最终识别效果不佳，也不容易定位问题出现在哪一步。而且传统车牌识别对于输入图片的要求相对来说比较高，有严格的角度以及清晰度要求。传统车牌识别的种种限制导致在安装摄像头，监控场景有着严格要求，并且识别率容易受到天气，光照等影响。因此，传统车牌号码识别识别错误就输出错误的识别结果，输出错误率较高，没有验证的机制，识别结果的验证通常需要人工进行验证。
 技术解决方案
本发明实施例提供一种车牌号码识别方法，能够降低车牌号码识别的输出错误率。
第一方面，本发明实施例提供一种车牌号码识别方法，包括：
通过预先训练好的卷积神经网络提取待识别图像的车牌号码特征，所述待识别图像包括车牌号码；
在提取所述车牌号码特征的过程中提取中间卷积结果，根据所述中间卷积结果提取第一验证特征和/或第二验证特征；
根据所述第一验证特征和/或第二验证特征验证所述车牌号码特征是否正确；
若正确，则输出根据所述车牌号码特征预测得到的车牌号码结果。
可选的，所述通过预先训练好的卷积神经网络提取待识别图像的车牌号码特征，包括：
通过空间变换网络将待识别图像进行矫正，得到矫正后的待识别图像；
依次通过第一卷积网络、第二卷积网络对所述矫正后的待识别图像进行特征提取，对应依次得到尺寸相同的第一特征以及第二特征；
基于所述第一特征以及第二特征，得到所述车牌号码特征。
可选的，所述第一卷积网络包括第一卷积层、第二卷积层、第三卷积层以及第一下采样层、第二下采样层、第三下采样层以及第四下采样层，第二卷积网络包括第四卷积层，其中，所述第一下采样层、第二下采样层的输入分别与第一卷积层的输出进行连接，所述依次通过第一卷积网络、第二卷积网络对所述矫正后的待识别图像进行特征提取，对应依次得到尺寸相同的第一特征以及第二特征，包括：
通过第一卷积层对所述矫正后的待识别图像进行特征提取，并将第一卷积结果通过第一下采样层进行下采样，以采样得到符合目标尺寸的第一特征；
通过第四下采样层所述第一卷积结果进行下采样，再将第四下采样结果通过第二卷积层进行特征提取，并将第二卷积结果通过第二下采样层进行下采样，以采样得到符合目标尺寸的第三特征；
通过第三下采样层所述第二卷积结果进行下采样，以采样得到符合目标尺寸的第三下采样结果，再将第三下采样结果通过第三卷积层进行特征提取，基于第三卷积结果得到第四特征；
通过四卷积网络对所述第四特征进行特征提取，基于第四卷积结果得到第二特征；
其中，所述目标尺寸为第二特征的尺寸。
可选的，所述基于所述第一特征以及第二特征，得到所述车牌号码特征，包括：
将所述第一特征、第二特征、第三特征、第四特征在通道维度上进行通道堆叠，得到车牌号码特征。
可选的，所述中间卷积结果包括所述第一卷积结果、第二卷积结果、第三卷积结果以及第四卷积结果中至少一项，所述在提取所述车牌号码特征的过程中提取中间卷积结果，根据所述中间卷积结果提取第一验证特征和/或第二验证特征，包括：
将所述第一卷积结果、第二卷积结果、第三卷积结果中任一项输入第一验证网络，提取得到第一验证特征；和/或
根据所述第四卷积结果，提取得到第二验证特征。
可选的，所述第一验证特征为颜色特征，所述根据所述第一验证特征验证所述车牌号码特征，包括：
对所述第一验证特征进行识别，确定所述第一验证特征的所属颜色类别；
对所述车牌号码特征进行预测，确定所述车牌号码的所属车牌类别；
根据所述车牌号码的所属车牌类别，确定所述车牌号码的所属颜色类别；
判断所述第一验证特征的所属颜色类别与所述车牌号码的所属颜色类别是否相同；
若相同，则验证结果为通过；
若不同，则验证结果为不通过。
可选的，所述待识别图像包括车牌高宽比信息，所述第二验证特征为尺寸特征，所述根据所述第二验证特征验证所述车牌号码特征，包括：
对所述第二验证特征进行识别，确定所述第二验证特征的所对应的尺寸参数，并根据所述尺寸参数计算得到所述第二验证特征的验证高宽比；
判断所述第二验证特征的验证高宽比与所述待识别图像包括车牌高宽比是否相同；
若相同，则验证结果为通过；
若不同，则验证结果为不通过。
第二方面，本发明实施例提供一种车牌号码识别装置，包括：
第一提取模块，用于通过预先训练好的卷积神经网络提取待识别图像的车牌号码特征，所述待识别图像包括车牌号码；
第二提取模块，用于在提取所述车牌号码特征的过程中提取中间卷积结果，根据所述中间卷积结果提取第一验证特征和/或第二验证特征；
验证模块，用于根据所述第一验证特征和/或第二验证特征验证所述车牌号码特征是否正确；
输出模块，用于若正确，则输出根据所述车牌号码特征预测得到的车牌号码结果。
第三方面，本发明实施例提供一种电子设备，包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现本发明实施例提供的车牌号码识别方法中的步骤。
第四方面，本发明实施例提供一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现发明实施例提供的车牌号码识别方法中的步骤。
本发明实施例中，通过预先训练好的卷积神经网络提取待识别图像的车牌号码特征，所述待识别图像包括车牌号码；在提取所述车牌号码特征的过程中，根据中间卷积结果提取第一验证特征和/或第二验证特征；根据所述第一验证特征和/或第二验证特征验证所述车牌号码特征是否正确；若正确，则输出根据所述车牌号码特征预测得到的车牌号码结果。在进行车牌号码特征的特征提取过程，提取中间特征做为验证特征，用来验证提取到的车牌号码特征是否正确，在验证通过的情况下，才进行车牌号码结果的输出，降低了车牌号码识别结果的输出错误率。
 附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例提供的一种车牌号码识别方法的流程图；
图2是本发明实施例提供的另一种车牌号码识别方法的流程图；
图3是本发明实施例提供的另一种车牌号码识别方法的流程图；
图4是本发明实施例提供的一种车牌号码识别装置的结构示意图；
图5是本发明实施例提供的另一种车牌号码识别装置的结构示意图；
图6是本发明实施例提供的另一种车牌号码识别装置的结构示意图；
图7是本发明实施例提供的另一种车牌号码识别装置的结构示意图；
图8是本发明实施例提供的另一种车牌号码识别装置的结构示意图；
图9是本发明实施例提供的另一种车牌号码识别装置的结构示意图；
图10是本发明实施例提供的一种电子设备的结构示意图。
 本发明的实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
请参见图1，图1是本发明实施例提供的一种车牌号码识别方法的流程图，如图1所示，包括以下步骤：
101、通过预先训练好的卷积神经网络提取待识别图像的车牌号码特征。
在该步骤中，待识别图像可以包括车牌号码、车牌参数。上述的待识别图像可以是通过用户上传的车辆车牌的静态图像或动态视频的图像帧，也可以是通过部署在交通道路上、小区出入口、停车场出入口的摄像头获取到的车辆车牌的静态图像或动态视频的图像帧。上述的车牌参数包括车牌的高、宽尺寸参数等。
上述的待识别图像中的车牌号码可以是一个或者多个，即一张待识别图像中有一个或多个待识别的车牌号码。
上述预先训练好的卷积神经网络包换卷积层（conv block）、下采样层（downsample）、全连接层（fully connected layers,FC）等。上述的卷积层中包括输入层（input）、激活层（relu）、批规范化层（batch norm）、卷积计算层（conv）等，在一个卷积计算层中，卷积计算完毕的输出会与该卷积层的输入相加，以使图像信息冗余，提高网络训练的收敛速度，下一卷积计算层可以选择是要卷积后的内容还是没有被卷积后的内容作为输入进行计算。上述的下采样层通过采样像素点，将图像尺寸变小，获取到更大的感受野，下采样层也可以称为池化层（pooling）。上述的全连接层将卷积得到的各个通道进行连接，以得到整体的特征表征。
将待识别图像输入卷积神经网络中，通过卷积神经网络中的各个卷积层依次对待识别图像进行卷积计算以提取到待识别图像中的车牌号码特征。
另外，在将待识别图像输入卷积层之前，可以对图像进行预处理，上述的预处理包括图像归一化，图像大小变换，图像矫正等。
在一种可能的实施例中，上述的卷积神经网络中前置有空间变换网络，比如STN（Spatial Transform Network）空间变换网络，对待识别的图像进行空间变换，以使待识别的图像满足卷积层的输入期望。
102、在提取所述车牌号码特征的过程中提取中间卷积结果，根据所述中间卷积结果提取第一验证特征和/或第二验证特征。
在该步骤中，上述提取车牌号码特征的过程可以理解为在卷积神经网络中对待识别图像的卷积过程，在卷积神经网络中，每一层卷积层对应一个卷积结果，具体来说，每一个卷积计算层对应一个卷积结果。在卷积神经网络中，卷积计算的过程是通过卷积核对输入矩阵进行计算的过程，上述的输入矩阵可以理解为待识别图像的像素点矩阵，或特征图像中的特征点矩阵，特征图像是由待识别图像经过卷积后得到的图像，具体的，通过卷积核对待识别图像进行卷积计算后，会产生与卷积核数量相同的通道，将通道进行相加则得到对应的特征图像，在一种理解方式中，通道也可以称为特征图像。
上述的第一验证特征与第二验证特征为不同的验证特征，上述的验证特征可以是车牌颜色、车牌尺寸、字符颜色等基于车牌本身结构的特征。由于车牌本身结构参数是被规范化的，这些规范化的结构参数可以作为先验信息对车牌号码识别进行验证。
上述的中间卷积结果为卷积神经网络中对待识别图像进行特征提取时，各卷积层的输出留输出的卷积结果。需要说明的是，由于卷积神经网络中有多个卷积层，第一验证特征、第二卷积特征可以是该多个卷积层中的任意一个卷积层的卷积结果。
103、根据第一验证特征和/或第二验证特征验证所述车牌号码特征。
在该步骤中，上述的第一验证特征与第二验证特征为不同的验证特征，上述的验证特征可以是车牌颜色、车牌尺寸、字符颜色等基于车牌本身结构的特征。可以根据上述的车牌颜色、车牌尺寸、字符颜色等特征作为车牌号码验证的先验信息，对车牌号码进行验证。
可选的，上述的第一验证特征为颜色特征，在卷积神经网络对待识别图像进行识别的过程中，提取中间卷积层计算得到的任一项中间卷积结果，作为第一验证特征。可以通过一个支线网络，对提取到的中间卷积结果进行特征识别，以识别到第一验证特征，确定第一验证特征的所属颜色类别；对车牌号码特征进行预测，确定所述车牌号码的所属车牌类别；根据车牌号码的所属车牌类别，确定车牌号码的所属颜色类别；判断所述第一验证特征的所属颜色类别与所述车牌号码的所属颜色类别是否相同；若相同，则验证结果为通过；若不同，则验证结果为不通过。比如，车牌包括车牌标识字符以及车牌颜色，车牌标识字符包括：省、自治区、直辖市简称的汉字字符，发牌机构代号的大写字母标识，序号的大写字母/数字标识；另外，还可以包括车辆所属性质的汉字标识，比如：警用车的序列后会有“警”字符，驾校训练车的序列后会有“学”字符等。上述车牌颜色指的车牌底色，包括：蓝色（一般家用小车车牌）、黄色（大车或农用车用的车牌及教练车的车牌）、白色（特种车车牌，如军车警车车牌及赛车车牌）、黑色（外商及外商的企业所用的车的车牌或外籍车）以国内常用的民用车牌为例，车牌的颜色为蓝色。在卷积神经网络对该车牌图像进行卷积计算时，从中间卷积层选取任意一个卷积结果，输入到颜色识别支线网络中进行识别，以识别到车牌的颜色。在对车牌号码特征进行预测，预测得到对应的车牌号码，从而在车牌号码数据库中查找与该号码对应的车牌颜色，将识别到车牌颜色与查找到的车牌颜色进行比对，从而判断识别到的车牌颜色与查找到的车牌颜色是否相同，若相同，即都为蓝色，则说明颜色的识别不存在错误，可以认为验证通过，若不同，即查找到的车牌颜色不为蓝色，则说明颜色的识别出现错误，由于是在识别的过程中提取到的颜色，可以认为识别过程出现错误，进而认为验证不通过。
可选的，上述待识别图像包括车牌高宽比，上述第二验证特征为尺寸特征，上述根据第二验证特征验证车牌号码特征是否正确可以是：对第二验证特征进行识别，确定第二验证特征的所对应的尺寸参数，并根据尺寸参数计算得到第二验证特征的验证高宽比；判断第二验证特征的验证高宽比与待识别图像包括车牌高宽比是否相同；若相同，则验证结果为通过；若不同，则验证结果为不通过。上述的车牌高宽比可以是预先设置的，在提取到第二验证特征时，将高宽对应的尺寸特征进行计算，得到对应的识别高宽比，将识别高宽比与预先设置的车牌高宽比进行比对，判断是否相同。另外，车牌高宽比也可以是在待识别图像输入后，根据车牌在待识别图像中的像素坐标进行计算得到，再与识别高宽比进行比对，判断是否相同。
需要说明的是，上述的第一验证特征与第二验证特征可单独对车牌号码特征进行验证，也可以结合起来对车牌号码特征进行验证。
在一种可能的实施例方式中，可以通过第一验证特征结合第二验证特征对车牌号码特征进行验证，其中，第一验证特征可以是车牌底色，第二验证特征可以是字符颜色。在车牌颜色系统中，车牌的底色有蓝色、黄色、白色、黑色。其中：蓝色是小车车牌（包括小吨位的货车）；黄色是大车或农用车用的车牌及教练车车牌，还有新产品为定型的试验车，摩托车等；白色是特种车车牌（如军车警车车牌及赛车车牌）；黑色是外商及外商的企业由国外自带车的车牌。车牌的底色和字符颜色有：大型民用汽车：黄底黑字；小型民用汽车：蓝底白字；武警专用汽车：白底红“WJ”、黑字；其它外籍汽车：黑底白字；使、领馆外籍汽车：黑底白字及空心“使”字标志；试车牌照：白底红字，数字前有“试”字标志；临时牌照：白底红字，数字前有“临时”二字；汽车补用牌照：白底黑字。因此，可以结合车牌底色和字符颜色对车牌号码特征进行验证。通过对第一验证特征进行识别，得到识别车牌颜色，通过对第二验证特征进行识别，得到识别字符颜色，对车牌号码特征进行预测，得到车牌号码，根据车牌号码在车牌号码数据库中查找与该车牌号码对应的车牌颜色及字符颜色，并与识别车牌颜色及识别字符颜色进行比对，判断比对结果，从而判断验证是否通过。
需要说明的是，在训练卷积神经网络时，需要加入对第一验证特征和/或第二验证特征对应的训练，以使卷积神经网络学习到第一验证特征和/或第二验证特征的提取，存在足够的卷积核对第一验证特征和/或第二验证特征进行提取。
104、若验证通过，则输出根据车牌号码特征预测得到的车牌号码结果。
在该步骤中，只有验证通过的车牌号码特征对应的车牌号码结果才会输出显示。这样，可以降低车牌号码识别结果的输出错误率。
在一种可能的实施例中，若验证不通过，则可以输出车牌号码结果以及对应的异常提示，异常提示包括验证记录，比如，提示可以是：“识别结果颜色不匹配，请注意”。
在另一种可能的实施例中，可以部署多个并行的卷积神经网络分别对多个待识别图像进行识别，其中，多个并行的卷积神经网络为通过同一数据集分别训练得到，在验证不通过的情况下，将待识别图像转入并行的另一个卷积神经网络进行二次识别，若两次识别得到的车牌号码结果相同，则进行输出；若两次识别得到的车牌号码不同，则输出第一验证特征和/或第二验证特征验证通过的车牌号码结果；若两次识别得到的车牌号码不同，且第一验证特征和/或第二验证特征验证不通过，则输出两次车牌号码结果以及对应的异常提示，提示包括两次识别过程中的验证记录。
本发明实施例中，通过预先训练好的卷积神经网络提取待识别图像的车牌号码特征，所述待识别图像包括车牌号码；在提取所述车牌号码特征的过程中，根据中间卷积结果提取第一验证特征和/或第二验证特征；根据所述第一验证特征和/或第二验证特征验证所述车牌号码特征是否正确；若正确，则输出根据所述车牌号码特征预测得到的车牌号码结果。在进行车牌号码特征的特征提取过程，提取中间特征做为验证特征，用来验证提取到的车牌号码特征是否正确，在验证通过的情况下，才进行车牌号码结果的输出，降低了车牌号码识别结果的输出错误率。
需要说明的是，本发明实施例提供的车牌号码识别方法可以应用于需要对车牌号码进行识别的手机、监控器、计算机、服务器等设备。
可选的，请参见图2，图2是本发明实施例提供的另一种车牌号码识别方法的流程图，与图1实施例不同的是，预先训练好的卷积神经网络包括第一卷积网络及第二卷积网络，如图2所示，包括以下步骤：
201、通过空间变换网络将待识别图像进行矫正，得到矫正后的待识别图像。
在该步骤中，上述的预先训练好的空间变换网络可以是STN（Spatial Transform Network）空间变换网络。上述的空间变换网络可以设置在第一卷积网络以及第二卷积网络之前对待识别图像进行矫正，以使待识别图像被矫正为卷积层所期望的输入。同时，也可以在第一卷积网络与第二卷积网络之间设置空间变换网络，对下一卷积网络的输入进行矫正，以使上一卷积网络的输出被矫正为下一卷积网络期望的输入。另外，第一卷积网络与第二卷积网络中包括多个卷积层，也可以在卷积层之间设置空间变换网络，对下一卷积层的输入进行矫正，以使上一卷积层的输出被矫正为下一卷积层期望的输入，上述的矫正可以理解为将待识别图像进行空间变换和对齐，可以包括对待识别图像的平移、缩放、旋转等。这样，可以降低对图像采集设备的部署要求。
202、依次通过第一卷积网络、第二卷积网络对矫正后的待识别图像进行特征提取，对应依次得到尺寸相同的第一特征以及第二特征。
在该步骤中，上述的第一特征以及第二特征可以理解为经过卷积计算的特征图像，上述的尺寸相同指的是特征图像的尺寸。上述的特征图像的尺寸相同，便于对各个特征图像进行相加。
上述的特征提取是根据训练好的卷积核对待识别图像进行卷积计算，以提取到符合期望的特征信息。根据第一卷积网络的卷积结果，必要时进行下采样，得到第一特征，根据第二卷积网络的卷积结果，得到第二特征。
在第一卷积网络所处的深度小于第二卷积网络所处的深度时，第一卷积网络提取到的特征图像的尺寸可能会大于第二卷积网络提取到特征图像的尺寸，此时，可以对第一卷积网络提取到的特征图像进行下采样，得到第一特征对应的特征图像，将第一卷积网络提取到的特征图像下采样到与第二卷积提取到的特征图像的尺寸相同。
203、基于第一特征以及第二特征，得到车牌号码特征。
在该步骤中，可以将第一特征与第二特征进行相加，得到车牌号码特征。第一特征与第二特征进行相加可以理解为是对应特征图进行相加，即特征图像上对应的特征点进行相加。
将第一特征、第二特征在通道维度上进行相加，得到车牌号码特征。
进一步的，第一卷积网络包括第一卷积层、第二卷积层、第三卷积层以及第一下采样层、第二下采样层、第三下采样层以及第四下采样层，第二卷积网络包括第四卷积层，其中，第一下采样层、第二下采样层的输入分别与第一卷积区域的输出进行连接。具体的，通过第一卷积层对矫正后的待识别图像进行特征提取，并将第一卷积结果通过第一下采样层进行下采样，以采样得到符合目标尺寸的第一特征；通过第四下采样层第一卷积结果进行下采样，再将第四下采样结果通过第二卷积层进行特征提取，并将第二卷积结果通过第二下采样层进行下采样，以采样得到符合目标尺寸的第三特征；通过第三下采样层第二卷积结果进行下采样，以采样得到符合目标尺寸的第三下采样结果，再将第三下采样结果通过第三卷积层进行特征提取，基于第三卷积结果得到第四特征；通过四卷积网络对所述第四特征进行特征提取，基于第四卷积结果得到第二特征；其中，所述目标尺寸为第二特征的尺寸。
将第一特征、第二特征、第三特征、第四特征在通道维度上进行相加，得到车牌号码特征。
204、将第一卷积网络的卷积结果输入第一验证网络，提取得到第一验证特征。
在该步骤中，第一验证网络可以理解为是卷积网络的支线网络，具体的，第一验证网络与第二卷积网络并行，第一卷积网络的卷积结果同时输入到第二卷积网络与第一验证网络。
进一步的，第一卷积网络包括第一卷积层、第二卷积层、第三卷积层以及第一下采样层、第二下采样层、第三下采样层以及第四下采样层，第二卷积网络包括第四卷积层，其中，第一下采样层、第二下采样层的输入分别与第一卷积区域的输出进行连接。通过第一卷积层、第二卷积层、第三卷积层依次对矫正后的待识别图像进行特征提取，分别得到第一卷积结果、第二卷积结果、第三卷积结果，将第一卷积结果、第二卷积结果、第三卷积结果中任一项输入第一验证网络，提取得到第一验证特征。
需要说明的是，上述第一卷积网络、第二卷积网络、第一验证网络同属于一个卷积神经网络，并可以通过同一个数据集进行训练。
可选的，上述的卷积神经网络通过CTC loss（Connectionist temporal classification loss，连接时序分类损失）进行训练，具体的，第一卷积网络、第二卷积网络通过CTC loss进行训练，第一验证网络通过cross entropy（交叉熵）进行训练。这样，可以使第一卷积网络、第二卷积网络学习到时序分类以根据学习到的时序对车牌号码进行识别，使第一验证网络学习到对验证特征的分类。
205、根据第二卷积网络的卷积结果，提取得到第二验证特征。
进一步的，在步骤204的基础上，根据第四卷积结果，提取得到第二验证特征。
在一种可能的实施例中，将第二卷积网络的卷积结果输入第二验证网络，提取得到第二验证特征。其中，第二验证网络可以理解为是卷积网络的支线网络，具体的，第二验证网络与第二卷积网络并行，第一卷积网络的卷积结果同时输入到第二卷积网络与第二验证网络。同样的，上述第一卷积网络、第二卷积网络、第一验证网络同属于一个卷积神经网络，并可以通过同一个数据集进行训练。
206、根据所述第一验证特征和/或第二验证特征验证所述车牌号码特征。
此步骤与图1实施例中步骤103相似，在此不再赘述。
207、若验证通过，则输出根据所述车牌号码特征预测得到的车牌号码结果。
在该步骤中，只有验证通过的车牌号码特征对应的车牌号码结果才会输出显示。这样，可以降低车牌号码识别结果的输出错误率。
在本发明实施例中，通过卷积神经网络中不同网络深度得到的卷积结果提取验证特征，可以使验证特征和车牌号码特征在一个卷积神经中的卷积结果中被提取到，也只需要同一个数据集对卷积神经网络进行训练，降低卷积神经网络训练的难度和部署所需要的资源。
可选的，请参见图3，图3是本发明实施例提供的另一种车牌号码识别方法的流程图，如图3所示，该方法包括以下：
输入待识别图像。其中，上述的待识别图像会被处理为（198，48，3）的RGB图像。
在空间变换网络层将待识别图像进行矫正。被矫正后的待识别图像会输入到卷积层通过3×3卷积核进行卷积，得到的卷积结果进行下采样，得到（96，48，64）特征图像；对该（96，48，64）特征图像进行卷积计算，得到对应的卷积结果，对该卷积结果进行下采样后，得到（24，24，64）的特征图像作为相加特征，该下采样可以是平均池化，即取池化核所提取到特征图像中的特征值的平均值为新的特征值，比如，一个2×2池化核提取到4个特征点，对应的特征值为（12，13，15，16），则经过平均池化后，4个特征点映射为一个特征点，该特征点的值为14；另外，还会对该卷积结果进行一次下采样，得到（48，48，128）的特征图像作为下一卷积层的输入；依次通过两个卷积层对该（48，48，128）的特征图像进行两次卷积计算，得到对应的卷积结果，对该卷积结果进行平均池化，得到（24，24，128）的特征图像作为相加特征，以及对该卷积结果进行下采样，得到（24，24，256）的特征图像作为下一卷积层的输入；依次通过两个卷积层对该（24，24，256）的特征图像进行两次卷积计算，得到对应的卷积结果，该卷积结果为（24，24，256）的特征图像，该（24，24，256）的特征图像作为相加特征的同时，还作为下一卷积层的输入；将该（24，24，256）的特征图像依次输入到横向卷积层、纵向卷积层中进行卷积计算，得到（24，24，256）的特征图像作为相加特征。其中，上述的横向卷积层、纵向卷积层可以提取到车牌的尺寸参数，该尺寸参数可以作为验证特征验证车牌号码特征。其中，横向卷积层中的卷积核为11×1，纵向卷积层中的卷积核为1×3，横向卷积层与纵向卷积层的卷积核形状与车牌形状相关。
将上述得到的相加特征（24，24，64）、（24，24，128）、（24，24，256）、（24，24，256）进行通道维度的堆叠，得到（24，24，704）的车牌号码特征，该车牌号码特征的通道维度为704。将该号码特征输入卷积层通过3×3卷积核进行卷积计算，得到（24，24，16）的车牌号码特征，该车牌号码特征的通道维度为16。通过全连接层将车牌号码特征进行连接，得到一个特征向量，全连接层则起到将学到的“分布式特征表示”映射到样本标记空间的作用，即将一个16维的向量映射到一个70维的样本标记空间中进行预测。最后通过输出层输出一个（10，1）的车牌号码结果，其中的10中包括字符、起止符及对齐符，1为结果个数。
在得到（24，24，256）的特征图像后，将该（24，24，256）的特征图像作为支线网络的输入，输入到该支线网络中进行验证特征的提取，该支线网络可以是颜色分类网络，对该（24，24，256）的特征图像进行颜色分类。该支线网络输出一类颜色分类结果，作为验证特征所属颜色类别。该验证特征所属颜色类别用于验证车牌号码特征。具体的，根据车牌号码特征预测得到对应的车牌号码，在车牌号码数据库中查找对应车牌号码的车牌颜色，作为车牌号码的所属颜色类别，从而可以通过判断验证特征所属颜色类别与车牌号码的所属颜色类别是否相同，来判断车牌号码特征是否通过验证。
在本发明实施例中，通过卷积神经网络中不同网络深度得到的卷积结果提取验证特征，可以使验证特征和车牌号码特征在一个卷积神经中的卷积结果中被提取到，也只需要同一个数据集对卷积神经网络进行训练，降低卷积神经网络训练的难度和部署所需要的资源。另外，在卷积过程中，通过多次下采样操作，使卷积神经网络的感受野更大，可以提高神经网络的鲁棒性。
需要说明的是，本发明实施例提供的车牌号码识别方法可以应用于需要进行车牌号码识别的手机、监控器、计算机、服务器等设备。
请参见图4，图4是本发明实施例提供的一种车牌号码识别装置的结构示意图，如图4所示，所述装置包括：
第一提取模块401，用于通过预先训练好的卷积神经网络提取待识别图像的车牌号码特征，所述待识别图像包括车牌号码；
第二提取模块402，用于在提取所述车牌号码特征的过程中提取中间卷积结果，根据所述中间卷积结果提取第一验证特征和/或第二验证特征；
验证模块403，用于根据所述第一验证特征和/或第二验证特征验证所述车牌号码特征是否正确；
输出模块404，用于若正确，则输出根据所述车牌号码特征预测得到的车牌号码结果。
可选的，如图5所示，所述预先训练好的卷积神经网络包括第一卷积网络及第二卷积网络，所述第一提取模块401，包括：
预处理单元4011，用于通过空间变换网络将待识别图像进行矫正，得到矫正后的待识别图像；
第一特征提取单元4012，用于依次通过第一卷积网络、第二卷积网络对所述矫正后的待识别图像进行特征提取，对应依次得到尺寸相同的第一特征以及第二特征；
处理单元4013，用于基于所述第一特征以及第二特征，得到所述车牌号码特征。
可选的，如图6所示，所述第一卷积网络包括第一卷积层、第二卷积层、第三卷积层以及第一下采样层、第二下采样层、第三下采样层以及第四下采样层，第二卷积网络包括第四卷积层，其中，所述第一下采样层、第二下采样层的输入分别与第一卷积区域的输出进行连接，所述第一特征提取单元4012，包括：
第一计算子单元40121，用于通过第一卷积层对所述矫正后的待识别图像进行特征提取，并将第一卷积结果通过第一下采样层进行下采样，以采样得到符合目标尺寸的第一特征；
第二计算子单元40122，用于通过第四下采样层所述第一卷积结果进行下采样，再将第四下采样结果通过第二卷积层进行特征提取，并将第二卷积结果通过第二下采样层进行下采样，以采样得到符合目标尺寸的第三特征；
第三计算子单元40123，用于通过第三下采样层所述第二卷积结果进行下采样，以采样得到符合目标尺寸的第三下采样结果，再将第三下采样结果通过第三卷积层进行特征提取，基于第三卷积结果得到第四特征；
第四计算子单元40124，用于通过四卷积网络对所述第四特征进行特征提取，基于第四卷积结果得到第二特征；
其中，所述目标尺寸为第二特征的尺寸。
可选的，如图5所示，所述处理单元4013还用于将所述第一特征、第二特征、第三特征、第四特征在通道维度上进行堆叠，得到车牌号码特征。
可选的，如图7所示，所述中间卷积结果包括所述第一卷积结果、第二卷积结果、第三卷积结果以及第四卷积结果中至少一项，所述第二提取模块402，包括：
第二特征提取单元4021，用于将所述第一卷积结果、第二卷积结果、第三卷积结果中任一项输入第一验证网络，提取得到第一验证特征；和/或
第三特征提取单元4022，用于根据所述第四卷积结果，提取得到第二验证特征。
可选的，如图8所示，所述第一验证特征为颜色特征，所述验证模块403，包括：
第一确定单元4031，用于对所述第一验证特征进行识别，确定所述第一验证特征的所属颜色类别；
第二确定单元4032，用于对所述车牌号码特征进行预测，确定所述车牌号码的所属车牌类别；
第三确定单元4033，用于根据所述车牌号码的所属车牌类别，确定所述车牌号码的所属颜色类别；
第一判断单元4034，用于判断所述第一验证特征的所属颜色类别与所述车牌号码的所属颜色类别是否相同；若相同，则验证结果为通过；若不同，则验证结果为不通过。
可选的，如图9所示，所述待识别图像包括车牌高宽比信息，所述第二验证特征为尺寸特征，所述验证模块403，包括：
第四确定单元4035，用于对所述第二验证特征进行识别，确定所述第二验证特征的所对应的尺寸参数，并根据所述尺寸参数计算得到所述第二验证特征的验证高宽比；
第二判断单元4036，用于判断所述第二验证特征的验证高宽比与所述待识别图像包括车牌高宽比是否相同；若相同，则验证结果为通过；若不同，则验证结果为不通过。
需要说明的是，本发明实施例提供的车牌号码识别装置可以应用于需要进行车牌号码识别的手机、监控器、计算机、服务器等设备。
本发明实施例提供的车牌号码识别装置能够实现上述方法实施例中车牌号码识别方法实现的各个过程，且可以达到相同的有益效果。为避免重复，这里不再赘述。
参见图10，图10是本发明实施例提供的一种电子设备的结构示意图，如图10所示，包括：存储器1002、处理器1001及存储在所述存储器1002上并可在所述处理器1001上运行的计算机程序，其中：
处理器1001用于调用存储器1002存储的计算机程序，执行如下步骤：
通过预先训练好的卷积神经网络提取待识别图像的车牌号码特征，所述待识别图像包括车牌号码；
在提取所述车牌号码特征的过程中提取中间卷积结果，根据所述中间卷积结果提取第一验证特征和/或第二验证特征；
根据所述第一验证特征和/或第二验证特征验证所述车牌号码特征；
若验证通过，则输出根据所述车牌号码特征预测得到的车牌号码结果。
可选的，所述预先训练好的卷积神经网络包括第一卷积网络及第二卷积网络，所述处理器1001执行的通过预先训练好的卷积神经网络提取待识别图像的车牌号码特征，包括：
通过空间变换网络将待识别图像进行矫正，得到矫正后的待识别图像；
依次通过第一卷积网络、第二卷积网络对所述矫正后的待识别图像进行特征提取，对应依次得到尺寸相同的第一特征以及第二特征；
基于所述第一特征以及第二特征，得到所述车牌号码特征。
可选的，所述第一卷积网络包括第一卷积层、第二卷积层、第三卷积层以及第一下采样层、第二下采样层、第三下采样层以及第四下采样层，第二卷积网络包括第四卷积层，其中，所述第一下采样层、第二下采样层的输入分别与第一卷积区域的输出进行连接，所述处理器1001执行的依次通过第一卷积网络、第二卷积网络对所述矫正后的待识别图像进行特征提取，对应依次得到尺寸相同的第一特征以及第二特征，包括：
通过第一卷积层对所述矫正后的待识别图像进行特征提取，并将第一卷积结果通过第一下采样层进行下采样，以采样得到符合目标尺寸的第一特征；
通过第四下采样层所述第一卷积结果进行下采样，再将第四下采样结果通过第二卷积层进行特征提取，并将第二卷积结果通过第二下采样层进行下采样，以采样得到符合目标尺寸的第三特征；
通过第三下采样层所述第二卷积结果进行下采样，以采样得到符合目标尺寸的第三下采样结果，再将第三下采样结果通过第三卷积层进行特征提取，基于第三卷积结果得到第四特征；
通过四卷积网络对所述第四特征进行特征提取，基于第四卷积结果得到第二特征；
其中，所述目标尺寸为第二特征的尺寸。
可选的，所处理器1001执行的述基于所述第一特征以及第二特征，得到所述车牌号码特征，包括：
将所述第一特征、第二特征、第三特征、第四特征在通道维度上进行堆叠，得到车牌号码特征。
可选的，所述中间卷积结果包括所述第一卷积结果、第二卷积结果、第三卷积结果以及第四卷积结果中至少一项，所述处理器1001执行的在提取所述车牌号码特征的过程中提取中间卷积结果，根据所述中间卷积结果提取第一验证特征和/或第二验证特征，包括：
将所述第一卷积结果、第二卷积结果、第三卷积结果中任一项输入第一验证网络，提取得到第一验证特征；和/或
根据所述第四卷积结果，提取得到第二验证特征。
可选的，所述第一验证特征为颜色特征，所述处理器1001执行的根据所述第一验证特征验证所述车牌号码特征，包括：
对所述第一验证特征进行识别，确定所述第一验证特征的所属颜色类别；
对所述车牌号码特征进行预测，确定所述车牌号码的所属车牌类别；
根据所述车牌号码的所属车牌类别，确定所述车牌号码的所属颜色类别；
判断所述第一验证特征的所属颜色类别与所述车牌号码的所属颜色类别是否相同；
若相同，则验证结果为通过；
若不同，则验证结果为不通过。
可选的，所述待识别图像包括车牌高宽比信息，所述第二验证特征为尺寸特征，所述处理器1001执行的根据所述第二验证特征验证所述车牌号码特征是否正确，包括：
对所述第二验证特征进行识别，确定所述第二验证特征的所对应的尺寸参数，并根据所述尺寸参数计算得到所述第二验证特征的验证高宽比；
判断所述第二验证特征的验证高宽比与所述待识别图像包括车牌高宽比是否相同；
若相同，则验证结果为通过；
若不同，则验证结果为不通过。
需要说明的是，上述电子设备可以是可以应用于需要进行车牌号码识别的手机、监控器、计算机、服务器等设备。
本发明实施例提供的电子设备能够实现上述方法实施例中车牌号码识别方法实现的各个过程，且可以达到相同的有益效果，为避免重复，这里不再赘述。
本发明实施例还提供一种计算机可读存储介质，计算机可读存储介质上存储有计算机程序，该计算机程序被处理器执行时实现本发明实施例提供的车牌号码识别方法的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体（Read-Only Memory，ROM）或随机存取存储器（Random Access Memory，简称RAM）等。

Claims (10)

  1. 一种车牌号码识别方法，其特征在于，包括以下步骤：
    通过预先训练好的卷积神经网络提取待识别图像的车牌号码特征，所述待识别图像包括车牌号码；
    在提取所述车牌号码特征的过程中提取中间卷积结果，根据所述中间卷积结果提取第一验证特征和/或第二验证特征；
    根据所述第一验证特征和/或第二验证特征验证所述车牌号码特征；
    若验证通过，则输出根据所述车牌号码特征预测得到的车牌号码结果。
  2. 如权利要求1所述的方法，其特征在于，所述预先训练好的卷积神经网络包括第一卷积网络及第二卷积网络，所述通过预先训练好的卷积神经网络提取待识别图像的车牌号码特征，包括：
    通过空间变换网络将待识别图像进行矫正，得到矫正后的待识别图像；
    依次通过第一卷积网络、第二卷积网络对所述矫正后的待识别图像进行特征提取，对应依次得到尺寸相同的第一特征以及第二特征；
    基于所述第一特征以及第二特征，得到所述车牌号码特征。
  3. 如权利要求2所述的方法，其特征在于，所述第一卷积网络包括第一卷积层、第二卷积层、第三卷积层以及第一下采样层、第二下采样层、第三下采样层以及第四下采样层，第二卷积网络包括第四卷积层，其中，所述第一下采样层、第二下采样层的输入分别与第一卷积区域的输出进行连接，所述依次通过第一卷积网络、第二卷积网络对所述矫正后的待识别图像进行特征提取，对应依次得到尺寸相同的第一特征以及第二特征，包括：
    通过第一卷积层对所述矫正后的待识别图像进行特征提取，并将第一卷积结果通过第一下采样层进行下采样，以采样得到符合目标尺寸的第一特征；
    通过第四下采样层所述第一卷积结果进行下采样，再将第四下采样结果通过第二卷积层进行特征提取，并将第二卷积结果通过第二下采样层进行下采样，以采样得到符合目标尺寸的第三特征；
    通过第三下采样层所述第二卷积结果进行下采样，以采样得到符合目标尺寸的第三下采样结果，再将第三下采样结果通过第三卷积层进行特征提取，基于第三卷积结果得到第四特征；
    通过四卷积网络对所述第四特征进行特征提取，基于第四卷积结果得到第二特征；
    其中，所述目标尺寸为第二特征的尺寸。
  4. 如权利要求3所述的方法，其特征在于，所述基于所述第一特征以及第二特征，得到所述车牌号码特征，包括：
    将所述第一特征、第二特征、第三特征、第四特征在通道维度上进行堆叠，得到车牌号码特征。
  5. 如权利要求3所述的方法，其特征在于，所述中间卷积结果包括所述第一卷积结果、第二卷积结果、第三卷积结果以及第四卷积结果中至少一项，所述在提取所述车牌号码特征的过程中提取中间卷积结果，根据所述中间卷积结果提取第一验证特征和/或第二验证特征，包括：
    将所述第一卷积结果、第二卷积结果、第三卷积结果中任一项输入第一验证网络，提取得到第一验证特征；和/或
    根据所述第四卷积结果，提取得到第二验证特征。
  6. 如权利要求1所述的方法，其特征在于，所述第一验证特征为颜色特征，所述根据所述第一验证特征验证所述车牌号码特征，包括：
    对所述第一验证特征进行识别，确定所述第一验证特征的所属颜色类别；
    对所述车牌号码特征进行预测，确定所述车牌号码的所属车牌类别；
    根据所述车牌号码的所属车牌类别，确定所述车牌号码的所属颜色类别；
    判断所述第一验证特征的所属颜色类别与所述车牌号码的所属颜色类别是否相同；
    若相同，则验证结果为通过；
    若不同，则验证结果为不通过。
  7. 如权利要求1所述的方法，其特征在于，所述待识别图像包括车牌高宽比信息，所述第二验证特征为尺寸特征，所述根据所述第二验证特征验证所述车牌号码特征是否正确，包括：
    对所述第二验证特征进行识别，确定所述第二验证特征的所对应的尺寸参数，并根据所述尺寸参数计算得到所述第二验证特征的验证高宽比；
    判断所述第二验证特征的验证高宽比与所述待识别图像包括车牌高宽比是否相同；
    若相同，则验证结果为通过；
    若不同，则验证结果为不通过。
  8. 一种车牌号码识别装置，其特征在于，所述装置包括：
    第一提取模块，用于通过预先训练好的卷积神经网络提取待识别图像的车牌号码特征，所述待识别图像包括车牌号码；
    第二提取模块，用于在提取所述车牌号码特征的过程中提取中间卷积结果，根据所述中间卷积结果提取第一验证特征和/或第二验证特征；
    验证模块，用于根据所述第一验证特征和/或第二验证特征验证所述车牌号码特征是否正确；
    输出模块，用于若正确，则输出根据所述车牌号码特征预测得到的车牌号码结果。
  9. 一种电子设备，其特征在于，包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如权利要求1至7中任一项所述的车牌号码识别方法中的步骤。
  10. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1至7中任一项所述的车牌号码识别方法中的步骤。
PCT/CN2020/124085 2019-11-29 2020-10-27 车牌号码识别方法、装置、电子设备及存储介质 WO2021103897A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/623,903 US11645857B2 (en) 2019-11-29 2020-10-27 License plate number recognition method and device, electronic device and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911207033.2 2019-11-29
CN201911207033.2A CN111062396B (zh) 2019-11-29 2019-11-29 车牌号码识别方法、装置、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2021103897A1 true WO2021103897A1 (zh) 2021-06-03

Family

ID=70299256

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/124085 WO2021103897A1 (zh) 2019-11-29 2020-10-27 车牌号码识别方法、装置、电子设备及存储介质

Country Status (3)

Country Link
US (1) US11645857B2 (zh)
CN (1) CN111062396B (zh)
WO (1) WO2021103897A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113326843A (zh) * 2021-06-17 2021-08-31 讯飞智元信息科技有限公司 车牌识别方法、装置、电子设备和可读存储介质
CN114299487A (zh) * 2021-12-31 2022-04-08 郑州信大先进技术研究院 基于语义分割的车牌检测及矫正方法
CN114882490A (zh) * 2022-07-08 2022-08-09 松立控股集团股份有限公司 一种基于点引导定位的无受限场景车牌检测分类方法
CN116453105A (zh) * 2023-06-20 2023-07-18 青岛国实科技集团有限公司 基于知识蒸馏深度神经网络的船牌号识别方法及系统

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111062396B (zh) * 2019-11-29 2022-03-25 深圳云天励飞技术有限公司 车牌号码识别方法、装置、电子设备及存储介质
JP2022546885A (ja) * 2020-08-01 2022-11-10 商▲湯▼国▲際▼私人有限公司 目標対象認識方法及び装置
CN112215229B (zh) * 2020-08-27 2023-07-18 北京英泰智科技股份有限公司 基于轻量网络端到端的车牌识别方法及装置
CN112001386B (zh) * 2020-08-31 2021-06-15 重庆紫光华山智安科技有限公司 一种基于车牌字符识别方法、系统、介质及终端
CN112507801A (zh) * 2020-11-14 2021-03-16 武汉中海庭数据技术有限公司 车道路面数字颜色识别方法、限速信息识别方法及系统
CN112418168B (zh) * 2020-12-10 2024-04-02 深圳云天励飞技术股份有限公司 车辆识别方法、装置、系统、电子设备及存储介质
CN114639090A (zh) * 2021-10-28 2022-06-17 中国公路工程咨询集团有限公司 一种非可控环境下的鲁棒中文车牌识别方法
TWI827996B (zh) * 2021-11-02 2024-01-01 英屬開曼群島商睿能創意公司 影像擷取方法
US12096091B2 (en) * 2021-11-12 2024-09-17 William Frederick Vartorella Facial recognition software (FRS) interactive images placed on a moving race vehicle

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109350032A (zh) * 2018-10-16 2019-02-19 武汉中旗生物医疗电子有限公司 一种分类方法、系统、电子设备及存储介质
CN109508715A (zh) * 2018-10-30 2019-03-22 南昌大学 一种基于深度学习的车牌定位和识别方法
CN109784186A (zh) * 2018-12-18 2019-05-21 深圳云天励飞技术有限公司 一种行人重识别方法、装置、电子设备及计算机可读存储介质
WO2019141741A1 (en) * 2018-01-19 2019-07-25 Arcus Holding A/S License plate reader using optical character recognition on plural detected regions
US20190251369A1 (en) * 2018-02-11 2019-08-15 Ilya Popov License plate detection and recognition system
CN110399880A (zh) * 2019-07-31 2019-11-01 深圳市捷顺科技实业股份有限公司 一种车牌字符及车牌类别的识别方法、装置及设备
CN111062396A (zh) * 2019-11-29 2020-04-24 深圳云天励飞技术有限公司 车牌号码识别方法、装置、电子设备及存储介质

Family Cites Families (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9229988B2 (en) * 2013-01-18 2016-01-05 Microsoft Technology Licensing, Llc Ranking relevant attributes of entity in structured knowledge base
CN104361343B (zh) * 2014-10-30 2018-08-31 中盟科技有限公司 车辆类型识别方法及其装置
US10275684B2 (en) * 2015-11-04 2019-04-30 Samsung Electronics Co., Ltd. Authentication method and apparatus, and method and apparatus for training a recognizer
US10984289B2 (en) * 2016-12-23 2021-04-20 Shenzhen Institute Of Advanced Technology License plate recognition method, device thereof, and user equipment
CN107688819A (zh) * 2017-02-16 2018-02-13 平安科技（深圳）有限公司 车辆的识别方法及装置
CN109145927A (zh) * 2017-06-16 2019-01-04 杭州海康威视数字技术股份有限公司 一种对形变图像的目标识别方法及装置
US10902252B2 (en) * 2017-07-17 2021-01-26 Open Text Corporation Systems and methods for image based content capture and extraction utilizing deep learning neural network and bounding box detection training techniques
CN108229646A (zh) * 2017-08-08 2018-06-29 北京市商汤科技开发有限公司 神经网络模型压缩方法、装置、存储介质和电子设备
US11755889B2 (en) * 2017-10-10 2023-09-12 Nokia Technologies Oy Method, system and apparatus for pattern recognition
CN108154160B (zh) * 2017-12-27 2020-02-21 苏州科达科技股份有限公司 车牌颜色识别方法及系统
US10795933B1 (en) * 2018-05-01 2020-10-06 Flock Group Inc. System and method for object based query of video content captured by a dynamic surveillance network
EP3579138B1 (en) * 2018-06-04 2020-11-11 CogVis Software und Consulting GmbH Method for determining a type and a state of an object of interest
LT3599572T (lt) * 2018-07-27 2021-12-27 JENOPTIK Traffic Solutions UK Ltd Transporto priemonės valstybinių numerių atpažinimo būdas ir prietaisas
US10740643B2 (en) * 2018-08-28 2020-08-11 Sony Corporation Automatic license plate recognition based on augmented datasets
US10878071B2 (en) * 2018-10-23 2020-12-29 International Business Machines Corooration Biometric authentication anomaly detection
US11017542B2 (en) * 2018-12-21 2021-05-25 Beijing Voyager Technology Co., Ld. Systems and methods for determining depth information in two-dimensional images
US10963719B1 (en) * 2019-03-04 2021-03-30 Ccc Information Services Inc. Optimized vehicle license plate recognition
KR102420039B1 (ko) * 2019-05-16 2022-07-13 삼성전자주식회사 전자 장치 및 이의 제어 방법
CN110414451B (zh) * 2019-07-31 2023-11-10 深圳市捷顺科技实业股份有限公司 一种基于端对端的车牌识别方法、装置、设备及存储介质
US11076088B2 (en) * 2019-09-24 2021-07-27 Sony Corporation Artificial intelligence (AI)-based control of imaging parameters of image-capture apparatus
US11043003B2 (en) * 2019-11-18 2021-06-22 Waymo Llc Interacted object detection neural network
CN112906429A (zh) * 2019-11-19 2021-06-04 英业达科技有限公司 图文影像辨识系统及方法
KR20210061839A (ko) * 2019-11-20 2021-05-28 삼성전자주식회사 전자 장치 및 그 제어 방법
US20210158540A1 (en) * 2019-11-21 2021-05-27 Sony Corporation Neural network based identification of moving object

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019141741A1 (en) * 2018-01-19 2019-07-25 Arcus Holding A/S License plate reader using optical character recognition on plural detected regions
US20190251369A1 (en) * 2018-02-11 2019-08-15 Ilya Popov License plate detection and recognition system
CN109350032A (zh) * 2018-10-16 2019-02-19 武汉中旗生物医疗电子有限公司 一种分类方法、系统、电子设备及存储介质
CN109508715A (zh) * 2018-10-30 2019-03-22 南昌大学 一种基于深度学习的车牌定位和识别方法
CN109784186A (zh) * 2018-12-18 2019-05-21 深圳云天励飞技术有限公司 一种行人重识别方法、装置、电子设备及计算机可读存储介质
CN110399880A (zh) * 2019-07-31 2019-11-01 深圳市捷顺科技实业股份有限公司 一种车牌字符及车牌类别的识别方法、装置及设备
CN111062396A (zh) * 2019-11-29 2020-04-24 深圳云天励飞技术有限公司 车牌号码识别方法、装置、电子设备及存储介质

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113326843A (zh) * 2021-06-17 2021-08-31 讯飞智元信息科技有限公司 车牌识别方法、装置、电子设备和可读存储介质
CN114299487A (zh) * 2021-12-31 2022-04-08 郑州信大先进技术研究院 基于语义分割的车牌检测及矫正方法
CN114882490A (zh) * 2022-07-08 2022-08-09 松立控股集团股份有限公司 一种基于点引导定位的无受限场景车牌检测分类方法
CN114882490B (zh) * 2022-07-08 2022-09-20 松立控股集团股份有限公司 一种基于点引导定位的无受限场景车牌检测分类方法
CN116453105A (zh) * 2023-06-20 2023-07-18 青岛国实科技集团有限公司 基于知识蒸馏深度神经网络的船牌号识别方法及系统
CN116453105B (zh) * 2023-06-20 2023-08-18 青岛国实科技集团有限公司 基于知识蒸馏深度神经网络的船牌号识别方法及系统

Also Published As

Publication number Publication date
US11645857B2 (en) 2023-05-09
CN111062396A (zh) 2020-04-24
US20220319204A1 (en) 2022-10-06
CN111062396B (zh) 2022-03-25

Similar Documents

Publication Publication Date Title
WO2021103897A1 (zh) 车牌号码识别方法、装置、电子设备及存储介质
CN111191663B (zh) 车牌号码识别方法、装置、电子设备及存储介质
US10949687B2 (en) Method and apparatus for outputting information
CN112381775B (zh) 一种图像篡改检测方法、终端设备及存储介质
CN103473950B (zh) 一种停车场车位监控方法
CN109886210A (zh) 一种交通图像识别方法、装置、计算机设备和介质
CN111598182B (zh) 训练神经网络及图像识别的方法、装置、设备及介质
CN111985451A (zh) 一种基于YOLOv4的无人机场景检测方法
CN112150450B (zh) 一种基于双通道U-Net模型的图像篡改检测方法及装置
CN111274942A (zh) 一种基于级联网络的交通锥识别方法及装置
CN115082492A (zh) 一种基于多光谱图像的语义分割方法、装置及存储介质
CN111008608B (zh) 一种基于深度学习的夜间车辆检测方法
CN109993806A (zh) 一种颜色识别方法、装置及电子设备
Zhang et al. Automatic detection of road traffic signs from natural scene images based on pixel vector and central projected shape feature
CN108323209A (zh) 信息处理方法、系统、云处理设备以及计算机程序产品
CN112784834A (zh) 一种自然场景下的车牌自动识别方法
CN112364864A (zh) 一种车牌识别方法、装置、电子设备及存储介质
US10949710B2 (en) Methods, systems and apparatuses of feature extraction and object detection
CN114202775A (zh) 基于红外图像的变电站危险区域行人闯入检测方法及系统
CN114596548A (zh) 目标检测方法、装置、计算机设备及计算机可读存储介质
CN117437615A (zh) 雾天交通标志检测方法、装置、存储介质和电子设备
CN112528994A (zh) 一种自由角度车牌检测方法、车牌识别方法和识别系统
CN116071625A (zh) 深度学习模型的训练方法、目标检测方法及装置
CN116363072A (zh) 一种轻型航拍图像检测方法及系统
CN112633264B (zh) 一种车辆属性识别方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20893958

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20893958

Country of ref document: EP

Kind code of ref document: A1