CN107273832B - License plate recognition method and system based on integral channel characteristics and convolutional neural network - Google Patents

License plate recognition method and system based on integral channel characteristics and convolutional neural network Download PDF

Info

Publication number
CN107273832B
CN107273832B CN201710416776.5A CN201710416776A CN107273832B CN 107273832 B CN107273832 B CN 107273832B CN 201710416776 A CN201710416776 A CN 201710416776A CN 107273832 B CN107273832 B CN 107273832B
Authority
CN
China
Prior art keywords
neural network
convolutional neural
sample image
image
gradient
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710416776.5A
Other languages
Chinese (zh)
Other versions
CN107273832A (en
Inventor
房建宏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Institute Of Traffic Sciences
Original Assignee
Qingdao Institute Of Traffic Sciences
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao Institute Of Traffic Sciences filed Critical Qingdao Institute Of Traffic Sciences
Priority to CN201710416776.5A priority Critical patent/CN107273832B/en
Publication of CN107273832A publication Critical patent/CN107273832A/en
Application granted granted Critical
Publication of CN107273832B publication Critical patent/CN107273832B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Biology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a license plate recognition method and a license plate recognition system based on integral channel characteristics and a convolutional neural network, wherein the license plate recognition method comprises the following steps: acquiring a sample image of the license plate image, and generating a convolutional neural network detector according to the sample image; acquiring an image to be detected, and calculating to generate a feature pyramid of the image to be detected in different scales; detecting the characteristic pyramid through a sliding window by using a convolutional neural network detector to obtain target candidate regions under different scales; and distinguishing characters from non-characters in the target candidate area by using a first fully-connected layer of the convolutional neural network detector, and identifying the characters in the target candidate area by using a second fully-connected layer of the convolutional neural network detector.

Description

License plate recognition method and system based on integral channel characteristics and convolutional neural network
Technical Field
The invention relates to a computer vision identification technology, in particular to a license plate identification method and a license plate identification system based on integral channel characteristics and a convolutional neural network.
Background
The automatic detection and identification of the license plate number of the passing vehicle at the positions of an expressway entrance, a parking lot entrance and the like are important technologies in an intelligent traffic system. In general, the detection and recognition of the license plate become difficult due to the extremely difficult conditions of strong illumination, large side angle, blur and the like. Currently, there are three major types of license plate detection methods:
the first is an edge information based approach. The method combines edge information with Hough transformation, morphological operation and the like to process an image to obtain a candidate region of a target in the image, then processes the regions through specific priori knowledge such as region edge density information, aspect ratio, shape and the like, screens layer by layer, and finally obtains the region where a license plate is located. Faradji et al, in documents "f.faradji, a.h.rezaie, and m.ziaratban, a morphological-based license plate location, in proc.ieee int.conf.image Process, pp. 57-60, sep. -oct.2007" propose a method for detecting a license plate by combining edge information and morphological operations, which obtains an object candidate region by performing edge detection on an image and then performing morphological operations. And then screening by using the geometric attributes of the candidate regions, and finally realizing the positioning of the license plate. The defects of the method are that the method is easily interfered by objects with abundant texture characteristics and similar shapes, and in addition, when the license plate is deformed, the license plate is easily missed.
The second method is a method based on color information, which comprises modules of color segmentation, target positioning and the like, and adopts a multilayer perceptron to segment color images, and then segments potential license plate areas by a projection method. For example, in documents "w.jia, h.zhang, x.he, and m.piccrardi, Mean shift for acquisition license plate localization. in proc.ieee conf.inner. trans.syst.pp.566-571, sep.2005" by w.jia et al, a number of candidate regions are obtained by dividing a color image by means of a Mean shift algorithm, and a detection result is finally obtained by classifying whether the candidate regions are license plate regions according to geometric attributes and edge density information. The method has the defects of sensitivity to illumination change and susceptibility to interference of areas with the same color characteristics in the image.
The third method is a method based on machine learning, such as the method proposed by l.dlagnekov et al in the documents "l.dlagnekov, and s.belongie, recogning cars.dept.comput.sci.eng.ucsd, San Diego, tech.rep.cs2005-083,2005", and based on Harr-Like features, the AdaBoost classifier is used to classify candidate regions to realize the positioning of the license plate. The method has the defects that the false alarm rate is high, a plurality of false detections occur, and the license plate region is difficult to detect completely, so that the algorithm effect of machine learning alone is not good.
Disclosure of Invention
The embodiment of the invention mainly aims to provide a license plate recognition method and system based on integral channel characteristics and a convolutional neural network, so as to solve the problem of license plate detection and recognition in a complex scene.
In order to achieve the above object, an embodiment of the present invention provides a license plate recognition method based on an integral channel feature and a convolutional neural network, where the license plate recognition method includes: obtaining a sample image of a license plate image, and generating a convolutional neural network detector according to the sample image; acquiring an image to be detected, and calculating to generate a feature pyramid of the image to be detected with different scales; detecting the characteristic pyramid through a sliding window by using the convolutional neural network detector to obtain target candidate regions under different scales; and distinguishing characters from non-characters in the target candidate area by using a first fully-connected layer of the convolutional neural network detector, and identifying the characters in the target candidate area by using a second fully-connected layer of the convolutional neural network detector.
In an embodiment, the above generating a convolutional neural network detector according to the sample image specifically includes: forming a training set by the sample images, wherein the sample images comprise positive samples and negative samples, the positive samples are images containing license plates, and the negative samples are background images not containing license plates; calculating integral channel characteristics of each sample image in the training set; pooling the integral channel characteristics of the sample image to generate pooled characteristics of the sample image; and inputting the pooled features into a decision tree forest, optimizing a distribution function of Adaboost by adopting an Adaboost algorithm and a spatial distribution probability, and generating the convolutional neural network detector.
In an embodiment, the calculating the integral channel characteristics of each sample in the training set specifically includes: step a: converting the sample image to an HSV color space, calculating color characteristics of three channels:
Figure BDA0001313871040000021
wherein i, j represents the spatial coordinates of the sample image; h, S and V respectively correspond to values of three different channels, H represents color and is 60-330, red corresponds to 0, green corresponds to 120, and blue corresponds to 240; s represents saturation and brightness of color; v represents a color tone; step b: calculating gradient direction histogram characteristics of the sample image; step c: and generating the integral channel characteristics according to the characteristics of the three channels and the gradient direction histogram characteristics.
Further, the step b: calculating the gradient direction histogram feature of the sample image, specifically comprising: step b 1: calculating a gradient direction value of each pixel in the sample image: gx(x,y)＝H(x+1,y)- H(x-1,y)；Gy(x, y) ═ H (x, y +1) -H (x, y-1); where H (x, y) represents a pixel value in a gray space, Gx(x,y)，Gy(x, y) indicating the gradient in the horizontal and vertical directions at (x, y), respectively; step b 2: calculating gradient magnitude G (x, y) and direction according to the gradient direction value
Figure BDA0001313871040000037
Figure BDA0001313871040000031
Step b 3: projecting the gradient of each pixel in the unit in different intervals according to the direction according to the size of the specified unit to generate a gradient direction histogram of the whole unit; wherein, the angle range of each interval is 360/N, and N is the number of gradient directions; step b 4: determining the histogram of gradient directions features according to the histogram of gradient directions:
Figure BDA0001313871040000038
wherein i, j is the coordinate of the sample image; n is 1,2 … N, and indicates the section numbers corresponding to different gradient directions.
In an embodiment, the above calculating and generating the feature pyramid of the image to be detected in different scales specifically includes: step 1: calculating the multichannel characteristics of the image to be detected under different scales by the following formula:
Figure BDA0001313871040000032
wherein, FsRepresenting the feature corresponding to the scale s, wherein R represents resampling the image by using the scale s, F ═ Ω (I) represents the feature of the channel corresponding to the image, and Ω corresponds to different feature channels; step 2: according to the multichannel characteristics, calculating and generating characteristic graphs of the image to be detected under different scales through the following formula:
Figure BDA0001313871040000033
wherein, Fs′Is a feature map, λ, of the image to be detected at a scale sΩIs the scale factor corresponding to the omega channel and the characteristic F corresponding to other scales ssAccording to different scale imagesThe scale proportion relation between the two and the calculated characteristic diagram under a certain scale are approximated to obtain the characteristic diagram under another scale; and step 3: and generating a characteristic pyramid according to the characteristic graph of the image to be detected under each scale.
In one embodiment, λ corresponding to different channels Ω is calculatedΩThe process comprises the following steps: mean of global features of statistical data set with scale transformation
Figure BDA0001313871040000034
By the formula
Figure BDA0001313871040000035
Can obtain musAnd λΩThe relationship of (A) is as follows:
Figure BDA0001313871040000036
wherein, E2]Expected value, f, indicating errorΩ(Is) Is a weighted sum of all channels, i.e. fΩ(I)＝∑i,j,kωi,j,kF (i, j, k), ω feature is the weight of the corresponding channel, k represents the sequence of channels.
The embodiment of the invention also provides a license plate recognition system based on the integral channel characteristics and the convolutional neural network, and the license plate recognition system comprises: the convolutional neural network detector generating unit is used for acquiring a sample image of the license plate image and generating a convolutional neural network detector according to the sample image; the characteristic pyramid generating unit is used for acquiring an image to be detected and calculating to generate characteristic pyramids of different scales of the image to be detected; a target candidate region acquisition unit, configured to detect the feature pyramid through a sliding window by using the convolutional neural network detector, and acquire target candidate regions at different scales; and the identification unit is used for distinguishing characters from non-characters in the target candidate area by using a first full-connection layer of the convolutional neural network detector and identifying the characters in the target candidate area by using a second full-connection layer of the convolutional neural network detector.
In an embodiment, the convolutional neural network detector generating unit is specifically configured to: forming a training set by the sample images, wherein the sample images comprise positive samples and negative samples, the positive samples are images containing license plates, and the negative samples are background images not containing license plates; calculating integral channel characteristics of each sample image in the training set; pooling the integral channel characteristics of the sample image to generate pooled characteristics of the sample image; and inputting the pooled features into a decision tree forest, optimizing a distribution function of Adaboost by adopting an Adaboost algorithm and a spatial distribution probability, and generating the convolutional neural network detector.
In an embodiment, the calculating the integral channel characteristics of each sample in the training set specifically includes: step a: converting the sample image to an HSV color space, calculating color characteristics of three channels:
Figure BDA0001313871040000041
wherein i, j represents the spatial coordinates of the sample image; h, S and V respectively correspond to values of three different channels, H represents color and is 60-330, red corresponds to 0, green corresponds to 120, and blue corresponds to 240; s represents saturation and brightness of color; v represents a color tone; step b: calculating gradient direction histogram characteristics of the sample image; step c: and generating the integral channel characteristics according to the characteristics of the three channels and the gradient direction histogram characteristics.
Further, the step b: calculating the gradient direction histogram feature of the sample image, specifically comprising: step b 1: calculating a gradient direction value of each pixel in the sample image: gx(x,y)＝H(x+1,y)- H(x-1,y)；Gy(x, y) ═ H (x, y +1) -H (x, y-1); where H (x, y) represents a pixel value in a gray space, Gx(x,y)，Gy(x, y) indicating the gradient in the horizontal and vertical directions at (x, y), respectively; step b 2: calculating gradient magnitude G (x, y) and direction according to the gradient direction value
Figure BDA0001313871040000045
Figure BDA0001313871040000042
Step b 3: projecting the gradient of each pixel in the unit in different intervals according to the direction according to the size of the specified unit to generate a gradient direction histogram of the whole unit; wherein, the angle range of each interval is 360/N, and N is the number of gradient directions; step b 4: determining the histogram of gradient directions features according to the histogram of gradient directions:
Figure BDA0001313871040000043
wherein i, j is the coordinate of the sample image; n is 1,2 … N, and indicates the section numbers corresponding to different gradient directions.
In an embodiment, the feature pyramid generating unit is specifically configured to: step 1: calculating the multichannel characteristics of the image to be detected under different scales by the following formula:
Figure BDA0001313871040000044
wherein, FsRepresenting the feature corresponding to the scale s, wherein R represents resampling the image by using the scale s, F ═ Ω (I) represents the feature of the channel corresponding to the image, and Ω corresponds to different feature channels; step 2: according to the multichannel characteristics, calculating and generating characteristic graphs of the image to be detected under different scales through the following formula:
Figure BDA0001313871040000051
wherein, Fs′Is a feature map, λ, of the image to be detected at a scale sΩIs the scale factor corresponding to the omega channel and the characteristic F corresponding to other scales ssObtaining a feature map under another scale approximately according to the scale proportion relation between the images with different scales and the calculated feature map under a certain scale; and step 3: and generating a characteristic pyramid according to the characteristic graph of the image to be detected under each scale.
In one embodiment, λ corresponding to different channels Ω is calculatedΩThe process comprises the following steps: mean of global features of statistical data set with scale transformation
Figure BDA0001313871040000052
By the formula
Figure BDA0001313871040000053
Can obtain musAnd λΩThe relationship of (A) is as follows:
Figure BDA0001313871040000054
wherein, E2]Expected value, f, indicating errorΩ(Is) Is a weighted sum of all channels, i.e. fΩ(I)＝∑i,j,kωi,j,kF (i, j, k), ω feature is the weight of the corresponding channel, k represents the sequence of channels.
The recognition method has the advantages that the recognition method can obtain better detection effect under different illumination conditions (different brightness degrees in the day and at night) and weather conditions (different states in sunny days, rainy days and the like), can accurately recognize key information of the license plate, can be applied to detection and recognition of the license plate in a static mode (aiming at a certain frame of image captured) and can also be applied to detection and recognition of the license plate in a dynamic mode (aiming at continuous video streams), and has the advantages of powerful function, flexible application, high running speed, strong adaptability and less resource occupation.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive labor.
FIG. 1 is a flowchart of a license plate recognition method based on integral channel features and a convolutional neural network according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a specific process for generating a convolutional neural network detector from a sample image according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a specific process of computing integral channel characteristics of each sample image in a training set according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a license plate recognition system based on an integral channel feature and a convolutional neural network according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention provides a license plate recognition method and system based on integral channel characteristics and a convolutional neural network. The present invention will be described in detail below with reference to the accompanying drawings.
The embodiment of the invention provides a license plate recognition method based on integral channel characteristics and a convolutional neural network, which mainly comprises the following steps of:
step S101: acquiring a sample image of the license plate image, and generating a convolutional neural network detector according to the sample image;
step S102: acquiring an image to be detected, and calculating to generate a feature pyramid of the image to be detected in different scales;
step S103: detecting the characteristic pyramid through a sliding window by using a convolutional neural network detector to obtain target candidate regions under different scales;
step S104: and distinguishing characters from non-characters in the target candidate area by using a first fully-connected layer of the convolutional neural network detector, and identifying the characters in the target candidate area by using a second fully-connected layer of the convolutional neural network detector.
Through the steps S101 to S104, the license plate recognition method based on the integral channel feature and the convolutional neural network according to the embodiment of the present invention realizes the solution of the image feature map by learning the integral channel feature of the object and by using a sliding window method, quickly detects the object by using the cascaded decision tree forest, and realizes the recognition of the license plate by using the convolutional neural network.
Each step of the license plate recognition method based on the integral channel feature and the convolutional neural network of the embodiment of the present invention is further described below with reference to specific embodiments.
In the step S101, a sample image of the license plate image is obtained, and the convolutional neural network detector is generated according to the sample image.
Specifically, as shown in fig. 2, this step S101 mainly includes the following processes:
step S1011: and forming a training set by using the sample images, wherein the sample images comprise positive samples and negative samples, the positive samples are images containing license plates, and the negative samples are background images not containing license plates. In specific implementation, the sample image may include samples of license plates under different inclination angles, different illumination conditions and different degrees of dirt interference as positive samples, and corresponding regions without license plates are selected as negative samples, and the positive samples and the negative samples form a training set together.
Step S1012: and calculating integral channel characteristics of each sample image in the training set. As shown in fig. 3, the process of calculating the integral channel feature of each sample in the training set specifically includes the following steps:
step S301: converting the sample image into an HSV color space, and calculating the color characteristics of three channels:
Figure BDA0001313871040000071
wherein i, j represents the spatial coordinates of the sample image; h, S and V respectively correspond to values of three different channels, H represents color and is 60-330, red corresponds to 0, green corresponds to 120, and blue corresponds to 240; s represents saturation and brightness of color; v represents a color tone;
step S302: calculating the gradient direction histogram characteristics of the sample image; the method specifically comprises the following steps:
step b 1: calculating a gradient direction value of each pixel in the sample image:
Gx(x,y)＝H(x+1,y)-H(x-1,y)；
Gy(x,y)＝H(x,y+1)-H(x,y-1)；
where H (x, y) represents a pixel value in a gray space, Gx(x,y)，Gy(x, y) indicating the gradient in the horizontal and vertical directions at (x, y), respectively;
step b 2: calculating gradient magnitude G (x, y) and direction according to the gradient direction value
Figure BDA0001313871040000075
Figure BDA0001313871040000072
Figure BDA0001313871040000073
Step b 3: projecting the gradient of each pixel in the unit in different intervals according to the direction according to the size of the specified unit to generate a gradient direction histogram of the whole unit; wherein, the angle range of each interval is 360/N, and N is the number of gradient directions;
step b 4: determining the characteristics of the histogram of the gradient direction according to the histogram of the gradient direction:
Figure BDA0001313871040000074
wherein i, j is the coordinate of the sample image; n is 1,2 … N, and indicates the section numbers corresponding to different gradient directions.
After the gradient direction histogram feature of the sample image is obtained through calculation, through step S303: and generating an integral channel Feature (i, j) according to the features of the three channels and the gradient direction histogram Feature.
Figure BDA0001313871040000081
I.e. the integrated channel features at the i, j positions of the sample image are a combination of color features and gradient features.
After the integral channel features of the sample images in the training set are calculated, step S1013 is performed: and pooling the integral channel characteristics of the sample image to generate pooled characteristics of the sample image.
Specifically, the pooling process is as follows: and carrying out region segmentation on the obtained integral characteristic channel, and determining the size of each block of region as the pooling size. Then, max posing (using the maximum value of each pixel value in the image area as the result of the area) or average posing (using the average value of each pixel value in the image area as the result of the area) is performed on each area to obtain the pooled channel characteristics.
In one embodiment, during the pooling process, the features before and after pooling need to be processed, feature dimensions are unified, and the final features are combined to obtain a final feature for training the convolutional neural network detector, for example, the processing process described herein includes operations such as normalization and the like, which are common processing methods of general feature vectors, for example, the final result is a 4 × 4 result feature map, the actual form of which is 1 × 16 feature vector, and the normalization process is performed to satisfy the requirement of the feature vector
Figure BDA0001313871040000082
Unifying feature dimensions mainly comprises zero-padding the pooled features, i.e. zero-padding at the head and tail ends of the pooled features, so that the feature vector dimensions after pooling are the same as those before pooling.
Then, step S1014 is executed: inputting the pooled features into a decision tree forest, optimizing a distribution function of Adaboost by adopting an Adaboost algorithm and spatial distribution probability to generate a convolutional neural network detector, and training the convolutional neural network detector.
The general Adaboost is to directly add several weak classifiers obtained by training, that is, the weight of each weak classifier is 1 by default. In this embodiment, the distribution function of Adaboost is optimized by using the samples classified by the first (n-1) weak classifiers as the training of the nth weak classifier, and in the process of combining the last weak classifiers, the weights of the weak classifiers are not all 1, but different weights are assigned according to the performances of the weak classifiers in the training and testing processes.
In specific implementation, the training of the whole detector is mainly carried out in four stages. The first stage is 64 decision trees, (a decision tree is a tree-like decision graph based on the prior art and is generated with additional probability results, is a visual graph method applying statistical probability analysis, each non-leaf node of the decision tree represents a test on a characteristic attribute, each branch represents the output of the characteristic attribute on a certain value range, and each leaf node stores a category, the decision process using the decision tree is to start from a root node, test the corresponding characteristic attribute in the item to be classified, select the output branch according to the value of the corresponding characteristic attribute until reaching the leaf node, and take the category stored by the leaf node as the decision result), input training data into the decision tree forest (a classifier comprising a plurality of decision trees, wherein the decision trees in the forest are not related, when the data enters the decision tree forest, each decision tree classifies the sample to which category the sample belongs, and finally, taking the class with the highest classification result in all the decision trees as a final result), and taking the detection result thereof to place the data with wrong classification as 'important training data' into the next stage, namely the second stage, wherein 128 decision trees … … exist in the second stage, and so on until the training of the four stages is finished. The decision trees of the four stages are 64, 128, 256 and 1024 respectively, and the data are determined by referring to the common parameters in various applications.
In the step S102, the image to be detected is obtained, and the feature pyramid of the image to be detected with different scales is calculated and generated.
In an embodiment, the step S102 specifically includes the following processes:
step 1: calculating the multichannel characteristics of the image to be detected under different scales by the following formula:
Figure BDA0001313871040000091
wherein, FsRepresenting the feature corresponding to the scale s, wherein R represents resampling the image by using the scale s, F ═ Ω (I) represents the feature of the channel corresponding to the image, and Ω corresponds to different feature channels;
step 2: according to the multichannel characteristics, calculating and generating characteristic graphs of the images to be detected under different scales by the following formula:
Figure BDA0001313871040000092
wherein, Fs′Is a feature map, λ, of the image to be detected at a scale sΩIs the scale factor corresponding to the omega channel and the characteristic F corresponding to other scales ssAnd approximating to obtain a characteristic diagram under another scale according to the scale proportion relation between the images with different scales and the calculated characteristic diagram under a certain scale.
Wherein λ corresponds to different channel Ω in the above equations (1-1) and (1-2)ΩThe calculation process of (2) is as follows:
mean value mu of integral features of statistical data set along with scale transformations：
Figure BDA0001313871040000093
By the formula:
Figure BDA0001313871040000094
the following can be obtained:
Figure BDA0001313871040000095
in the above formula, E [, ]]Expected value, f, indicating errorΩ(Is) For the weighted sum of all channels, i.e.:
fΩ(I)＝∑i,j,kωi,j,kF(i,j,k) (1-6)，
the omega characteristic is pairK denotes the sequence of channels, depending on their weight. By combining the above equations (1-3) to (1-6), λ can be obtainedΩ。
And step 3: and generating a characteristic pyramid according to the characteristic graph of the image to be detected under each scale.
In the step S103, the convolutional neural network detector is used to detect the feature pyramid through a sliding window, so as to obtain target candidate regions at different scales. And obtaining the characteristics of the image to be detected in different scales by using the characteristic pyramid calculation method, and detecting the image to be detected in each scale through a sliding window by using a trained detector to obtain a target candidate region.
In one embodiment, a non-maximum suppression method (sorting the candidate regions with overlapping detection results according to their detection scores, obtaining the highest score as the detection result region, and removing other regions) is used for the obtained target candidate regions, and a suitable threshold is used to screen out the final detection region. In the non-maximum suppression, the overlapping ratio of the detected bounding boxes is calculated as follows:
the overlap is the intersection (bbs), the intersection (bbs) calculates the intersection rate of the detection result areas with intersection, the two detection frames with intersection are bb1 and bb2 respectively, the inter (bb1, bb2) represents the area of intersection of the two rectangular frames, the intersection (bb1, bb2) represents the union area of the two rectangular frames, and the intersection (bbs) is defined as inter (bb1, bb 2)/intersection (bb1, bb 2).
If overlap is greater than threshold τ, only bbs with the highest score is retainedi. In the invention, the value of tau is set to be 0.5. bbs are summed by the thresholds corresponding to the nodes of the decision tree.
In the step S104, the first fully-connected layer of the convolutional neural network detector is used to distinguish between characters and non-characters in the target candidate region, and the second fully-connected layer of the convolutional neural network detector is used to identify characters in the target candidate region.
Training a convolutional neural network by using positive and negative samples in the data set, and extracting the characteristics of the license plate area in the detection result; training a full connection layer for distinguishing seven character features, and distinguishing the seven character features in a license plate region; and training a full connection layer for character recognition, and classifying characters to realize the recognition of the characters in the license plate area.
The two fully-connected layers are essentially used for classification, and are connected in parallel, that is, after a feature map is obtained in a front convolution part (the feature map is a plurality of numerical value matrixes (similar to a form that an image is a two-dimensional matrix in a computer) obtained after an image to be detected is processed by each layer of a convolutional neural network, in the convolutional neural network, a front convolution layer part is mainly used for feature extraction, and a rear fully-connected layer part is mainly used for classification, so that data obtained through the convolution part is called the feature map and is used as a feature extracted from the image to be detected by the convolution part), and then the two fully-connected layers are connected in parallel. Firstly, bounding box information labeled by a feature map (the bounding box information refers to labeling information of a rectangular frame of a license plate area in training data, for each training picture, a corresponding xml file corresponds to the bounding box information, the content of the file is a character string corresponding to the rectangular frame (such as coordinates of an upper left corner point and a lower right corner point) of the license plate in the labeled picture and the content of the license plate, and a convolutional neural network is trained by taking the labeling information as guidance to determine what a desired result is, so that the adjustment of network parameters is performed according to the difference between a network prediction result and actual labeling data, the prediction result of the network is as the same as the actual data as possible, namely how the convolutional neural network is trained to recognize), and the network is input into a first full-connection layer for training, wherein only the classification of the license plate and a non-license plate is required; the training of the second full-connection layer needs a feature map and a bounding box, and also needs the labeling information of the license plate content, and the character recognition, namely the multi-classification process, is carried out, and the two can be parallel during the training, and the training has no dependence due to different classification tasks; however, during testing, the feature map needs to pass through the first full connection layer, the license plate region is further determined, and then the result is input into the second full connection layer to perform character recognition, that is, the second full connection layer is dependent on the result of the first full connection layer in the testing stage.
According to the license plate recognition method based on the integral channel characteristics and the convolutional neural network, the information of the license plate in different states is fully mined by utilizing the integral channel characteristics, so that the license plate can be more accurately expressed in characteristics, and the influence caused by deformation can be effectively dealt with through pooling operation. And the cascaded decision tree forest detectors are adopted, so that the target can be quickly detected. And fully searching the space where the image is located by using the sliding window, and positioning the region where the license plate appears. The model obtained through learning can be better expanded to different scenes, and meanwhile, the influence of other factors such as different illumination changes can be better responded.
The recognition method of the embodiment of the invention can obtain better detection effect under different illumination conditions (different brightness degrees in the daytime and at night) and weather conditions (different states in sunny days, rainy days and the like), can accurately recognize key information of the license plate, can be applied to detection and recognition of the license plate in a static mode (aiming at a certain frame of image captured) and can also be applied to detection and recognition of the license plate in a dynamic mode (aiming at continuous video streams), and has the advantages of strong function, flexible application, high running speed, strong adaptability and less resource occupation.
The embodiment of the present invention further provides a license plate recognition system based on the integral channel feature and the convolutional neural network, as shown in fig. 4, the license plate recognition system based on the integral channel feature and the convolutional neural network mainly includes: the device comprises a convolutional neural network detector generating unit 1, a characteristic pyramid generating unit 2, a target candidate region acquiring unit 3 and a recognition unit 4.
The convolutional neural network detector generating unit 1 is used for acquiring a sample image of a license plate image and generating a convolutional neural network detector according to the sample image; the characteristic pyramid generating unit 2 is used for acquiring an image to be detected and calculating and generating characteristic pyramids of different scales of the image to be detected; the target candidate region acquisition unit 3 is configured to detect the feature pyramid through a sliding window by using a convolutional neural network detector, and acquire target candidate regions at different scales; the recognition unit 4 is configured to distinguish characters from non-characters in the target candidate region by using a first fully-connected layer of the convolutional neural network detector, and recognize characters in the target candidate region by using a second fully-connected layer of the convolutional neural network detector.
Through the cooperative work among the components, the license plate recognition system based on the integral channel characteristics and the convolutional neural network realizes the solution of the image characteristic diagram by learning the integral channel characteristics of the object and by a sliding window method, quickly detects the object by utilizing the cascaded decision tree forest, and realizes the recognition of the license plate by using the convolutional neural network.
The following further describes each component and functions of the license plate recognition system based on the integral channel feature and the convolutional neural network according to the embodiment of the present invention with reference to specific embodiments.
The convolutional neural network detector generating unit 1 is configured to obtain a sample image of the license plate image, and generate a convolutional neural network detector according to the sample image.
Specifically, as shown in fig. 2, the convolutional neural network detector generating unit 1 mainly performs the following processes:
step S1011: and forming a training set by using the sample images, wherein the sample images comprise positive samples and negative samples, the positive samples are images containing license plates, and the negative samples are background images not containing license plates. In specific implementation, the sample image may include samples of license plates under different inclination angles, different illumination conditions and different degrees of dirt interference as positive samples, and corresponding regions without license plates are selected as negative samples, and the positive samples and the negative samples form a training set together.
Step S1012 calculates the integral channel characteristics of each sample image in the training set. As shown in fig. 3, the process of calculating the integral channel feature of each sample in the training set specifically includes the following steps:
step S301: converting the sample image into an HSV color space, and calculating the color characteristics of three channels:
Figure BDA0001313871040000121
wherein i, j represents the spatial coordinates of the sample image; h, S and V respectively correspond to values of three different channels, H represents color and is 60-330, red corresponds to 0, green corresponds to 120, and blue corresponds to 240; s represents saturation and brightness of color; v represents a color tone;
step S302: calculating the gradient direction histogram characteristics of the sample image; the method specifically comprises the following steps:
step b 1: calculating a gradient direction value of each pixel in the sample image:
Gx(x,y)＝H(x+1,y)-H(x-1,y)；
Gy(x,y)＝H(x,y+1)-H(x,y-1)；
where H (x, y) represents a pixel value in a gray space, Gx(x,y)，Gy(x, y) indicating the gradient in the horizontal and vertical directions at (x, y), respectively;
step b 2: calculating gradient magnitude G (x, y) and direction according to the gradient direction value
Figure BDA0001313871040000135
Figure BDA0001313871040000131
Figure BDA0001313871040000132
Step b 3: projecting the gradient of each pixel in the unit in different intervals according to the direction according to the size of the specified unit to generate a gradient direction histogram of the whole unit; wherein, the angle range of each interval is 360/N, and N is the number of gradient directions;
step b 4: determining the characteristics of the histogram of the gradient direction according to the histogram of the gradient direction:
Figure BDA0001313871040000133
wherein i, j is the coordinate of the sample image; n is 1,2 … N, and indicates the section numbers corresponding to different gradient directions.
After the gradient direction histogram feature of the sample image is obtained through calculation, through step S303: and generating an integral channel Feature (i, j) according to the features of the three channels and the gradient direction histogram Feature.
Figure BDA0001313871040000134
I.e. the integrated channel features at the i, j positions of the sample image are a combination of color features and gradient features.
After the integral channel features of each sample image in the training set are calculated, the convolutional neural network detector generating unit 1 performs step S1013: and pooling the integral channel characteristics of the sample image to generate pooled characteristics of the sample image.
Specifically, the pooling process is as follows: and carrying out region segmentation on the obtained integral characteristic channel, and determining the size of each block of region as the pooling size. Then, maxporoling (using the maximum value of each pixel value in the image area as the result of the area) or average posing (using the average value of each pixel value in the image area as the result of the area) is performed on each area to obtain the pooled channel characteristics.
In one embodiment, during the pooling process, the features before and after pooling need to be processed, feature dimensions are unified, and the final features are combined to obtain a final feature for training the convolutional neural network detector, for example, the processing process described herein includes operations such as normalization and the like, which are common processing methods of general feature vectors, for example, the final result is a 4 × 4 result feature map, the actual form of which is 1 × 16 feature vector, and the normalization process is performed to satisfy the requirement of the feature vector
Figure BDA0001313871040000141
Unifying feature dimensions mainly comprises zero-padding the pooled features, i.e. zero-padding at the head and tail ends of the pooled features to make the pooled features zero-paddedThe eigenvector dimensions are the same as before pooling.
Then, the convolutional neural network detector generating unit 1 performs step S1014: inputting the pooled features into a decision tree forest, optimizing a distribution function of Adaboost by adopting an Adaboost algorithm and spatial distribution probability to generate a convolutional neural network detector, and training the convolutional neural network detector.
The general Adaboost is to directly add several weak classifiers obtained by training, that is, the weight of each weak classifier is 1 by default. In this embodiment, the distribution function of Adaboost is optimized by using the samples classified by the first (n-1) weak classifiers as the training of the nth weak classifier, and in the process of combining the last weak classifiers, the weights of the weak classifiers are not all 1, but different weights are assigned according to the performances of the weak classifiers in the training and testing processes.
In specific implementation, the training of the whole detector is mainly carried out in four stages. The first stage is 64 decision trees, (a decision tree is a tree-like decision graph based on the prior art and is generated with additional probability results, is a visual graph method applying statistical probability analysis, each non-leaf node of the decision tree represents a test on a characteristic attribute, each branch represents the output of the characteristic attribute on a certain value range, and each leaf node stores a category, the decision process using the decision tree is to start from a root node, test the corresponding characteristic attribute in the item to be classified, select the output branch according to the value of the corresponding characteristic attribute until reaching the leaf node, and take the category stored by the leaf node as the decision result), input training data into the decision tree forest (a classifier comprising a plurality of decision trees, wherein the decision trees in the forest are not related, when the data enters the decision tree forest, each decision tree classifies the sample to which category the sample belongs, and finally, taking the class with the highest classification result in all the decision trees as a final result), and taking the detection result thereof to place the data with wrong classification as 'important training data' into the next stage, namely the second stage, wherein 128 decision trees … … exist in the second stage, and so on until the training of the four stages is finished. The decision trees of the four stages are 64, 128, 256 and 1024 respectively, and the data are determined by referring to the common parameters in various applications.
The feature pyramid generation unit 2 is configured to obtain an image to be detected, and calculate and generate feature pyramids of different scales of the image to be detected.
The feature pyramid generation unit 2 specifically executes the following processes:
step 1: calculating the multichannel characteristics of the image to be detected under different scales by the following formula:
Figure BDA0001313871040000142
wherein, FsRepresenting the feature corresponding to the scale s, wherein R represents resampling the image by using the scale s, F ═ Ω (I) represents the feature of the channel corresponding to the image, and Ω corresponds to different feature channels;
step 2: according to the multichannel characteristics, calculating and generating characteristic graphs of the images to be detected under different scales by the following formula:
Figure BDA0001313871040000151
wherein, Fs′Is a feature map, λ, of the image to be detected at a scale sΩIs the scale factor corresponding to the omega channel and the characteristic F corresponding to other scales ssAnd approximating to obtain a characteristic diagram under another scale according to the scale proportion relation between the images with different scales and the calculated characteristic diagram under a certain scale.
Wherein λ corresponds to different channel Ω in the above equations (1-1) and (1-2)ΩThe calculation process of (2) is as follows:
mean value mu of integral features of statistical data set along with scale transformations：
Figure BDA0001313871040000152
By the formula:
Figure BDA0001313871040000153
the following can be obtained:
Figure BDA0001313871040000154
in the above formula, E [, ]]Expected value, f, indicating errorΩ(Is) For the weighted sum of all channels, i.e.:
fΩ(I)＝∑i,j,kωi,j,kF(i,j,k) (1-6)，
the ω feature is the weight of the corresponding channel, and k represents the sequence of channels. By combining the above equations (1-3) to (1-6), λ can be obtainedΩ。
And step 3: and generating a characteristic pyramid according to the characteristic graph of the image to be detected under each scale.
The target candidate region obtaining unit 3 is configured to use a convolutional neural network detector to detect the feature pyramid through a sliding window, so as to obtain target candidate regions at different scales. And obtaining the characteristics of the image to be detected in different scales by using the characteristic pyramid calculation method, and detecting the image to be detected in each scale through a sliding window by using a trained detector to obtain a target candidate region.
In one embodiment, a non-maximum suppression method (sorting the candidate regions with overlapping detection results according to their detection scores, obtaining the highest score as the detection result region, and removing other regions) is used for the obtained target candidate regions, and a suitable threshold is used to screen out the final detection region. In the non-maximum suppression, the overlapping ratio of the detected bounding boxes is calculated as follows:
the overlap is the intersection (bbs), the intersection (bbs) calculates the intersection rate of the detection result areas with intersection, the two detection frames with intersection are bb1 and bb2 respectively, the inter (bb1, bb2) represents the area of intersection of the two rectangular frames, the intersection (bb1, bb2) represents the union area of the two rectangular frames, and the intersection (bbs) is defined as inter (bb1, bb 2)/intersection (bb1, bb 2).
If overlap is greater than threshold τ, only bbs with the highest score is retainedi. In the invention, the value of tau is set to be 0.5. bbs are summed by the thresholds corresponding to the nodes of the decision tree.
The above-mentioned recognition unit 4 is configured to distinguish between a character and a non-character in the target candidate region by using the first fully-connected layer of the convolutional neural network detector, and recognize the character in the target candidate region by using the second fully-connected layer of the convolutional neural network detector.
Training a convolutional neural network by using positive and negative samples in the data set, and extracting the characteristics of the license plate area in the detection result; training a full connection layer for distinguishing seven character features, and distinguishing the seven character features in a license plate region; and training a full connection layer for character recognition, and classifying characters to realize the recognition of the characters in the license plate area.
The two fully-connected layers are essentially used for classification, and are connected in parallel, that is, after a feature map is obtained in a front convolution part (the feature map is a plurality of numerical value matrixes (similar to a form that an image is a two-dimensional matrix in a computer) obtained after an image to be detected is processed by each layer of a convolutional neural network, in the convolutional neural network, a front convolution layer part is mainly used for feature extraction, and a rear fully-connected layer part is mainly used for classification, so that data obtained through the convolution part is called the feature map and is used as a feature extracted from the image to be detected by the convolution part), and then the two fully-connected layers are connected in parallel. Firstly, bounding box information labeled by a feature map (the bounding box information refers to labeling information of a rectangular frame of a license plate area in training data, for each training picture, a corresponding xml file corresponds to the bounding box information, the content of the file is a character string corresponding to the rectangular frame (such as coordinates of an upper left corner point and a lower right corner point) of the license plate in the labeled picture and the content of the license plate, and a convolutional neural network is trained by taking the labeling information as guidance to determine what a desired result is, so that the adjustment of network parameters is performed according to the difference between a network prediction result and actual labeling data, the prediction result of the network is as the same as the actual data as possible, namely how the convolutional neural network is trained to recognize), and the network is input into a first full-connection layer for training, wherein only the classification of the license plate and a non-license plate is required; the training of the second full-connection layer needs a feature map and a bounding box, and also needs the labeling information of the license plate content, and the character recognition, namely the multi-classification process, is carried out, and the two can be parallel during the training, and the training has no dependence due to different classification tasks; however, during testing, the feature map needs to pass through the first full connection layer, the license plate region is further determined, and then the result is input into the second full connection layer to perform character recognition, that is, the second full connection layer is dependent on the result of the first full connection layer in the testing stage.
According to the license plate recognition system based on the integral channel characteristics and the convolutional neural network, the information of the license plate in different states is fully mined by utilizing the integral channel characteristics, so that the license plate can be more accurately expressed in characteristics, and the influence caused by deformation can be effectively dealt with through pooling operation. And the cascaded decision tree forest detectors are adopted, so that the target can be quickly detected. And fully searching the space where the image is located by using the sliding window, and positioning the region where the license plate appears. The model obtained through learning can be better expanded to different scenes, and meanwhile, the influence of other factors such as different illumination changes can be better responded.
The recognition system of the embodiment of the invention can obtain better detection effect under different illumination conditions (different brightness degrees in the daytime and at night) and weather conditions (different states in sunny days, rainy days and the like), can accurately recognize key information of the license plate, can be applied to detection and recognition of the license plate in a static mode (aiming at a certain frame of image captured) and can also be applied to detection and recognition of the license plate in a dynamic mode (aiming at continuous video streams), and has the advantages of strong function, flexible application, high running speed, strong adaptability and less resource occupation.
It will be understood by those skilled in the art that all or part of the steps in the method for implementing the above embodiments may be implemented by relevant hardware instructed by a program, and the program may be stored in a computer readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (4)

1. A license plate recognition method based on integral channel characteristics and a convolutional neural network is characterized by comprising the following steps:
obtaining a sample image of a license plate image, and generating a convolutional neural network detector according to the sample image;
acquiring an image to be detected, and calculating to generate a feature pyramid of the image to be detected with different scales;
detecting the characteristic pyramid through a sliding window by using the convolutional neural network detector to obtain target candidate regions under different scales;
distinguishing characters from non-characters in the target candidate area by using a first fully-connected layer of the convolutional neural network detector, and identifying the characters in the target candidate area by using a second fully-connected layer of the convolutional neural network detector; generating a convolutional neural network detector according to the sample image, specifically comprising:
forming a training set by the sample images, wherein the sample images comprise positive samples and negative samples, the positive samples are images containing license plates, and the negative samples are background images not containing license plates;
calculating integral channel characteristics of each sample image in the training set;
pooling the integral channel characteristics of the sample image to generate pooled characteristics of the sample image;
inputting the pooled features into a decision tree forest, optimizing a distribution function of Adaboost by adopting an Adaboost algorithm and through spatial distribution probability, and generating the convolutional neural network detector;
calculating integral channel characteristics of each sample in the training set, specifically comprising:
step a: converting the sample image to an HSV color space, calculating color characteristics of three channels:
Figure FDA0002478559140000011
wherein i, j represents the spatial coordinates of the sample image; h, S and V respectively correspond to values of three different channels, H represents color and is 60-330, red corresponds to 0, green corresponds to 120, and blue corresponds to 240; s represents saturation and brightness of color; v represents a color tone;
step b: calculating gradient direction histogram characteristics of the sample image;
step c: and generating the integral channel characteristics according to the characteristics of the three channels and the gradient direction histogram characteristics.
2. The license plate recognition method based on the integral channel feature and the convolutional neural network of claim 1, wherein the step b: calculating the gradient direction histogram feature of the sample image, specifically comprising:
step b 1: calculating a gradient direction value of each pixel in the sample image:
Gx(x,y)＝H(x+1,y)-H(x-1,y)；
Gy(x,y)＝H(x,y+1)-H(x,y-1)；
where H (x, y) represents a pixel value in a gray space, Gx(x,y)，Gy(x, y) indicating the gradient in the horizontal and vertical directions at (x, y), respectively;
step b 2: calculating gradient magnitude G (x, y) and direction according to the gradient direction value
Figure FDA0002478559140000024
Figure FDA0002478559140000021
Figure FDA0002478559140000022
Step b 3: projecting the gradient of each pixel in the unit in different intervals according to the direction according to the size of the specified unit to generate a gradient direction histogram of the whole unit; wherein, the angle range of each interval is 360/N, and N is the number of gradient directions;
step b 4: determining the histogram of gradient directions features according to the histogram of gradient directions:
Figure FDA0002478559140000023
wherein i, j is the coordinate of the sample image; n is 1,2 … N, and indicates the section numbers corresponding to different gradient directions.
3. A license plate recognition system based on integral channel characteristics and a convolutional neural network is characterized by comprising:
the convolutional neural network detector generating unit is used for acquiring a sample image of the license plate image and generating a convolutional neural network detector according to the sample image;
the characteristic pyramid generating unit is used for acquiring an image to be detected and calculating to generate characteristic pyramids of different scales of the image to be detected;
a target candidate region acquisition unit, configured to detect the feature pyramid through a sliding window by using the convolutional neural network detector, and acquire target candidate regions at different scales;
the recognition unit is used for distinguishing characters from non-characters in the target candidate area by using a first full-connection layer of the convolutional neural network detector and recognizing the characters in the target candidate area by using a second full-connection layer of the convolutional neural network detector;
the convolutional neural network detector generating unit is specifically configured to:
forming a training set by the sample images, wherein the sample images comprise positive samples and negative samples, the positive samples are images containing license plates, and the negative samples are background images not containing license plates;
calculating integral channel characteristics of each sample image in the training set;
pooling the integral channel characteristics of the sample image to generate pooled characteristics of the sample image;
inputting the pooled features into a decision tree forest, optimizing a distribution function of Adaboost by adopting an Adaboost algorithm and through spatial distribution probability, and generating the convolutional neural network detector;
calculating integral channel characteristics of each sample in the training set, specifically comprising:
step a: converting the sample image to an HSV color space, calculating color characteristics of three channels:
Figure FDA0002478559140000031
wherein i, j represents the spatial coordinates of the sample image; h, S and V respectively correspond to values of three different channels, H represents color and is 60-330, red corresponds to 0, green corresponds to 120, and blue corresponds to 240; s represents saturation and brightness of color; v represents a color tone;
step b: calculating gradient direction histogram characteristics of the sample image;
step c: and generating the integral channel characteristics according to the characteristics of the three channels and the gradient direction histogram characteristics.
4. The license plate recognition system based on the integral channel feature and the convolutional neural network of claim 3, wherein the step b: calculating the gradient direction histogram feature of the sample image, specifically comprising:
step b 1: calculating a gradient direction value of each pixel in the sample image:
Gx(x,y)＝H(x+1,y)-H(x-1,y)；
Gy(x,y)＝H(x,y+1)-H(x,y-1)；
where H (x, y) represents a pixel value in a gray space, Gx(x,y)，Gy(x, y) indicating the gradient in the horizontal and vertical directions at (x, y), respectively;
step b 2: calculating gradient magnitude G (x, y) and direction according to the gradient direction value
Figure FDA0002478559140000035
Figure FDA0002478559140000032
Figure FDA0002478559140000033
Step b 3: projecting the gradient of each pixel in the unit in different intervals according to the direction according to the size of the specified unit to generate a gradient direction histogram of the whole unit; wherein, the angle range of each interval is 360/N, and N is the number of gradient directions;
step b 4: determining the histogram of gradient directions features according to the histogram of gradient directions:
Figure FDA0002478559140000034
wherein i, j is the coordinate of the sample image; n is 1,2 … N, and indicates the section numbers corresponding to different gradient directions.
CN201710416776.5A 2017-06-06 2017-06-06 License plate recognition method and system based on integral channel characteristics and convolutional neural network Active CN107273832B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710416776.5A CN107273832B (en) 2017-06-06 2017-06-06 License plate recognition method and system based on integral channel characteristics and convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710416776.5A CN107273832B (en) 2017-06-06 2017-06-06 License plate recognition method and system based on integral channel characteristics and convolutional neural network

Publications (2)

Publication Number Publication Date
CN107273832A CN107273832A (en) 2017-10-20
CN107273832B true CN107273832B (en) 2020-09-22

Family

ID=60064600

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710416776.5A Active CN107273832B (en) 2017-06-06 2017-06-06 License plate recognition method and system based on integral channel characteristics and convolutional neural network

Country Status (1)

Country Link
CN (1) CN107273832B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110069651B (en) * 2017-10-23 2023-04-07 腾讯科技（北京）有限公司 Picture screening method and device and storage medium
CN108154149B (en) * 2017-12-08 2021-12-10 济南中维世纪科技有限公司 License plate recognition method based on deep learning network sharing
CN109325449B (en) * 2018-01-04 2022-06-21 苏州中科天启遥感科技有限公司 Convolutional neural network target detection framework based on sample updating
CN108960026B (en) * 2018-03-10 2019-04-12 王梦梦 Unmanned plane during flying Orientation system
CN108520278A (en) * 2018-04-10 2018-09-11 陕西师范大学 A kind of road surface crack detection method and its evaluation method based on random forest
CN111801689A (en) * 2018-04-17 2020-10-20 赫尔实验室有限公司 System for real-time object detection and recognition using image and size features
WO2020003345A1 (en) * 2018-06-25 2020-01-02 オリンパス株式会社 Arithmetic processing device
CN110020651B (en) * 2019-04-19 2022-07-08 福州大学 License plate detection and positioning method based on deep learning network
CN111127327B (en) * 2019-11-14 2024-04-12 贝壳技术有限公司 Picture inclination detection method and device
CN112381150A (en) * 2020-11-17 2021-02-19 上海科技大学 Confrontation sample detection method based on sample robustness difference

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104239867B (en) * 2014-09-17 2017-12-01 深圳市捷顺科技实业股份有限公司 License plate locating method and system
CN104809443B (en) * 2015-05-05 2018-12-28 上海交通大学 Detection method of license plate and system based on convolutional neural networks
US9785855B2 (en) * 2015-12-17 2017-10-10 Conduent Business Services, Llc Coarse-to-fine cascade adaptations for license plate recognition with convolutional neural networks
CN106295507B (en) * 2016-07-25 2019-10-18 华南理工大学 A kind of gender identification method based on integrated convolutional neural networks
CN106529522A (en) * 2016-11-07 2017-03-22 湖南源信光电科技有限公司 License plate location method based on integral channel features

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Fast Feature Pyramids for Object Detection;Piotr Doll´ar等;《IEEE Transactions on Pattern Analysis and Machine Intelligence》;IEEE;20140116;第36卷(第8期);第1532-1545页 *

Also Published As

Publication number Publication date
CN107273832A (en) 2017-10-20

Similar Documents

Publication Publication Date Title
CN107273832B (en) License plate recognition method and system based on integral channel characteristics and convolutional neural network
CN110543837B (en) Visible light airport airplane detection method based on potential target point
US8509526B2 (en) Detection of objects in digital images
CN110929593B (en) Real-time significance pedestrian detection method based on detail discrimination
CN111767882A (en) Multi-mode pedestrian detection method based on improved YOLO model
WO2012139228A1 (en) Video-based detection of multiple object types under varying poses
CN109902576B (en) Training method and application of head and shoulder image classifier
US10255511B2 (en) Real time traffic sign recognition
CN104036284A (en) Adaboost algorithm based multi-scale pedestrian detection method
CN103632170A (en) Pedestrian detection method and device based on characteristic combination
Lyu et al. Small object recognition algorithm of grain pests based on SSD feature fusion
CN111915583B (en) Vehicle and pedestrian detection method based on vehicle-mounted thermal infrared imager in complex scene
CN110008899B (en) Method for extracting and classifying candidate targets of visible light remote sensing image
Naufal et al. Preprocessed mask RCNN for parking space detection in smart parking systems
CN114049572A (en) Detection method for identifying small target
Mammeri et al. North-American speed limit sign detection and recognition for smart cars
CN111274964B (en) Detection method for analyzing water surface pollutants based on visual saliency of unmanned aerial vehicle
Ahmed et al. Traffic sign detection and recognition model using support vector machine and histogram of oriented gradient
CN114708566A (en) Improved YOLOv 4-based automatic driving target detection method
CN113128476A (en) Low-power consumption real-time helmet detection method based on computer vision target detection
CN115620090A (en) Model training method, low-illumination target re-recognition method and device and terminal equipment
Dousai et al. Detecting humans in search and rescue operations based on ensemble learning
CN112347967B (en) Pedestrian detection method fusing motion information in complex scene
CN114596548A (en) Target detection method, target detection device, computer equipment and computer-readable storage medium
Chughtai et al. Traffic Surveillance System: Robust Multiclass Vehicle Detection and Classification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant