CN113343977B - Multipath automatic identification method for container terminal truck collection license plate - Google Patents

Multipath automatic identification method for container terminal truck collection license plate Download PDF

Info

Publication number
CN113343977B
CN113343977B CN202110534074.3A CN202110534074A CN113343977B CN 113343977 B CN113343977 B CN 113343977B CN 202110534074 A CN202110534074 A CN 202110534074A CN 113343977 B CN113343977 B CN 113343977B
Authority
CN
China
Prior art keywords
license plate
truck
network
image
representing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110534074.3A
Other languages
Chinese (zh)
Other versions
CN113343977A (en
Inventor
郭建明
张港
刘清
王灏天
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University of Technology WUT
Original Assignee
Wuhan University of Technology WUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University of Technology WUT filed Critical Wuhan University of Technology WUT
Priority to CN202110534074.3A priority Critical patent/CN113343977B/en
Publication of CN113343977A publication Critical patent/CN113343977A/en
Application granted granted Critical
Publication of CN113343977B publication Critical patent/CN113343977B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/02—Affine transformations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Credit Cards Or The Like (AREA)

Abstract

The invention provides a multipath automatic identification method for container terminal truck-collecting license plates. Firstly, a network monitoring camera is utilized to shoot a truck head image of a truck to construct a truck head image data set and a truck license plate image data set of the truck under the condition that the truck does not stop, and a truck head image training set and a truck license plate image training set are respectively formed after corresponding preprocessing operations. Then, a card collection license plate detection network, a card collection license plate recognition network and a corresponding loss function are constructed, corresponding training set images are input, and the corresponding networks are optimized through a gradient descent algorithm. During testing, inputting an image to be detected into the optimized card collecting license plate detection network and the card collecting license plate identification network to obtain a primary identification result; and obtaining a final recognition result after post-processing operations such as confidence screening and the like. The invention has the advantages of high recognition rate, strong real-time performance and the like, realizes the multi-path automatic recognition of the container terminal truck-collecting license plate, and greatly improves the high-efficiency, safety and intelligent level of the port container transportation industry.

Description

Multipath automatic identification method for container terminal truck collection license plate
Technical Field
The invention belongs to the field of visual identification of license plates of container terminals, and particularly relates to a multipath automatic identification method for container terminal truck-collecting license plates.
Background
In recent years, the port container Transportation industry is rapidly developed, and establishment of an efficient intelligent Transportation system its (intelligent Transportation system) becomes a problem to be solved at present, and the license plate identification technology is an important means for realizing traffic management automation. At present, the automatic identification of the license plate number of the motor vehicle is very important in traffic management, social security and the like, particularly highway charging, violation detection, community vehicle management and the like, but the automatic identification is not widely applied to container terminals, most of the automatic identification is manual or semi-automatic treatment at present, the manpower is wasted, and the efficiency is low.
For the application scene of multipath automatic identification of container terminal truck license plates, the license plate identification technology based on deep learning has natural advantages compared with manual detection, and the automatic identification of the truck license plate number can improve the passing efficiency of a truck passing through a gate and a crossing; because the license plate number can be fully automatically entered into the computer system without the intervention of personnel, the automatic identification of the license plate number can realize the automation of the operation flow of the storage yard. Therefore, the automatic identification technology of the container terminal truck-collecting license plate greatly promotes the port container transportation industry to develop to high efficiency, safety and intellectualization.
Disclosure of Invention
The invention provides a multipath automatic identification method for container terminal truck-collecting license plates, which aims at solving the problems that manual recording is carried out on the container terminal truck-collecting license plate in a manual identification mode, so that recording errors caused by manual identification are easy to occur, the efficiency is low and the like.
In order to achieve the purpose, the technical scheme adopted by the invention is a multipath automatic identification method for container terminal truck collection license plates, which is characterized by comprising the following steps:
step 1: a network monitoring camera is used for shooting high-resolution images of the front side or the front side of the truck collection under the condition that the truck collection does not stop at a container terminal gate, a crossing or a yard passageway and the like to construct a truck collection head image data set, and the high-resolution images of the truck collection head are reduced to a proper proportion by an equal proportion through a double-cubic interpolation method to construct the truck collection head image data set. Obtaining a corrected license plate image from the high-resolution image of the head of the truck by using a transmission transformation method, and reducing the truck collection license plate image to a proper proportion by using a double-cubic interpolation method in an equal proportion to construct a truck collection license plate image data set;
step 2: and (3) manually labeling the license plate mark frame of each truck head image in the truck head image data set in the step 1, and constructing a truck license plate detection network training set. Manually labeling license plate number information of each license plate image in the collection card license plate image data set in the step 1, and constructing a collection card license plate recognition network training set;
and step 3: and (3) constructing a truck collection license plate detection network, taking the truck collection license plate detection training set in the step (2) as input data, constructing a truck collection license plate detection network loss function, and training through a gradient descent algorithm to obtain the optimized truck collection license plate detection network. Constructing a truck collection license plate recognition network, taking the truck collection license plate recognition training set in the step 2 as input data, constructing a truck collection license plate recognition network loss function, and obtaining the optimized truck collection license plate recognition network through gradient descent algorithm training;
and 4, step 4: inputting the image to be detected into the optimized truck license plate detection network, predicting to obtain a low-resolution prediction characteristic diagram, a medium-resolution prediction characteristic diagram and a high-resolution prediction characteristic diagram of the image to be detected, splicing the low-resolution prediction characteristic diagram, the medium-resolution prediction characteristic diagram and the high-resolution prediction characteristic diagram of the image to be detected to obtain a primary detection result of the image to be detected, and obtaining a final detection result of the image to be detected after confidence screening. The method comprises the steps of performing transmission transformation on an image to be detected to obtain a corrected image to be recognized, inputting the image to be recognized into an optimized truck license plate recognition network, predicting to obtain a primary recognition result of the image to be recognized, and obtaining a final recognition result of the image to be recognized after operations such as confidence screening, greedy algorithm decoding and the like;
preferably, the truck head image dataset in step 1 is:
{trains(m，n)，s∈[1，S]，m∈[1，M]，n∈[1，N]}
wherein, trains(m, n) represents the s-th locomotive in the truck locomotive image data setThe pixel information of the mth row and the nth column of the image, S represents the number of all image samples in the locomotive image data set, M is the row number of each locomotive image in the truck locomotive image data set, and N is the column number of each locomotive image in the truck locomotive image data set;
preferably, the collection card license plate image data set in step 1 is:
{trainp(m，n)，p∈[1，P]，m∈[1，M]，n∈[1，N]}
wherein trainp(M, N) pixel information of the mth row and nth column of the pth head image in the card collection license plate image data set is represented, P represents the number of all image samples in the card collection license plate image data set, M is the row number of each license plate image in the card collection license plate image data set, and N is the column number of each license plate image in the card collection license plate image data set;
preferably, in the step 2, the license plate mark frame coordinates of each vehicle head image in the truck head image data set are as follows:
Figure BDA0003069063460000031
Figure BDA0003069063460000032
Figure BDA0003069063460000033
Figure BDA0003069063460000034
Figure BDA0003069063460000035
where l represents the left on the truck head image, t represents the top on the truck head image, r represents the right on the truck head image, b represents the top on the truck head imageThe following steps (1); s represents the number of all the truck head images in the truck head image dataset, KsRepresenting the total number of the truck board mark boxes in the truck head image data set; boxs，kThe coordinates of the kth license plate mark frame in the sth truck head image in the truck head image data set are shown,
Figure BDA0003069063460000036
the coordinates of the upper left corner of the kth license plate marking frame in the sth truck head image in the truck head image data set are represented,
Figure BDA0003069063460000037
the abscissa representing the upper left corner of the kth license plate mark frame in the sth truck head image data set,
Figure BDA0003069063460000038
the ordinate of the top left corner of the kth license plate marking frame in the sth truck head image data set is represented;
Figure BDA0003069063460000039
the coordinates of the top right corner of the kth license plate marking frame in the sth truck head image in the truck head image data set are represented,
Figure BDA00030690634600000310
the abscissa representing the upper right corner of the kth license plate mark frame in the sth truck head image data set,
Figure BDA00030690634600000311
the ordinate of the top right corner of the kth license plate mark frame in the sth truck head image data set is represented;
Figure BDA00030690634600000312
the coordinates of the lower right corner of the kth license plate mark frame in the sth truck head image in the truck head image data set are represented,
Figure BDA00030690634600000313
the abscissa representing the lower right corner of the kth license plate mark frame in the s-th truck head image data set,
Figure BDA00030690634600000314
the ordinate represents the lower right corner of the kth license plate mark frame in the sth truck head image data set;
preferably, the marked license plate number of each license plate image in the collection card license plate image data set in the step 2 is as follows:
Figure BDA00030690634600000315
wherein S represents the total number of the card collecting license plates in the card collecting license plate image data set; textsThe license plate number text information of the s-th album card license plate image in the album card license plate image data set,
Figure BDA00030690634600000316
the 1 st license plate number character of the sth collection card license plate image in the collection card license plate image data set is represented;
Figure BDA0003069063460000041
2 nd license plate number character of the sth album license plate image in the album license plate image data set;
Figure BDA0003069063460000042
a 3 rd license plate number character representing the sth license plate image of the card collecting license plate image data set;
Figure BDA0003069063460000043
the 4 th license plate number character of the sth album license plate image in the album license plate image data set is represented;
Figure BDA0003069063460000044
the 5 th license plate number character of the sth album license plate image in the album license plate image data set is represented;
Figure BDA0003069063460000045
the 6 th license plate number character of the sth album license plate image in the album license plate image data set is represented;
Figure BDA0003069063460000046
the 7 th license plate number character of the sth album license plate image in the album license plate image data set is represented;
step 2, the collection card license plate detection network training set is as follows:
{trains(m，n)，boxs，k}，s∈[1，S]，m∈[1，M]，n∈[1，N]，k∈[1，Ks]
wherein, trains(m, n) represents the pixel information of the mth row and the nth column of the mth truck head image in the truck collecting license plate detection network training set, boxs，kRepresenting the coordinates of a kth license plate marking frame in the sth truck head image in the truck collection license plate detection network training set; s represents the number of all image samples in the truck collection license plate detection network training set, M is the number of lines of each locomotive image in the truck collection license plate detection network training set, and N is the number of columns of each locomotive image in the truck collection license plate detection network training set;
step 2, the collection card license plate recognition network training set is as follows:
(trains(m，n)，texts}，s∈[1，S]，m∈[1，M]，n∈[1，N]
wherein, trains(m, n) represents pixel information of the mth row and nth column of the mth album card license plate image in the album card license plate recognition network training set, textsRepresenting the license plate number of the No. s card collecting license plate image in the card collecting license plate recognition network training set; s represents the number of all image samples in the truck collection license plate detection network training set, M is the number of lines of each locomotive image in the truck collection license plate detection network training set, and N is the number of columns of each locomotive image in the truck collection license plate detection network training set;
preferably, the card collection and license plate detection network in step 3 specifically includes: the system comprises a feature extraction network, a multi-scale up-sampling network, a semantic segmentation network and a weighted prediction layer;
the feature fusion network is serially cascaded with the multi-scale up-sampling network, and the semantic segmentation network is embedded into the multi-scale up-sampling network as a sub-module; the multi-scale upsampling network is serially cascaded with the weighted prediction layer;
the feature extraction network: sequentially stacking and cascading a high-definition convolution module and a feature fusion module;
the high-definition convolution module is formed by sequentially and crossly connecting a plurality of Firemodule modules;
the Firemodule module is formed by sequentially stacking and cascading a compression convolutional layer, an expansion batch normalization layer and a ReLU activation layer;
the characteristic fusion module is formed by sequentially stacking and cascading a down-sampling convolution layer, a down-sampling batch normalization layer and a ReLU activation layer;
the feature extraction network is defined as:
Figure BDA0003069063460000051
a1∈[1，NG]，a2∈[1，NR]，a3∈[1，NF]
b1∈[1，NHR1]，b2∈[1，NHR2]，b3∈[1，NHR3]，b4∈[1，NHR4]，b5∈[1，NHR5]
wherein N isGRepresenting the number of high definition convolution modules in a feature extraction network, NRRepresenting the number of feature fusion modules in a feature extraction network, NFRepresenting the number of Firemodule modules in each high-definition convolution module in the feature extraction network, NHR1Representing the number of layers of the compressed convolutional layer in each FireModule Module, NHR2Indicates the number of layers, N, of expansion convolution layers in each FireModule ModuleHR3Number of layers, N, representing expanded batch normalization layer in each FireModule ModuleHR4Representing the number of downsampled convolutional layers in each feature fusion module, NHR5Representing downsampled batch regression in each feature fusion moduleThe number of layers of a normalization layer;
Figure BDA0003069063460000052
representing the parameters in the b1 th compressed convolutional layer in the a1 th high-definition convolutional module as the parameters to be optimized;
Figure BDA0003069063460000053
representing the parameters in the b2 expansion convolutional layer in the a1 high-definition convolutional module as the parameters to be optimized;
Figure BDA0003069063460000054
representing the translation amount of a b3 expansion batch normalization layer in an a1 high-definition convolution module as a parameter to be optimized;
Figure BDA0003069063460000055
representing the scaling amount of a b3 th expansion batch normalization layer in an a1 th high-definition convolution module as a parameter to be optimized;
Figure BDA0003069063460000056
representing the parameters in the b4 down-sampling convolutional layer in the a2 th feature fusion module as the parameters to be optimized;
Figure BDA0003069063460000057
representing the translation amount of a b5 down-sampling batch normalization layer in an a2 th feature fusion module as a parameter to be optimized;
Figure BDA0003069063460000058
representing the scaling quantity of a b5 down-sampling batch normalization layer in an a2 th feature fusion module as a parameter to be optimized;
the input data of the feature extraction network is a single image in the truck collection license plate detection network training set in the step 2, and the output data is a feature map, namely Feat1 (M)1×N1×C1)；
In the output data of the feature extraction network, M1Is a width, N, of the feature map Feat11Is a characteristic height, C, of Feat11The number of channels for feature map Feat 1;
the multi-scale up-sampling network: sequentially stacking and cascading an upsampling convolutional layer, an upsampling batch layer, an upsampling deconvolution layer and a ReLU active layer;
Figure BDA0003069063460000059
wherein,
Figure BDA0003069063460000061
representing the number of layers of the upsampled convolutional layer in the multi-scale upsampling network;
Figure BDA0003069063460000062
the number of layers of an upsampled batch layer in the multi-scale upsampling network is represented;
Figure BDA0003069063460000063
representing the number of layers of an upsampled deconvolution layer in a multi-scale upsampling network; UP _ kernelcRepresenting parameters in a c layer up-sampling convolution layer in the multi-scale up-sampling network, wherein the parameters are to-be-optimized parameters; UP _ GammadRepresenting the translation amount of a sampling batch normalization layer on the d-th layer in the multi-scale up-sampling network, and taking the translation amount as a parameter to be optimized; UP-betadRepresenting the scaling quantity of a sampling batch normalization layer on the d-th layer in the multi-scale up-sampling network, and taking the scaling quantity as a parameter to be optimized; UP _ dkerreleRepresenting parameters in an up-sampling deconvolution layer at the e-th layer in a multi-scale up-sampling network as parameters to be optimized;
the input data for the multiscale upsampling network is a feature map, Feat1, and the output data is a low resolution feature map, Feat2 (M)2×N2×C2) The medium resolution characteristic map is Feat3 (M)3×N3×C3) High resolution feature map, i.e., Feat4 (M)4×N4×C4)；
M of output data of the multi-scale up-sampling network2For the width, N, of the low resolution feature map Feat22High for low resolution feature map Feat2Degree C2The number of channels for the low resolution feature map, Feat 2; m3For the width, N, of the medium resolution feature Feat33Height, C, of medium resolution feature Feat33The number of channels is the middle resolution feature map Feat 3; m4For the width, N, of the high resolution feature map Feat44Height, C, of high resolution feature map Feat44Number of channels for high resolution feature map Feat 4;
the semantic segmentation network: the device is formed by sequentially stacking and cascading a segmentation convolution layer, a segmentation batch normalization layer and a ReLU activation layer;
the semantic segmentation network is defined as:
fmask(MASK_kernelf，MASK-γg，MASK_βg)
Figure BDA0003069063460000066
wherein,
Figure BDA0003069063460000064
indicating the number of layers of the segmented convolutional layers in the semantic segmentation network,
Figure BDA0003069063460000065
representing the number of layers of a segmentation batch normalization layer in the semantic segmentation network; MASK _ kernelfRepresenting the parameter in the f-th segmentation convolution layer in the semantic segmentation network as the parameter to be optimized; MASK-gammagRepresenting the translation amount of the g-th segmentation batch normalization layer in the semantic segmentation network, wherein the translation amount is a parameter to be optimized; MASK _ betagRepresenting the scaling quantity of the g-th segmentation batch normalization layer in the semantic segmentation network as a parameter to be optimized;
the input data of the semantic segmentation network is a low resolution feature map, i.e., Feat2, a medium resolution feature map, i.e., Feat3, and a high resolution feature map, i.e., Feat4, and the output data is a low resolution segmentation feature map, i.e., Feat5 (M)5，N5，C5) The medium resolution segmentation feature map is Feat6 (M)6，N6，C6) High resolution of the light beamCutting the characteristic diagram, i.e. Feat7 (M)7，N7，C7)；
In the output data of the semantic segmentation network, M5Segmenting the width, N, of the feature map Feat5 for low resolution5For low resolution segmentation of the height, C, of the feature map Feat55The number of channels for the low resolution segmented feature map, Feat 5; m is a group of6For medium resolution segmentation of the width, N, of the feature map Feat66Is the height, C, of the medium resolution segmentation feature map Feat66The number of channels of the feature map Feat6 is divided with medium resolution; m7Segmenting the width, N, of the feature map Feat7 for high resolution7For high resolution segmentation of the height, C, of the feature map Feat77The number of channels of the high-resolution segmentation feature map Feat 7;
the weighted prediction layer: the system is formed by sequentially stacking and cascading a prediction convolution layer, a maximum pooling layer, a ReLU activation layer and a Sigmoid activation layer;
the weighted prediction layer is defined as:
fWE(WE_kernelh)，h∈[1，NWE]
wherein N isWERepresenting the number of predicted convolutional layers in the weighted prediction layer; WE _ kernelhRepresenting the parameters in the h prediction convolution layer in the weighted prediction layer as the parameters to be optimized;
the input data of the weighted prediction layer is Feat2, which is a low resolution feature map, Feat3, Feat4, Feat5, which is a low resolution segmentation feature map, Feat6, Feat7, which is a medium resolution segmentation feature map, and the output data is Feat8, which is a low resolution prediction feature map (M)8，N8，C8) Feat9 (M), which is a medium resolution prediction feature map9，N9，C9) High resolution predictive feature map, Feat10 (M)10，N10，C10)；
Ma is the width of the low resolution prediction characteristic map Feat8, N, in the output data of the weighted prediction layer8Predicting the height, C, of the feature map Feat8 for low resolution8Predicting the number of channels of the feature map Feat8 for the low resolution; m9Predicting the width, N, of feature map Feat9 for medium resolution9Predicting the height, C, of feature map Feat9 for medium resolution9Predicting the number of channels of the feature map Feat9 for medium resolution; m is a group of10Predicting the width, N, of the feature map Feat10 for high resolution10Predicting the height, C, of the feature map Feat10 for high resolution10Predicting the number of channels of the feature map Feat10 for high resolution;
preferably, the card collection license plate recognition network in step 3 specifically includes: dense feature extraction network, sequence feature extraction network;
the dense feature extraction network and the sequence feature extraction network are in serial cascade connection;
the dense feature extraction network is formed by sequentially stacking and cascading a plurality of dense convolution modules;
the dense convolution module is formed by sequentially stacking and cascading a dense convolution layer, a dense batch normalization layer and a Mish activation layer;
the dense feature extraction network is defined as:
Figure BDA0003069063460000083
wherein,
Figure BDA0003069063460000084
representing the number of layers of the dense convolution layer in the dense feature extraction network;
Figure BDA0003069063460000085
representing the number of layers of a dense batch normalization layer in a dense feature extraction network; DENSE _ kerneliRepresenting the parameters in the ith dense convolution layer in the dense feature extraction network as the parameters to be optimized; DENSE _ GammajRepresenting the translation quantity of the jth dense batch normalization layer in the dense feature extraction network, and taking the translation quantity as a parameter to be optimized; DENSE _ betajRepresenting the scaling of the jth dense batch normalization layer in the dense feature extraction network as a parameter to be optimized;
the input data of the dense feature extraction network is the result of step 2The output data of a single image of the truck license plate recognition network training set is a dense feature map, namely Feat11 (M)11×N11×C11)；
In the output data of the dense feature extraction network, M11For the width, N, of the dense feature map Feat1111Height, C, of dense feature map Feat1111Number of channels being dense feature map Feat 11;
the sequence feature extraction network: the system is formed by sequentially stacking a bidirectional LSTM recursive layer, a sequence batch normalization layer and a ReLU activation layer;
the sequence feature extraction network is defined as:
Figure BDA0003069063460000081
wherein,
Figure BDA0003069063460000086
representing the number of layers of a bidirectional LSTM recursive layer in a sequence feature extraction network;
Figure BDA0003069063460000082
representing the number of layers of a sequence batch normalization layer in a sequence feature extraction network; SEQ _ kernelxRepresenting the parameters in the x-th bidirectional LSTM recursive layer in the sequence feature extraction network, and taking the parameters as the parameters to be optimized; SEQ _ gammayRepresenting the translation amount of a y-th layer sequence batch normalization layer in the sequence feature extraction network, wherein the translation amount is a parameter to be optimized; SEQ _ betayRepresenting the scaling quantity of a y-th layer sequence batch normalization layer in the sequence feature extraction network as a parameter to be optimized;
the input data of the sequence feature extraction network is a dense feature map, namely Feat11, and the output data is sequence features, namely Feat12 (L)12，C12)；
In the output data of the sequence feature extraction network, L12Sequence length being a characteristic of the sequence, C12The number of channels that are characteristic of the sequence.
And 3, constructing a truck license plate detection network loss function through the corner confidence coefficient loss function and the segmentation confidence coefficient loss function.
When an image of a truck head of the truck is input into a truck collection and license plate detection network for training, four circular areas are obtained by dividing the four corner points of a license plate by taking the four corner points as the circle center and taking r as the radius, weight distribution is obtained in the circular areas through two-dimensional Gaussian distribution, an H multiplied by W corner point thermodynamic diagram is obtained through network prediction of the input image, feature points falling in the four circles are selected as positive samples, and the rest points are all regarded as negative samples and used for learning feature information of the corner points of the license plate.
The corner confidence loss function is
Figure BDA0003069063460000091
Wherein, N represents the number of all prediction sample points;
Figure BDA0003069063460000092
representing the confidence coefficient of the corner point with x as the abscissa and y as the ordinate in the predicted corner point thermodynamic diagram; alpha represents a positive and negative sample balance parameter; y isxyRepresenting the weight of the corner point with x as the abscissa and y as the ordinate in the truck head image; beta represents a difficult and easy sample equilibrium parameter;
the segmentation confidence loss function is:
Figure BDA0003069063460000093
wherein, N represents the number of all prediction sample points;
Figure BDA0003069063460000094
representing pixel confidence coefficients with x as the abscissa and y as the ordinate in the prediction segmentation graph; alpha represents a positive and negative sample balance parameter; y isxyRepresenting the pixel weight with x as the abscissa and y as the ordinate in the truck head image; beta represents a difficult and easy sample equilibrium parameter;
the network loss function of the collection card license plate detection is as follows:
Ld＝Lk+Lm
wherein L iskAs a corner confidence loss function, LmIs a segmentation confidence loss function.
And 3, constructing a card collection license plate identification network loss function through the CTC.
The collection card license plate recognition network loss function is as follows:
Figure BDA0003069063460000101
and is
Figure BDA0003069063460000102
Wherein, p (l | y) represents the probability sum of all CTC sequences alpha of the predicted sequence l obtained by the beta mapping function under the condition that the prediction result is y; beta represents a mapping function that removes the repeated elements first and then the blank labels for the prediction result, so that beta-1Represents the inverse function of β; l istAnd the method shows that the beta of the prediction result is mapped into the entropy sum of x under the condition that the license plate text label is x and the collection card license plate recognition network prediction result is y.
Preferably, the low-resolution prediction feature map of the image to be detected in step 4 is Feat8；
Step 4, the medium resolution ratio prediction characteristic diagram of the image to be detected is Feat9；
Step 4, the high resolution prediction characteristic diagram of the image to be detected is Feat10；
And 4, taking the preliminary detection result of the image to be detected as the probability that the predicted point belongs to the foreground, and defining the probability as follows:
Ki∈[0，1]，i∈NDS
wherein N isDSRepresenting the number of predicted points, K, in the preliminary test result of the image to be testediThe predicted point in the preliminary detection result of the ith image to be detected belongs toProbability of a foreground;
the preliminary detection result of the image to be detected is defined as:
RDS＝{Ki}，i∈NDS
and the final detection result of the image to be detected is defined as:
RDE＝{Kε}，ε∈NDE
wherein N isDERepresenting the number of predicted points in the final detection result of the image to be detected, KεRepresenting the probability that the predicted point in the final detection result of the epsilon-th image to be detected belongs to the foreground;
preferably, the preliminary identification result of the image to be identified in step 4 is defined as:
RRS＝{Ti}，i∈NRS
wherein N isRSRepresenting the number of elements, T, of the predicted sequence in the preliminary recognition result of the image to be recognizediRepresenting the character category of the ith element in the prediction sequence in the preliminary recognition result of the image to be recognized;
and 4, defining the final identification result of the image to be identified as:
RRE＝{Tε}，ε∈NRE
wherein N isREAnd T epsilon represents the character category of the epsilon-th element in the prediction sequence in the final recognition result of the image to be recognized.
The invention has the following beneficial effects:
some problems possibly encountered in the actual recognition process are simulated by adopting a specific data enhancement mode for the image, for example, the problems of uneven ambient light, image distortion, motion blur caused by too fast vehicle speed and the like exist, the robustness of the recognition network is improved, and meanwhile, the over-fitting problem caused by few training samples can be avoided to a certain extent.
In the HRNet, a FireModule module is adopted, firstly, the characteristic diagram channels are compressed through an squeeze part, then, in a following expanded part, the characteristic diagram channels are respectively expanded through two convolution layers of 1 × 1 and 3 × 3, and finally, the two part characteristic diagrams are spliced. Under the condition of ensuring the network identification precision, the network parameters are greatly reduced, and the network operation speed is accelerated.
A Mask branch is introduced into the HRNet, so that a network model can learn the specific positions of four corner points of a license plate and the position of the whole license plate in the whole image, the Mask branch is used for limiting the range of the corner points when the positions of the corner points are predicted, and the error identification rate of corner point prediction can be obviously reduced under the condition of increasing the operation time of a micro algorithm.
The DenseNet feature extraction network is adopted in the CRNN algorithm, the utilization degree of features of each layer in the feature extraction process is improved in a dense connection mode, the input of each layer is formed by splicing the outputs of all the layers, meanwhile, the network parameters are far smaller than those of the traditional convolution network, and the overfitting problem caused by few training samples can be reduced to a certain degree.
The improved license plate recognition network can be deployed on an industrial computer with low hardware configuration, and can read the video streams of a plurality of network monitoring cameras in real time and recognize the hub license plate numbers at the passage and the gate on line. The card collecting license plate number is not required to be recorded one by workers, and any influence is not generated on the running process of the card collecting, so that the waste of human resources is reduced, and the automation level of the container terminal is greatly improved.
Drawings
FIG. 1: the invention relates to a network structure diagram for detecting a license plate of a collection card;
FIG. 2: the invention is a flow chart of a training collection card license plate detection network;
FIG. 3: the invention relates to a structure diagram of a card collecting license plate recognition network;
FIG. 4: the invention is a flow chart of a license plate recognition network of a training collection card;
FIG. 5: the invention relates to an execution flow chart of an automatic identification method of a card collection license plate;
FIG. 6: the invention relates to an image example which can be identified by the automatic identification method of the plate number of the collection card.
Detailed Description
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
The specific embodiment of the invention is described below with reference to fig. 1-6, which is a multipath automatic identification method for container terminal truck-collecting license plates, comprising the following steps:
step 1: a network monitoring camera is used for shooting high-resolution images of the front side or the front side of the truck collection under the condition that the truck collection does not stop at a container terminal gate, a crossing or a yard passageway and the like to construct a truck collection head image data set, and the high-resolution images of the truck collection head are reduced to a proper proportion by an equal proportion through a double-cubic interpolation method to construct the truck collection head image data set. Obtaining a corrected license plate image from the high-resolution image of the head of the truck by using a transmission transformation method, and reducing the truck collection license plate image to a proper proportion by using a double-cubic interpolation method in an equal proportion to construct a truck collection license plate image data set;
step 1, the truck head image dataset of the container truck is:
{trains(m，n)，s∈[1，S]，m∈[1，M]，n∈[1，N]}
wherein, trains(M, N) represents the pixel information of the mth row and nth column of the mth head image in the truck head image dataset, wherein S is 2329 represents the number of all image samples in the head image dataset, M is 512 is the row number of each head image in the truck head image dataset, and N is 512 is the column number of each head image in the truck head image dataset;
step 1, the collection card license plate image data set is as follows:
{trainp(m，n)，p∈[1，P]，m∈[1，M]，n∈[1，N]}
wherein trainp(m, n) represents the pixel information of the mth row and nth column of the pth head image in the truck collection license plate image data set, and P-2329 represents the number of all image samples in the truck collection license plate image data setM is 32 lines of each license plate image in the card collection license plate image data set, and N is 100 lines of each license plate image in the card collection license plate image data set;
step 2: and (3) manually labeling the license plate mark frame of each truck head image in the truck head image data set in the step 1, and constructing a truck license plate detection network training set. Manually labeling license plate number information of each license plate image in the collection card license plate image data set in the step 1, and constructing a collection card license plate recognition network training set;
step 2, the license plate marking frame coordinates of each vehicle head image in the truck head image data set are as follows:
Figure BDA0003069063460000121
Figure BDA0003069063460000131
Figure BDA0003069063460000132
Figure BDA0003069063460000133
Figure BDA0003069063460000134
wherein l represents the left on the truck head image, t represents the top on the truck head image, r represents the right on the truck head image, and b represents the bottom on the truck head image; s2329 denotes the number of all the truck head images in the truck head image dataset, KsRepresenting the total number of the truck board mark boxes in the truck head image data set; boxs，kThe coordinates of the kth license plate mark frame in the sth truck head image in the truck head image data set are shown,
Figure BDA0003069063460000135
the coordinates of the upper left corner of the kth license plate marking frame in the sth truck head image in the truck head image data set are shown,
Figure BDA0003069063460000136
the abscissa representing the upper left corner of the kth license plate mark frame in the sth truck head image data set,
Figure BDA0003069063460000137
the ordinate of the top left corner of the kth license plate marking frame in the sth truck head image data set is represented;
Figure BDA0003069063460000138
the coordinates of the top right corner of the kth license plate marking frame in the sth truck head image in the truck head image data set are represented,
Figure BDA0003069063460000139
the abscissa representing the upper right corner of the kth license plate mark frame in the sth truck head image data set,
Figure BDA00030690634600001310
the ordinate of the top right corner of the kth license plate mark frame in the sth truck head image data set is represented;
Figure BDA00030690634600001311
the coordinates of the lower right corner of the kth license plate mark frame in the sth truck head image in the truck head image data set are represented,
Figure BDA00030690634600001312
the abscissa representing the lower right corner of the kth license plate mark frame in the s-th truck head image data set,
Figure BDA00030690634600001313
representing number of truck head imagesThe vertical coordinate of the lower right corner of a kth license plate marking frame in the sth truck head image in the data set;
step 2, the marked license plate number of each license plate image in the card collection license plate image data set is as follows:
Figure BDA00030690634600001314
wherein, S-2329 represents the total number of the license plates of the collection card in the image data set of the license plates of the collection card; texts represents the license plate number text information of the s-th album license plate image in the album license plate image data set,
Figure BDA00030690634600001315
the 1 st license plate number character of the sth collection card license plate image in the collection card license plate image data set is represented;
Figure BDA00030690634600001316
2 nd license plate number character of the sth album license plate image in the album license plate image data set;
Figure BDA00030690634600001317
the 3 rd license plate number character of the sth album license plate image in the album license plate image data set is represented;
Figure BDA0003069063460000141
the 4 th license plate number character of the sth album license plate image in the album license plate image data set is represented;
Figure BDA0003069063460000142
the 5 th license plate number character of the sth album license plate image in the album license plate image data set is represented;
Figure BDA0003069063460000143
the 6 th license plate number character of the sth album license plate image in the album license plate image data set is represented;
Figure BDA0003069063460000144
the 7 th license plate number character of the sth album license plate image in the album license plate image data set is represented;
step 2, the collection card license plate detection network training set is as follows:
{trains(m，n)，boxs，k}，s∈[1，S]，m∈[1，M]，n∈[1，N]，k∈[1，Ks]
wherein, trains(m, n) represents pixel information of the mth row and nth column of the mth truck head image in the truck collection license plate detection network training set, boxs，kRepresenting the coordinates of a kth license plate marking frame in the sth truck head image in the truck collection license plate detection network training set; s is 2329 the number of all image samples in the truck collection license plate detection network training set, M is 512 the number of lines of each locomotive image in the truck collection license plate detection network training set, and N is 512 the number of columns of each locomotive image in the truck collection license plate detection network training set;
step 2, the collection card license plate recognition network training set is as follows:
{trains(m，n)，texts}，s∈[1，S]，m∈[1，M]，n∈[1，N]
wherein, trains(m, n) represents pixel information of the mth row and nth column of the mth album card license plate image in the album card license plate recognition network training set, textsRepresenting the license plate number of the No. s card collecting license plate image in the card collecting license plate recognition network training set; s is 2329 the number of all image samples in the truck collection license plate detection network training set, M is 32 the number of lines of each locomotive image in the truck collection license plate detection network training set, and N is 100 the number of columns of each locomotive image in the truck collection license plate detection network training set;
and step 3: and (3) constructing a truck collection license plate detection network, wherein the structure of the truck collection license plate detection network is shown in fig. 1, the truck collection license plate detection training set in the step (2) is used as input data, a truck collection license plate detection network loss function is constructed, the optimized truck collection license plate detection network is obtained through gradient descent algorithm training, and the training flow of the truck collection license plate detection network is shown in fig. 2. Constructing a truck-mounted license plate recognition network, wherein the structure diagram of the truck-mounted license plate recognition network is shown in FIG. 3, the truck-mounted license plate recognition training set in the step 2 is used as input data, a truck-mounted license plate recognition network loss function is constructed, an optimized truck-mounted license plate recognition network is obtained through gradient descent algorithm training, and the training flow of the truck-mounted license plate recognition network is shown in FIG. 4;
step 3, the collection card license plate detection network specifically comprises: the system comprises a feature extraction network, a multi-scale up-sampling network, a semantic segmentation network and a weighted prediction layer;
the feature fusion network is serially cascaded with the multi-scale up-sampling network, and the semantic segmentation network is embedded into the multi-scale up-sampling network as a sub-module; the multi-scale up-sampling network is serially cascaded with the weighted prediction layer;
the feature extraction network: sequentially stacking and cascading a high-definition convolution module and a feature fusion module;
the high-definition convolution module is formed by sequentially and crossly connecting a plurality of Firemodule modules;
the Firemodule module is formed by sequentially stacking and cascading a compression convolutional layer, an expansion batch normalization layer and a ReLU activation layer;
the characteristic fusion module is formed by sequentially stacking and cascading a down-sampling convolution layer, a down-sampling batch normalization layer and a ReLU activation layer;
the feature extraction network is defined as:
Figure BDA0003069063460000151
a1∈[1，NG]，a2∈[1，NR]，a3∈[1，NF]
b1∈[1，NHR1]，b2∈[1，NHR2]，b3∈[1，NHR3]，b4∈[1，NHR4]，b5∈[1，NHR5]
wherein, NGRepresenting the number of high definition convolution modules in a feature extraction network, NRRepresenting the number of feature fusion modules in a feature extraction network, NFRepresenting the number of Firemodule modules in each high-definition convolution module in the feature extraction network, NHR1Representing the number of layers of the compressed convolutional layer in each FireModule Module, NHR2Indicates the number of layers, N, of expansion convolution layers in each FireModule ModuleHR3Number of layers, N, representing expanded batch normalization layer in each FireModule ModuleHR4Representing the number of layers, N, of the downsampled convolutional layer in each feature fusion moduleHR5Representing the number of layers of the downsampling batch normalization layer in each feature fusion module;
Figure BDA0003069063460000152
representing the parameters in the b1 compressed convolutional layer in the a1 high-definition convolutional module as the parameters to be optimized;
Figure BDA0003069063460000153
representing the parameters in the b2 expansion convolutional layer in the a1 high-definition convolutional module as the parameters to be optimized;
Figure BDA0003069063460000154
representing the translation amount of a b3 expansion batch normalization layer in an a1 high-definition convolution module as a parameter to be optimized;
Figure BDA0003069063460000155
setting the scaling quantity of the b3 expansion batch normalization layer in the a1 high-definition convolution module as a parameter to be optimized;
Figure BDA0003069063460000156
representing the parameters in the b4 down-sampling convolutional layer in the a2 th feature fusion module as the parameters to be optimized;
Figure BDA0003069063460000157
representing the translation amount of a b5 down-sampling batch normalization layer in an a2 th feature fusion module as a parameter to be optimized;
Figure BDA0003069063460000158
representing the scaling quantity of a b5 down-sampling batch normalization layer in an a2 th feature fusion module as a parameter to be optimized;
the input data of the feature extraction network is a single image in the truck collection license plate detection network training set in the step 2, and the output data is a feature map, namely Feat1 (M)1×N1×C1)；
In the output data of the feature extraction network, M1128 is the width of the feature map Feat1, N1128 is the height of the feature map Feat1, C196 is the number of channels of the feature map Feat 1;
the multi-scale up-sampling network: sequentially stacking and cascading an upsampling convolutional layer, an upsampling batch layer, an upsampling deconvolution layer and a ReLU active layer;
Figure BDA0003069063460000161
wherein,
Figure BDA0003069063460000162
representing the number of layers of the upsampled convolutional layer in the multi-scale upsampling network;
Figure BDA0003069063460000163
the number of layers of an upsampled batch layer in the multi-scale upsampling network is represented;
Figure BDA0003069063460000164
representing the number of layers of the up-sampling deconvolution layer in the multi-scale up-sampling network; UP _ kernelcRepresenting parameters in a c layer up-sampling convolution layer in the multi-scale up-sampling network, wherein the parameters are to-be-optimized parameters; UP _ GammadRepresenting the translation amount of a sampling batch normalization layer on the d-th layer in the multi-scale up-sampling network, and taking the translation amount as a parameter to be optimized; UP-betadRepresenting the scaling quantity of a sampling batch normalization layer on the d-th layer in the multi-scale up-sampling network, and taking the scaling quantity as a parameter to be optimized; UP _ dkerreleRepresenting parameters in an up-sampling deconvolution layer at the e-th layer in a multi-scale up-sampling network as parameters to be optimized;
the input data of the multi-scale up-sampling network is a feature map Feat1, and the output data is a low resolution feature map, namely Feat2(M2×N2×C2) The medium resolution characteristic map is Feat3 (M)3×N3×C3) High resolution feature map, i.e., Feat4 (M)4×N4×C4)；
M of output data of the multi-scale up-sampling network2128 is the width of the low resolution feature map, Feat2, N2128 is the height, C, of the low resolution feature map, Feat2232 is the number of channels of the low resolution feature map Feat 2; m is a group of3256 is the width of the medium resolution feature map, Feat3, N3256 is the height of the medium resolution feature map, Feat3, C332 is the number of channels of the medium resolution feature map Feat 3; m is a group of4512 is the width of the high resolution feature map, Feat4, N4512 is the height of the high resolution feature map, Feat4, C432 is the number of channels of the high resolution feature map Feat 4;
the semantic segmentation network: the device is formed by sequentially stacking and cascading a segmentation convolution layer, a segmentation batch normalization layer and a ReLU activation layer;
the semantic segmentation network is defined as:
fmask(MASK_kernelf，MASK-γg，MASK_βg)
Figure BDA0003069063460000165
wherein,
Figure BDA0003069063460000171
indicating the number of layers of the partitioned convolutional layers in the semantic partitioning network,
Figure BDA0003069063460000172
representing the number of layers of a segmentation batch normalization layer in the semantic segmentation network; MASK _ kernelfRepresenting the parameter in the f-th segmentation convolution layer in the semantic segmentation network as the parameter to be optimized; MASK _ GammagRepresenting the translation amount of the g-th segmentation batch normalization layer in the semantic segmentation network, wherein the translation amount is a parameter to be optimized; MASK _ betagRepresenting semantic componentsThe scaling quantity of the g-th segmentation batch normalization layer in the segmentation network is a parameter to be optimized;
the input data of the semantic segmentation network is a low resolution feature map, i.e., Feat2, a medium resolution feature map, i.e., Feat3, and a high resolution feature map, i.e., Feat4, and the output data is a low resolution segmentation feature map, i.e., Feat5 (M)5，N5，C5) The medium resolution segmentation feature map is Feat6 (M)6，N6，C6) High resolution segmentation feature map, i.e., Feat7 (M)7，N7，C7)；
In the output data of the semantic segmentation network, M5128 is the width of the low resolution segmented feature map, Feat5, N5128 is the height, C, of the low resolution segmented feature map, Feat551 is the channel number of the low resolution segmentation feature map Feat 5; m6256 is the width of the medium resolution segmentation feature map Feat6, N6256 is the height of the medium resolution segmentation feature map, Feat6, C61 is the number of channels of the medium resolution segmentation feature map Feat 6; m7512 is the width of the high resolution segmentation feature map Feat7, N7512 is the height of the high resolution segmentation feature map, Feat7, C71 is the channel number of the high-resolution segmentation feature map Feat 7;
the weighted prediction layer: the system is formed by sequentially stacking and cascading a prediction convolution layer, a maximum pooling layer, a ReLU activation layer and a Sigmoid activation layer;
the weighted prediction layer is defined as:
fWE(WE_kernelh)，h∈[1，NWE]
wherein N isWERepresenting the number of predicted convolutional layers in the weighted prediction layer; WE _ kernelhRepresenting the parameters in the h prediction convolution layer in the weighted prediction layer as the parameters to be optimized;
the input data of the weighted prediction layer is Feat2 which is a low resolution feature map, Feat3 which is a medium resolution feature map, Feat4 which is a high resolution feature map, Feat5 which is a low resolution segmentation feature map, Feat6 which is a medium resolution segmentation feature map, and Feat7 which is a high resolution segmentation feature map, and the output data is a low resolution prediction feature mapNamely Feat8 (M)8，N8，C8) Feat9 (M), which is a medium resolution prediction feature map9，N9，C9) High resolution predictive feature map, Feat10 (M)10，N10，C10)；
In the output data of the weighted prediction layer, M8128 is the width, N, of the low resolution predicted feature map, Feat88128 is the height, C, of the low resolution predicted feature map, Feat884 is the number of channels of the low resolution prediction feature map Feat 8; m9256 is the width, N, of the medium resolution prediction feature map Feat99256 is the height, C, of the medium resolution prediction feature map Feat994 is the channel number of the medium resolution prediction feature map Feat 9; m10512 is the width, N, of the high resolution predicted feature map Feat1010512 is the height of the high resolution predicted feature map, Feat10, C104 is the channel number of the high-resolution prediction feature map Feat 10;
preferably, the card collection license plate recognition network in step 3 specifically includes: dense feature extraction network, sequence feature extraction network;
the dense feature extraction network and the sequence feature extraction network are in serial cascade connection;
the dense feature extraction network is formed by sequentially stacking and cascading a plurality of dense convolution modules;
the dense convolution module is formed by sequentially stacking and cascading a dense convolution layer, a dense batch normalization layer and a Mish activation layer;
the dense feature extraction network is defined as:
Figure BDA0003069063460000184
wherein,
Figure BDA0003069063460000185
representing the number of layers of the dense convolution layer in the dense feature extraction network;
Figure BDA0003069063460000186
representing the number of layers of a dense batch normalization layer in a dense feature extraction network; DENSE _ kerneliRepresenting the parameters in the ith dense convolution layer in the dense feature extraction network as the parameters to be optimized; DENSE _ GammajRepresenting the translation quantity of the jth dense batch normalization layer in the dense feature extraction network, and taking the translation quantity as a parameter to be optimized; DENSE _ betajRepresenting the scaling quantity of the jth dense batch normalization layer in the dense feature extraction network as a parameter to be optimized;
the input data of the dense feature extraction network is a single image of the truck license plate recognition network training set in the step 2, and the output data is a dense feature map, namely Feat11 (M)11×N11×C11)；
In the output data of the dense feature extraction network, M1113 is the width of the dense feature map Feat11, N111 is the height of the dense feature map, Feat11, C11512 is the number of channels of the dense feature map Feat 11;
the sequence feature extraction network: the system is formed by sequentially stacking a bidirectional LSTM recursive layer, a sequence batch normalization layer and a ReLU activation layer;
the sequence feature extraction network is defined as:
Figure BDA0003069063460000181
wherein,
Figure BDA0003069063460000182
representing the number of layers of a bidirectional LSTM recursive layer in a sequence feature extraction network;
Figure BDA0003069063460000183
representing the number of layers of a sequence batch normalization layer in a sequence feature extraction network; SEQ _ kernelxRepresenting the parameters in the x-th bidirectional LSTM recursive layer in the sequence feature extraction network, and taking the parameters as the parameters to be optimized; sEQ _ gammayRepresenting the translation amount of the y-th layer sequence batch normalization layer in the sequence feature extraction network as the parameter to be optimized；SEQ_βyRepresenting the scaling quantity of a y-th layer sequence batch normalization layer in the sequence feature extraction network as a parameter to be optimized;
the input data of the sequence feature extraction network is a dense feature map, namely Feat11, and the output data is sequence features, namely Feat12 (L)12，C12)；
In the output data of the sequence feature extraction network, L12Sequence length characterized by sequence number 13, C12The number of channels for the sequence feature is 45.
And 3, constructing a truck license plate detection network loss function through the corner confidence coefficient loss function and the segmentation confidence coefficient loss function.
When an image of a truck head of a truck is input into a truck collection license plate detection network for training, four circular areas are obtained by dividing the four corner points of a license plate by taking r as the center of a circle and taking r as 2 as the radius, weight distribution is obtained in the circular areas through two-dimensional Gaussian distribution, an H multiplied by W (H is W is 512) corner point thermodynamic diagram is obtained by the input image through network prediction, feature points falling in the four circles are selected as positive samples, and the rest points are all regarded as negative samples and used for learning feature information of the corner points of the license plate.
The corner confidence loss function is
Figure BDA0003069063460000191
Wherein N ═ 512 × 512 × 4 ═ 1048576 indicates the number of all prediction sample points;
Figure BDA0003069063460000192
representing the confidence coefficient of the corner point with x as the abscissa and y as the ordinate in the predicted corner point thermodynamic diagram; α -0.25 represents a positive and negative sample balance parameter; y isxyRepresenting the weight of the corner point with x as the abscissa and y as the ordinate in the truck head image; β ═ 2 denotes the difficult and easy sample equilibrium parameter;
the segmentation confidence loss function is:
Figure BDA0003069063460000193
wherein N ═ 512 × 512 × 4 ═ 1048576 indicates the number of all prediction sample points;
Figure BDA0003069063460000194
representing pixel confidence coefficients with x as the abscissa and y as the ordinate in the prediction segmentation graph; α -0.25 represents a positive and negative sample balance parameter; y isxyRepresenting the pixel weight with x as the abscissa and y as the ordinate in the truck head image; β ═ 2 denotes the difficult and easy sample equilibrium parameter;
the network loss function of the collection card license plate detection is as follows:
Ld＝Lk+Lm
wherein L iskAs a corner confidence loss function, LmIs a segmentation confidence loss function.
And 3, constructing a card collection license plate identification network loss function through the CTC.
The collection card license plate recognition network loss function is as follows:
Figure BDA0003069063460000201
and is
Figure BDA0003069063460000202
Wherein, p (l | y) represents the probability sum of all CTC sequences alpha of the predicted sequence l obtained by the beta mapping function under the condition that the prediction result is y; beta represents a mapping function that removes the repeated elements first and then the blank labels for the prediction result, so beta-1Represents the inverse function of β; l istAnd the entropy sum of the prediction result beta is mapped into the entropy sum of x under the condition that the license plate text label is x and the collection card license plate recognition network prediction result is y.
And 4, step 4: inputting the image to be detected into the optimized truck license plate detection network, predicting to obtain a low-resolution prediction characteristic diagram, a medium-resolution prediction characteristic diagram and a high-resolution prediction characteristic diagram of the image to be detected, splicing the low-resolution prediction characteristic diagram, the medium-resolution prediction characteristic diagram and the high-resolution prediction characteristic diagram of the image to be detected to obtain a primary detection result of the image to be detected, and obtaining a final detection result of the image to be detected after confidence screening. And performing transmission transformation on the image to be detected to obtain a corrected image to be recognized, inputting the image to be recognized into an optimized truck license plate recognition network, predicting to obtain a primary recognition result of the image to be recognized, and performing operations such as confidence screening and greedy algorithm decoding to obtain a final recognition result of the image to be recognized. The flow chart of the implementation of the automatic identification method of the plate number of the collection card is shown in FIG. 5, and the image needing to be identified by the identification method is shown in FIG. 6;
step 4, the low resolution prediction characteristic diagram of the image to be detected is Feat8；
Step 4, the medium resolution ratio prediction characteristic diagram of the image to be detected is Feat9；
Step 4, the high resolution prediction characteristic diagram of the image to be detected is Feat10；
And 4, taking the preliminary detection result of the image to be detected as the probability that the predicted point belongs to the foreground, and defining the probability as follows:
Ki∈[0，1]，i∈NDS
wherein N isDSRepresenting the number of predicted points, K, in the preliminary test result of the image to be testediRepresenting the probability that the predicted point in the preliminary detection result of the ith image to be detected belongs to the foreground;
the preliminary detection result of the image to be detected is defined as:
RDS＝{Ki}，i∈NDS
and the final detection result of the image to be detected is defined as:
RDE＝{Kε}，ε∈NDE
wherein N isDERepresenting the number of predicted points in the final detection result of the image to be detected，KεRepresenting the probability that the predicted point in the final detection result of the epsilon-th image to be detected belongs to the foreground;
preferably, the preliminary identification result of the image to be identified in step 4 is defined as:
RRS＝{Ti}，i∈NRS
wherein N isRSRepresenting the number of elements, T, of the predicted sequence in the preliminary recognition result of the image to be recognizediRepresenting the character category of the ith element in the prediction sequence in the preliminary recognition result of the image to be recognized;
step 4, defining the final recognition result of the image to be recognized as:
RRE＝{Tε}，ε∈NRE
wherein N isREAnd T epsilon represents the character category of the epsilon-th element in the prediction sequence in the final recognition result of the image to be recognized.
The above-mentioned embodiments only express the embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (4)

1. A multipath automatic identification method for container terminal truck-collecting license plates is characterized by comprising the following steps:
step 1: the method comprises the steps that a network monitoring camera is used for shooting high-resolution images of the front side face or the front face of a truck container under the condition that the truck container does not stop at a container terminal gate, a container terminal intersection or a container yard passageway to construct a truck head image data set, and the high-resolution images of the truck head are reduced to a proper proportion by a double-cubic interpolation method in an equal proportion manner to construct the truck head image data set; obtaining a corrected license plate image from the high-resolution image of the truck head by using a transmission transformation method, and reducing the truck collection license plate image to a proper proportion by using a double-cubic interpolation method in an equal proportion to construct a truck collection license plate image data set;
step 2: manually labeling a license plate labeling frame of each truck head image in the truck head image data set of the step 1 to construct a truck license plate detection network training set; manually labeling license plate number information of each license plate image in the collection card license plate image data set in the step 1, and constructing a collection card license plate recognition network training set;
and step 3: constructing a truck collection license plate detection network, taking the truck collection license plate detection training set in the step 2 as input data, constructing a truck collection license plate detection network loss function, and training through a gradient descent algorithm to obtain an optimized truck collection license plate detection network; constructing a truck collection license plate recognition network, taking the truck collection license plate recognition training set in the step 2 as input data, constructing a truck collection license plate recognition network loss function, and obtaining the optimized truck collection license plate recognition network through gradient descent algorithm training;
and 4, step 4: inputting an image to be detected into an optimized truck license plate detection network, predicting to obtain a low-resolution prediction characteristic diagram, a medium-resolution prediction characteristic diagram and a high-resolution prediction characteristic diagram of the image to be detected, splicing the low-resolution prediction characteristic diagram, the medium-resolution prediction characteristic diagram and the high-resolution prediction characteristic diagram of the image to be detected to obtain a primary detection result of the image to be detected, and screening confidence coefficients to obtain a final detection result of the image to be detected; the method comprises the steps of performing transmission transformation on an image to be detected to obtain a corrected image to be recognized, inputting the image to be recognized into an optimized truck license plate recognition network, predicting to obtain a primary recognition result of the image to be recognized, and obtaining a final recognition result of the image to be recognized after confidence screening and greedy algorithm decoding operation;
step 3, the collection card license plate detection network specifically comprises: the system comprises a feature extraction network, a multi-scale up-sampling network, a semantic segmentation network and a weighted prediction layer;
the feature extraction network is serially cascaded with the multi-scale up-sampling network, and the semantic segmentation network is embedded into the multi-scale up-sampling network as a sub-module; the multi-scale upsampling network is serially cascaded with the weighted prediction layer;
the feature extraction network: sequentially stacking and cascading a high-definition convolution module and a feature fusion module;
the high-definition convolution module is formed by sequentially and crossly connecting a plurality of Firemodule modules;
the Firemodule module is formed by sequentially stacking and cascading a compression convolutional layer, an expansion batch normalization layer and a ReLU activation layer;
the characteristic fusion module is formed by sequentially stacking and cascading a down-sampling convolution layer, a down-sampling batch normalization layer and a ReLU activation layer;
the feature extraction network is defined as:
Figure FDA0003588915540000021
wherein N isGRepresenting the number of high definition convolution modules in a feature extraction network, NRRepresenting the number of feature fusion modules in a feature extraction network, NFRepresenting the number of Firemodule modules in each high-definition convolution module in the feature extraction network, NHR1Representing the number of layers of the compressed convolutional layer in each FireModule Module, NHR2Indicates the number of layers, N, of expansion convolution layers in each FireModule ModuleHR3Number of layers, N, representing expanded batch normalization layer in each FireModule ModuleHR4Representing the number of layers, N, of the downsampled convolutional layer in each feature fusion moduleHR5Representing the number of layers of the downsampling batch normalization layer in each feature fusion module;
Figure FDA0003588915540000022
representing the parameters in the b1 compressed convolutional layer in the a1 high-definition convolutional module as the parameters to be optimized;
Figure FDA0003588915540000023
represents the parameters in the b2 th expansion convolutional layer in the a1 th high-definition convolutional module asParameters to be optimized;
Figure FDA0003588915540000024
representing the translation amount of a b3 expansion batch normalization layer in an a1 high-definition convolution module as a parameter to be optimized;
Figure FDA0003588915540000025
representing the scaling amount of a b3 th expansion batch normalization layer in an a1 th high-definition convolution module as a parameter to be optimized;
Figure FDA0003588915540000026
representing the parameters in the b4 down-sampling convolutional layer in the a2 th feature fusion module as the parameters to be optimized;
Figure FDA0003588915540000027
representing the translation amount of a b5 down-sampling batch normalization layer in an a2 th feature fusion module as a parameter to be optimized;
Figure FDA0003588915540000028
representing the scaling quantity of a b5 down-sampling batch normalization layer in an a2 th feature fusion module as a parameter to be optimized;
the input data of the feature extraction network is a single image in the truck collection license plate detection network training set in the step 2, and the output data is a feature map, namely Feat1 (M)1×N1×C1)；
In the output data of the feature extraction network, M1Is a width, N, of the feature map Feat11Height, C, of characteristic diagram Feat11The number of channels for feature map Feat 1;
the multi-scale up-sampling network: sequentially stacking and cascading an upsampling convolutional layer, an upsampling batch layer, an upsampling deconvolution layer and a ReLU active layer;
Figure FDA0003588915540000031
wherein,
Figure FDA0003588915540000032
representing the number of layers of the upsampled convolutional layer in the multi-scale upsampling network;
Figure FDA0003588915540000033
the number of layers of an upsampled batch layer in the multi-scale upsampling network is represented;
Figure FDA0003588915540000034
representing the number of layers of an upsampled deconvolution layer in a multi-scale upsampling network; UP _ kernelcRepresenting parameters in a c layer up-sampling convolution layer in the multi-scale up-sampling network, wherein the parameters are to-be-optimized parameters; UP _ GammadRepresenting the translation amount of a sampling batch normalization layer on the d-th layer in the multi-scale up-sampling network, and taking the translation amount as a parameter to be optimized; UP _ betadRepresenting the scaling quantity of a sampling batch normalization layer on the d-th layer in the multi-scale up-sampling network, and taking the scaling quantity as a parameter to be optimized; UP _ dkerneleRepresenting parameters in an up-sampling deconvolution layer on the e-th layer in the multi-scale up-sampling network, wherein the parameters are to-be-optimized parameters;
the input data to the multiscale upsampling network is a feature map, Feat1, and the output data is a low resolution feature map, Feat2 (M)2×N2×C2) The medium resolution characteristic map is Feat3 (M)3×N3×C3) High resolution feature map, i.e., Feat4 (M)4×N4×C4)；
M of output data of the multi-scale up-sampling network2For the width, N, of the low resolution feature map Feat22Height, C, of low resolution feature map Feat22The number of channels for the low resolution feature map, Feat 2; m3For the width, N, of the medium resolution feature Feat33Height, C, of medium resolution feature Feat33The number of channels is the middle resolution feature map Feat 3; m4For the width, N, of the high resolution feature map Feat44Height, C, of high resolution feature map Feat44Number of channels for high resolution feature map Feat 4;
the semantic segmentation network: the device is formed by sequentially stacking and cascading a segmentation convolution layer, a segmentation batch normalization layer and a ReLU activation layer;
the semantic segmentation network is defined as:
Figure FDA0003588915540000035
wherein,
Figure FDA0003588915540000036
indicating the number of layers of the partitioned convolutional layers in the semantic partitioning network,
Figure FDA0003588915540000037
representing the number of layers of a segmentation batch normalization layer in the semantic segmentation network; MASK _ kernelfRepresenting the parameter in the f-th segmentation convolution layer in the semantic segmentation network as the parameter to be optimized; MASK _ GammagRepresenting the translation amount of the g-th segmentation batch normalization layer in the semantic segmentation network, wherein the translation amount is a parameter to be optimized; MASK _ betagRepresenting the scaling quantity of the g-th segmentation batch normalization layer in the semantic segmentation network as a parameter to be optimized;
the input data of the semantic segmentation network is a low resolution feature map, i.e., Feat2, a medium resolution feature map, i.e., Feat3, and a high resolution feature map, i.e., Feat4, and the output data is a low resolution segmentation feature map, i.e., Feat5 (M)5，N5，C5) The medium resolution segmentation feature map is Feat6 (M)6，N6，C6) High resolution segmentation feature map, i.e., Feat7 (M)7，N7，C7)；
In the output data of the semantic segmentation network, M5Segmenting the width, N, of the feature map Feat5 for low resolution5For low resolution segmentation of the height, C, of the feature map Feat55The number of channels for the low resolution segmented feature map, Feat 5; m6For medium resolution segmentation of the width, N, of the feature map Feat66For segmenting features at medium resolutionHeight, C, of Feat66The number of channels of the feature map Feat6 is divided with medium resolution; m7Segmenting the width, N, of the feature map Feat7 for high resolution7For high resolution segmentation of the height, C, of the feature map Feat77The number of channels of the high-resolution segmentation feature map Feat 7;
the weighted prediction layer: the system is formed by sequentially stacking and cascading a prediction convolution layer, a maximum pooling layer, a ReLU activation layer and a Sigmoid activation layer;
the weighted prediction layer is defined as:
fWE(WE_kernelh)，h∈[1，NWE]
wherein N isWERepresenting the number of predicted convolutional layers in the weighted prediction layer; WE _ kernelhRepresenting the parameters in the h prediction convolution layer in the weighted prediction layer as the parameters to be optimized;
the input data of the weighted prediction layer is Feat2, which is a low resolution feature map, Feat3, Feat4, Feat5, which is a low resolution segmentation feature map, Feat6, Feat7, which is a medium resolution segmentation feature map, and the output data is Feat8, which is a low resolution prediction feature map (M)8，N8，C8) Feat9 (M), which is a medium resolution prediction feature map9，N9，C9) High resolution predictive feature map, Feat10 (M)10，N10，C10)；
In the output data of the weighted prediction layer, M8Predicting the width, N, of the feature map Feat8 for low resolution8Predicting the height, C, of the feature map Feat8 for low resolution8Predicting the number of channels of the feature map Feat8 for the low resolution; m9Predicting the width, N, of feature map Feat9 for medium resolution9Predicting the height, C, of feature map Feat9 for medium resolution9Predicting the number of channels of the feature map Feat9 for medium resolution; m10Predicting the width, N, of the feature map Feat10 for high resolution10Predicting the height, C, of the feature map Feat10 for high resolution10Predicting the number of channels of the feature map Feat10 for high resolution;
step 3, the collection card license plate recognition network specifically comprises: dense feature extraction network, sequence feature extraction network;
the dense feature extraction network and the sequence feature extraction network are in serial cascade connection;
the dense feature extraction network is formed by sequentially stacking and cascading a plurality of dense convolution modules;
the dense convolution module is formed by sequentially stacking and cascading a dense convolution layer, a dense batch normalization layer and a Mish activation layer;
the dense feature extraction network is defined as:
Figure FDA0003588915540000051
wherein,
Figure FDA0003588915540000052
representing the number of layers of the dense convolution layer in the dense feature extraction network;
Figure FDA0003588915540000053
representing the number of layers of a dense batch normalization layer in a dense feature extraction network; DENSE _ kerneliRepresenting the parameters in the ith dense convolution layer in the dense feature extraction network as the parameters to be optimized; DENSE-gammajRepresenting the translation quantity of the jth dense batch normalization layer in the dense feature extraction network as a parameter to be optimized; DENSE _ betajRepresenting the scaling quantity of the jth dense batch normalization layer in the dense feature extraction network as a parameter to be optimized;
the input data of the dense feature extraction network is a single image of the truck license plate recognition network training set in the step 2, and the output data is a dense feature map, namely Feat11 (M)11×N11×C11)；
In the output data of the dense feature extraction network, M11For the width, N, of the dense feature map Feat1111Height, C, of dense feature map Feat1111Number of channels being dense feature map Feat 11;
the sequence feature extraction network: the system is formed by sequentially stacking a bidirectional LSTM recursive layer, a sequence batch normalization layer and a ReLU activation layer;
the sequence feature extraction network is defined as:
Figure FDA0003588915540000054
wherein,
Figure FDA0003588915540000055
representing the number of layers of a bidirectional LSTM recursive layer in a sequence feature extraction network;
Figure FDA0003588915540000056
representing the number of layers of a sequence batch normalization layer in a sequence feature extraction network; SEQ _ kernelxRepresenting the parameters in the x-th bidirectional LSTM recursive layer in the sequence feature extraction network, and taking the parameters as the parameters to be optimized; SEQ _ gammayRepresenting the translation amount of a y-th layer sequence batch normalization layer in the sequence feature extraction network, wherein the translation amount is a parameter to be optimized; SEQ _ betayRepresenting the scaling quantity of a y-th layer sequence batch normalization layer in the sequence feature extraction network as a parameter to be optimized;
the input data of the sequence feature extraction network is a dense feature map, namely Feat11, and the output data is sequence features, namely Feat12 (L)12，C12)；
In the output data of the sequence feature extraction network, L12Sequence length, C, which is a characteristic of the sequence12The number of channels that are sequence features;
step 3, constructing a truck license plate detection network loss function through the corner confidence coefficient loss function and the segmentation confidence coefficient loss function;
when an image of a truck head of a truck is input into a truck collection and license plate detection network for training, four circular areas are obtained by dividing the four corner points of a license plate by taking the four corner points as the circle center and taking r as the radius, weight distribution is obtained in the circular areas through two-dimensional Gaussian distribution, an H multiplied by W corner point thermodynamic diagram is obtained by input images through network prediction, feature points falling in the four circles are selected as positive samples, and the rest points are all regarded as negative samples and used for learning feature information of the corner points of the license plate;
the corner confidence loss function is
Figure FDA0003588915540000061
Wherein, N represents the number of all prediction sample points;
Figure FDA0003588915540000062
representing the confidence coefficient of the corner point with x as the abscissa and y as the ordinate in the predicted corner point thermodynamic diagram; alpha represents a positive and negative sample balance parameter; y isxyRepresenting the weight of the corner point with x as the abscissa and y as the ordinate in the truck head image; beta represents a difficult and easy sample equilibrium parameter;
the segmentation confidence loss function is:
Figure FDA0003588915540000063
wherein, N represents the number of all prediction sample points;
Figure FDA0003588915540000064
representing pixel confidence coefficients with x as the abscissa and y as the ordinate in the prediction segmentation graph; alpha represents a positive and negative sample balance parameter; aYxyRepresenting the pixel weight with x as the abscissa and y as the ordinate in the truck head image; beta represents a difficult and easy sample equilibrium parameter;
the network loss function of the collection card license plate detection is as follows:
Ld＝Lk+Lm
wherein L iskAs a corner confidence loss function, LmIs a segmentation confidence loss function;
step 3, constructing a card collection license plate recognition network loss function through the CTC;
the collection card license plate recognition network loss function is as follows:
Figure FDA0003588915540000071
and is
Figure FDA0003588915540000072
Wherein, p (l | y) represents the probability sum of all CTC sequences alpha of the predicted sequence l obtained by the beta mapping function under the condition that the prediction result is y; beta represents a mapping function for removing the repeated elements and then removing blank labels from the prediction result, so that beta-1 represents an inverse function of beta; l istAnd the method shows that the beta of the prediction result is mapped into the entropy sum of x under the condition that the license plate text label is x and the collection card license plate recognition network prediction result is y.
2. The multi-channel automatic identification method for the container terminal truck-collecting license plate of claim 1, wherein the truck-collecting head image dataset in the step 1 is as follows:
{trains(m，n)，s∈[1，S]，m∈[1，M]，n∈[1，N]}
wherein, trains(M, N) represents pixel information of the mth row and nth column of the S-th head image in the truck head image dataset, S represents the number of all image samples in the head image dataset, M is the row number of each head image in the truck head image dataset, and N is the column number of each head image in the truck head image dataset;
preferably, the collection card license plate image data set in step 1 is:
{trainp(m，n)，p∈[1，P]，m∈[1，M]，n∈[1，N]}
wherein trainp(M, n) represents the pixel information of the mth row and the nth column of the pth locomotive image in the collection card license plate image data set, P represents the number of all image samples in the collection card license plate image data set, and M is a collection card license plate imageThe number of lines of each license plate image in the image data set is N, and the number of columns of each license plate image in the card collection license plate image data set is N;
preferably, in the step 2, the license plate mark frame coordinates of each vehicle head image in the truck head image data set are as follows:
Figure FDA0003588915540000073
Figure FDA0003588915540000081
Figure FDA0003588915540000082
Figure FDA0003588915540000083
Figure FDA0003588915540000084
wherein l represents the left on the truck head image, t represents the top on the truck head image, r represents the right on the truck head image, and b represents the bottom on the truck head image; s represents the number of all the truck head images in the truck head image dataset, KsRepresenting the total number of the truck board mark boxes in the truck head image data set; boxs，kThe coordinates of the kth license plate mark frame in the sth truck head image in the truck head image data set are shown,
Figure FDA0003588915540000085
the coordinates of the upper left corner of the kth license plate marking frame in the sth truck head image in the truck head image data set are represented,
Figure FDA0003588915540000086
the abscissa representing the upper left corner of the kth license plate mark frame in the sth truck head image data set,
Figure FDA0003588915540000087
the ordinate of the top left corner of the kth license plate marking frame in the sth truck head image in the truck head image data set is represented;
Figure FDA0003588915540000088
the coordinates of the top right corner of the kth license plate marking frame in the sth truck head image in the truck head image data set are represented,
Figure FDA0003588915540000089
the abscissa representing the upper right corner of the kth license plate mark frame in the sth truck head image data set,
Figure FDA00035889155400000810
the ordinate of the top right corner of the kth license plate mark frame in the sth truck head image data set is represented;
Figure FDA00035889155400000811
the coordinates of the lower right corner of the kth license plate mark frame in the sth truck head image in the truck head image data set are represented,
Figure FDA00035889155400000812
the abscissa representing the lower right corner of the kth license plate mark frame in the s-th truck head image data set,
Figure FDA00035889155400000813
and the ordinate represents the lower right corner of the kth license plate mark frame in the s-th truck head image data set.
3. The multi-channel automatic identification method for the container terminal truck-collecting license plate of claim 1, wherein the marked license plate number of each license plate image in the truck-collecting license plate image data set in the step 2 is as follows:
Figure FDA00035889155400000814
wherein S represents the total number of the card collecting license plates in the card collecting license plate image data set; textsThe license plate number text information of the s-th album card license plate image in the album card license plate image data set,
Figure FDA00035889155400000815
the 1 st license plate number character of the sth collection card license plate image in the collection card license plate image data set is represented;
Figure FDA00035889155400000816
2 nd license plate number character of the sth album license plate image in the album license plate image data set;
Figure FDA0003588915540000091
a 3 rd license plate number character representing the sth license plate image of the card collecting license plate image data set;
Figure FDA0003588915540000092
the 4 th license plate number character of the sth album license plate image in the album license plate image data set is represented;
Figure FDA0003588915540000093
the 5 th license plate number character of the sth album license plate image in the album license plate image data set is represented;
Figure FDA0003588915540000094
the 6 th license plate number character of the s card collecting license plate image in the card collecting license plate image data set is represented;
Figure FDA0003588915540000095
the 7 th license plate number character of the sth album license plate image in the album license plate image data set is represented;
step 2, the collection card license plate detection network training set is as follows:
{trains(m，n)，boxs，k}，s∈[1，S]，m∈[1，M]，n∈[1，N]，k∈[1，Ks]
wherein, trains(m, n) represents the pixel information of the mth row and the nth column of the mth truck head image in the truck collecting license plate detection network training set, boxs，kRepresenting the coordinates of a kth license plate marking frame in the sth truck head image in the truck collection license plate detection network training set; s represents the number of all image samples in the truck collection license plate detection network training set, M is the number of lines of each locomotive image in the truck collection license plate detection network training set, and N is the number of columns of each locomotive image in the truck collection license plate detection network training set;
step 2, the collection card license plate recognition network training set is as follows:
{trains(m，n)，texts}，s∈[1，S]，m∈[1，M]，n∈[1，N]
wherein, trains(m, n) represents pixel information of the mth row and nth column of the mth album card license plate image in the album card license plate recognition network training set, textsRepresenting the license plate number of the No. s card collecting license plate image in the card collecting license plate recognition network training set; s represents the number of all image samples in the truck collection license plate detection network training set, M is the number of lines of each locomotive image in the truck collection license plate detection network training set, and N is the number of columns of each locomotive image in the truck collection license plate detection network training set.
4. The multi-channel automatic identification method for container terminal truck-collecting license plates of claim 1, characterized in that in step 4, the low-resolution prediction feature map of the image to be detected is Feat8；
Step 4, the medium resolution ratio prediction characteristic diagram of the image to be detected is Feat9；
Step 4, the high resolution prediction characteristic diagram of the image to be detected is Feat10；
And 4, taking the preliminary detection result of the image to be detected as the probability that the predicted point belongs to the foreground, and defining the preliminary detection result as follows:
Ki∈[0，1]，i∈NDS
wherein N isDSRepresenting the number of predicted points, K, in the preliminary test result of the image to be testediRepresenting the probability that the predicted point in the preliminary detection result of the ith image to be detected belongs to the foreground;
the preliminary detection result of the image to be detected is defined as:
RDS＝{Ki}，i∈NDS
and the final detection result of the image to be detected is defined as:
RDE＝{Kε}，ε∈NDE
wherein N isDERepresenting the number of predicted points in the final detection result of the image to be detected, KεRepresenting the probability that the predicted point in the final detection result of the epsilon-th image to be detected belongs to the foreground;
and 4, defining the primary identification result of the image to be identified as follows:
RRS＝{Ti}，i∈NRS
wherein N isRSRepresenting the number of elements, T, of the predicted sequence in the preliminary recognition result of the image to be recognizediRepresenting the character category of the ith element in the prediction sequence in the preliminary recognition result of the image to be recognized;
and 4, defining the final identification result of the image to be identified as:
RRE＝{Tε}，ε∈NRE
wherein N isREAnd T epsilon represents the character category of the epsilon-th element in the prediction sequence in the final recognition result of the image to be recognized.
CN202110534074.3A 2021-05-17 2021-05-17 Multipath automatic identification method for container terminal truck collection license plate Active CN113343977B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110534074.3A CN113343977B (en) 2021-05-17 2021-05-17 Multipath automatic identification method for container terminal truck collection license plate

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110534074.3A CN113343977B (en) 2021-05-17 2021-05-17 Multipath automatic identification method for container terminal truck collection license plate

Publications (2)

Publication Number Publication Date
CN113343977A CN113343977A (en) 2021-09-03
CN113343977B true CN113343977B (en) 2022-06-14

Family

ID=77468811

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110534074.3A Active CN113343977B (en) 2021-05-17 2021-05-17 Multipath automatic identification method for container terminal truck collection license plate

Country Status (1)

Country Link
CN (1) CN113343977B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114140786B (en) * 2021-12-03 2024-05-17 杭州师范大学 HRNet coding and double-branch decoding-based scene text recognition method

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106446895A (en) * 2016-10-28 2017-02-22 安徽四创电子股份有限公司 License plate recognition method based on deep convolutional neural network
CN109670458A (en) * 2018-12-21 2019-04-23 北京市商汤科技开发有限公司 A kind of licence plate recognition method and device
CN110097044B (en) * 2019-05-13 2020-12-01 苏州大学 One-stage license plate detection and identification method based on deep learning
US10997469B2 (en) * 2019-09-24 2021-05-04 Motorola Solutions, Inc. Method and system for facilitating improved training of a supervised machine learning process
CN111709884A (en) * 2020-04-29 2020-09-25 高新兴科技集团股份有限公司 License plate key point correction method, system, equipment and storage medium
CN111598089B (en) * 2020-05-16 2023-04-14 湖南大学 License plate correction and recognition method based on deep learning

Also Published As

Publication number Publication date
CN113343977A (en) 2021-09-03

Similar Documents

Publication Publication Date Title
CN112884064B (en) Target detection and identification method based on neural network
CN109063723B (en) Weak supervision image semantic segmentation method based on common features of iteratively mined objects
CN111563557B (en) Method for detecting target in power cable tunnel
CN107886133A (en) A kind of underground piping defect inspection method based on deep learning
CN111160205B (en) Method for uniformly detecting multiple embedded types of targets in traffic scene end-to-end
CN112070135A (en) Power equipment image detection method and device, power equipment and storage medium
CN113688836A (en) Real-time road image semantic segmentation method and system based on deep learning
CN110648310A (en) Weak supervision casting defect identification method based on attention mechanism
CN113723377A (en) Traffic sign detection method based on LD-SSD network
CN111507275B (en) Video data time sequence information extraction method and device based on deep learning
CN110517270B (en) Indoor scene semantic segmentation method based on super-pixel depth network
CN113052057A (en) Traffic sign identification method based on improved convolutional neural network
CN112529090A (en) Small target detection method based on improved YOLOv3
CN110334709A (en) Detection method of license plate based on end-to-end multitask deep learning
CN110991374B (en) Fingerprint singular point detection method based on RCNN
CN112084890A (en) Multi-scale traffic signal sign identification method based on GMM and CQFL
CN111428664A (en) Real-time multi-person posture estimation method based on artificial intelligence deep learning technology for computer vision
CN114863098A (en) Tiny weak defect segmentation method for industrial scene
CN117197763A (en) Road crack detection method and system based on cross attention guide feature alignment network
CN115240259A (en) Face detection method and face detection system based on YOLO deep network in classroom environment
CN111652846B (en) Semiconductor defect identification method based on characteristic pyramid convolution neural network
CN113903022A (en) Text detection method and system based on feature pyramid and attention fusion
CN110991447A (en) Train number accurate positioning and identification method based on deep learning
CN113343977B (en) Multipath automatic identification method for container terminal truck collection license plate
CN114639067A (en) Multi-scale full-scene monitoring target detection method based on attention mechanism

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant