CN112257525A

CN112257525A - Logistics vehicle card punching identification method, device, equipment and storage medium

Info

Publication number: CN112257525A
Application number: CN202011072515.4A
Authority: CN
Inventors: 李斯; 赵齐辉
Original assignee: Dongpu Software Co Ltd
Current assignee: Dongpu Software Co Ltd
Priority date: 2020-10-09
Filing date: 2020-10-09
Publication date: 2021-01-22

Abstract

The invention discloses an identification method, device, equipment and storage medium for punching a logistics vehicle, aiming at the time-consuming and laborious problem of relying on manpower to report the arrival and departure time of logistics vehicles in the industry. The historical images of the station are classified, and the historical images are classified to establish an image data set; an improved SSD model is created; the image data set is input into the improved SSD model for training to obtain a vehicle punch-in recognition model; real-time output logistics through the vehicle punch-in recognition model The situation of vehicles entering and leaving the station; it is convenient for the management of the distribution center, and the operation efficiency of the distribution center and the running time of the logistics vehicles are improved.

Description

Logistics vehicle card punching identification method, device, equipment and storage medium

Technical Field

The invention belongs to the technical field of logistics vehicle management, and particularly relates to a method, a device, equipment and a storage medium for recognizing a logistics vehicle card punch.

Background

With the rapid development of the logistics industry, the traffic volume is increased dramatically. Due to the fact that the express logistics distribution center, the network points and the like are different in size, logistics vehicle arrival and departure time needs to be detected, waiting of other vehicles is avoided, and the redundancy problem is better solved. In the industry at present, the time for the logistics vehicles to reach the distribution center and leave the distribution center is generally reported manually, which wastes time and labor, so that the reasonable dispatching of the logistics vehicles is influenced, the smooth proceeding of express services is not facilitated, and the efficiency of express delivery is influenced.

Disclosure of Invention

The invention aims to provide a method, a device, equipment and a storage medium for identifying the punch card of a logistics vehicle, which can automatically identify the punch card type of the logistics vehicle through a machine learning network, reduce the burden of workers and improve the working efficiency.

In order to solve the problems, the technical scheme of the invention is as follows:

a logistics vehicle card punching identification method comprises the following steps:

step S1: acquiring historical images of the arrival and departure of the logistics vehicles, carrying out category marking on the historical images, and establishing an image data set; the categories comprise vehicle entering and vehicle leaving;

step S2: creating an SSD model, and replacing a VGG16 network in the SSD model with a ResNet50 network to obtain an improved SSD model;

step S3: inputting the image data set in the step S1 into an improved SSD model for training to obtain a vehicle card punching identification model;

step S4: and inputting the image of the logistics vehicle to be identified into the vehicle card punching identification model, and outputting the category of the image.

According to an embodiment of the present invention, the step S1 further includes:

carrying out category marking on the historical image by adopting a laboratory tool;

and storing the history images after class marking according to the format of the voc data set.

According to an embodiment of the present invention, the storing the history image after the class labeling in a format of a voc data set further includes:

creating a voc data set, and storing the unmarked historical images in a JPEGImages folder;

storing the marked historical image in an options folder; the names of the historical images in the JPEGImages folder correspond to the names of the xml files in the exceptions folder one by one;

establishing four txt files, namely, test.txt, train.txt, val.txt and train.txt in an ImageSets \ Main folder of the voc data set, and sequentially serving as a model test set, a model training set, a model verification set and a model training and verification set; and distributing image data for the four txt files according to a preset proportion.

According to an embodiment of the present invention, the step S2 further includes:

the SSD model comprises VGG backbones, Extra Layers and Multi-box Layers, and a VGG16 neural network in the VGG backbones is replaced by a ResNet50 neural network;

the Multi-box Layers comprise six feature map Layers, and deconvolution operations on output features of the corresponding last feature map layer are added in the second to sixth feature map Layers respectively.

According to an embodiment of the present invention, the adding, in the second to sixth feature map layers, a deconvolution operation on an output feature of a corresponding last feature map layer further includes:

deconvoluting the output feature of the first feature map layer, summing the feature obtained by deconvolution and the original feature input into the first feature map layer, and taking the obtained result as the input feature of the second feature map layer;

the same processing as that of the second feature map layer input feature is performed for the input features of the third to sixth feature map layers.

According to an embodiment of the present invention, the step S3 further includes:

dividing an image data set into a model training set, a model verification set and a model test set in sequence according to the proportion of 60%, 30% and 10%;

and inputting a model training set, a model verification set and a model test set into the improved SSD model for training to obtain the vehicle card punching identification model.

An identification device for checking a card of a logistics vehicle comprises:

the data set creating module is used for acquiring historical images of the arrival and departure of the logistics vehicles, performing category marking on the historical images and creating an image data set; the categories include vehicle arrival and vehicle departure;

the model creating module is used for creating the SSD model, and replacing a VGG16 network in the SSD model with a ResNet50 network to obtain an improved SSD model;

the model training module is used for inputting the image data set in the data set creating module into an improved SSD model for training to obtain a vehicle card punching identification model;

and the image identification module is used for inputting the image of the logistics vehicle to be identified into the vehicle card punching identification model and outputting the category of the image.

According to an embodiment of the present invention, the data set creating module includes an image acquiring unit and an image labeling unit;

the image acquisition unit is used for acquiring historical images of the physical distribution vehicle card punching and storing the historical images which are not marked in a JPEGImages folder;

the image Labeling unit is used for Labeling the type of the historical image by adopting a Labeling tool and storing the labeled historical image in an options folder; the names of the history images in the JPEGImages folder correspond one-to-one to the names of the xml files in the indices folder.

An identification device for checking a card of a logistics vehicle, comprising:

a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line;

the at least one processor calls the instructions in the memory to enable the identification device for the physical distribution vehicle card punching to execute the identification method for the physical distribution vehicle card punching in one embodiment of the invention.

A computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing a method for identifying a card punch of a logistics vehicle in an embodiment of the invention.

Due to the adoption of the technical scheme, compared with the prior art, the invention has the following advantages and positive effects:

the logistics vehicle card punching identification method in one embodiment of the invention aims at the problems of time and labor waste caused by the fact that the logistics vehicle arrival and departure time is reported by manpower in the industry, and establishes an image data set by acquiring historical images of the logistics vehicle arrival and departure, and carrying out category marking on the historical images; creating an improved SSD model; inputting the image data set into an improved SSD model for training to obtain a vehicle card punching identification model; outputting the conditions of entering and leaving of the logistics vehicles in real time through the vehicle card punching identification model; the distribution center management is convenient, and the operation efficiency of the distribution center and the operation timeliness of the logistics vehicles are improved.

Drawings

Fig. 1 is a flow chart of an identification method for a logistics vehicle card punching in an embodiment of the invention;

FIG. 2 is a diagram of an SSD model in accordance with an embodiment of the present invention;

FIG. 3 is a diagram of Extra Layers in an SSD model in accordance with an embodiment of the present invention;

FIG. 4 is a diagram of Mutli-box Layers in the SSD model according to an embodiment of the present invention;

FIG. 5 is a block diagram of a ResNet50 network in accordance with an embodiment of the present invention;

fig. 6 is a block diagram of an identification device for checking a card of a logistics vehicle according to an embodiment of the invention;

fig. 7 is a schematic diagram of an identification device for checking a card of a logistics vehicle in an embodiment of the invention.

Detailed Description

The identification method, device, equipment and storage medium for physical distribution vehicle card punching provided by the invention are further described in detail with reference to the accompanying drawings and specific embodiments. Advantages and features of the present invention will become apparent from the following description and from the claims.

Example one

Referring to fig. 1, the present embodiment provides an identification method for a logistics vehicle card punching, including:

Specifically, in step S1, since the logistics vehicles are usually recorded by the camera both when they enter and leave, images of the vehicles entering and leaving can be captured from the video recorded by the camera and stored as history images. A DSS digital monitoring system may also be installed, all cameras are accessed into one local area network, all cameras are accessed through a DSS monitoring platform, images of the logistics vehicles entering and exiting the station are captured, and stored in a bmp format as training samples for the vehicle card punching identification module in this embodiment.

Since the training sample of the SSD model in this embodiment uses the voc2007 data set, the history image needs to be saved in the format of the voc data set, and the voc data set is used as the image data set in this embodiment. The following folders are included under this voc data set: antotions, ImageSets, JPEGImages, SegmentationClass, and SegmentationObject. The JPEGImages folder is used for storing history images exported from the security inspection device, and names the history images in a form of "000001. jpg".

The options folder is used for storing the images after class labeling. In the embodiment, a label Labeling tool is adopted to label the category of the historical image. After the category marking is carried out on each historical image, an xml file is obtained, and the width, the height, the target vehicle, the category and the time of the image are recorded in the xml file. Saving the xml file in the options folder; moreover, one history image in the JPEGImages folder corresponds to one xml file with the same name in the exceptions folder.

It should be noted that the category names are indicated by lower case letters when labeling, for example, Arrival of a vehicle is indicated by Arrival, not by Arrival. This is because when using the password.

cls＝self._class_to_ind[obj.find('name').text.lower().strip()]

Only lower case letters are recognized and if the category label contains upper case letters, a KeyError error may occur.

The ImageSets folder is used for storing files such as train.txt, test.txt and val.txt created from the generated xml file. The four txt files can be stored in a Main folder under the ImageSets folder, wherein text represents a model test set, train represents a model training set, val represents a model validation set, and train represents a model training + validation set.

After four txt files are created, image data needs to be distributed for the four txt files respectively according to a preset proportion. In this embodiment, 50% of images in the image data set are assigned to a traffic.txt file, the remaining 50% of images in the image data set are assigned to a test.txt file, 50% of images in the traffic.txt file are assigned to a traffic.txt file, and the remaining 50% of images in the traffic.txt file are assigned to a val.txt file. Only the name of the image is stored in these four txt files, in the format:

000002

000003

above, the voc data set is basically created.

In step S2, an SSD model is created, and the VGG16 network in the SSD model is replaced with the ResNet50 network, resulting in an improved SSD model.

The SSD algorithm has the English full name of a Single Shot MultiBox Detector and a multi-classification Single-rod Detector, wherein the Single Shot indicates that the SSD algorithm belongs to a one-stage method, and the MultiBox indicates that the SSD is multi-frame prediction. The SSD algorithm is a combination of fast RCNN and YOLO, a regression-based mode (similar to YOLO) is adopted, and the category and the position of an object are directly regressed in a network, so that the detection speed is high; a region-based concept (similar to fasternn) is also utilized, and many candidate regions are used as ROIs in the detection process.

Referring to fig. 2, the entire SSD model consists of three major parts: VGG backbones, ExtraLayers, Multi-box Layers. The VGG backhaul adopts VGG16 partial network as basic network, and after 5-layer network, abandons full connection layer and changes it into hollow convolution network. For subsequent multi-scale extraction, a convolutional network is added after the VGG backhaul, as shown in fig. 3, and the convolutional network is an Extra feature layer. The hierarchy of the convolutional network is shown in the following table:

ExtraLayer	Output
		Conv8-1(s＝1)	[256,19,19]
Conv8-2(s＝2)	[512,10,10]
		Conv9-1(s＝1)	[128,10,10]
Conv9-2(s＝2)	[256,5,5]
		Conv10-1(s＝1)	[128,5,5]
Conv10-2(s＝2)	[256,3,3]
		Conv11-1(s＝1)	[128,3,3]
Conv11-2(s＝2)	[256,1,1]

for Multi-box Layers, please refer to fig. 4, the SSD model has a total of six Layers of Multi-scale extracted networks, and each layer convolves loc and conf to obtain the corresponding output.

In the embodiment, the model is improved on the basis of the SSD model. The specific improvement has two aspects:

first, the network of VGG16 in VGG Backbone is replaced by a ResNet50 network. ResNet50 is a residual error learning framework with the advantages of easy optimization, small computational burden and the like on the basis of the existing training deep network. The residual error is designed to solve the degradation and gradient problems, so that the performance of the network is improved while the depth is increased. ResNet50 includes 49 convolutional layers and 1 fully-connected layer, wherein ID BLOCKx2 in the second to fifth stages represents two residual BLOCKs with unchanged dimension, CONV BLOCK represents a residual BLOCK with added dimension, each residual BLOCK includes three convolutional layers, so that there are 49 convolutional layers with 1+3 (3+4+6+3), and the structure is shown in FIG. 5.

In FIG. 5, CONV is the convolution operation, BatchNorm is the batch regularization process, Relu is the activation function, MAXFOOL and AvgPOOL are two pooling operations; the ZERO PAD module is followed by the first stage to the fifth stage in sequence from left to right, wherein the second stage to the fifth stage represent the residual block. Since the size of the ResNet50 neural network input data is 224 × 3, the image needs to be preprocessed before inputting the data, clipping the data with size 700 × 460 × 3 to the specified size batch _ size × 224 × 3. The image is subjected to continuous convolution operation of a residual block, the Channel number of an image pixel matrix becomes deeper and deeper, the size of the image pixel matrix is changed into batch _ size × 2048 through a flat layer Flatten, and finally the image pixel matrix is input into a full connection layer FC and the corresponding class probability is output through a softmax layer.

Second, deconvolution operations were introduced in Multi-box Layers. The Multi-box Layers comprise six feature map Layers, and deconvolution operations of output features of the corresponding last feature map layer are added in the second to sixth feature map Layers respectively.

Specifically, the deconvolution operation of adding the output feature of the corresponding last feature map layer in the second to sixth feature map layers respectively includes: and performing deconvolution on the output features of the first feature map layer, and summing the features obtained through deconvolution and the original features input into the first feature map layer to obtain a result serving as the input features of the second feature map layer. In this way, the same processing as the input features of the second feature map layer is performed for the input features of the third to sixth feature map layers, respectively.

In practical application, the input features of the first feature map layer may be zero-filled, then the convolution kernel is transposed (left-right inversion is performed first, and then up-down inversion is performed), and the transposed convolution kernel is convolved with the input features of zero-filled, so as to obtain output features as the input features of the second feature map layer. And when the input characteristic is a matrix, converting the convolution kernel into a convolution matrix, transposing, and convolving with the input matrix to obtain an output matrix as the input characteristic of the second feature map layer.

Deconvolution operation is introduced into the Multi-box Layers, signal recovery can be realized, and upsampling is realized in a Full Convolution Network (FCN). The feature information of the previous feature map layer is added into the next feature map layer, so that good characterization features are provided for the subsequent priorbox extraction, box frame type and position prediction work, and the improved SSD model performance is improved. The modeling and optimization of the SSD can be realized through a pytorch technology.

In step S3, the image dataset in step S1 is input into a modified SSD model for training, resulting in a vehicle punch recognition model.

After the improved SSD model is created, the image data sets obtained in step S1 are classified into a model training set, a model verification set, and a model test set, and the image data amounts of the model training set, the model verification set, and the model test set account for 60%, 30%, and 10% of the entire image data set in sequence. The above classification of the image data set can be realized by writing script code. Then, in the SSD model, parameters such as the category, the batch _ size, the training times and the like which need to be trained are modified, and then the training can be carried out. In order to improve the article identification accuracy of the article identification classification model, the training can be repeated until the article identification accuracy reaches a preset standard (e.g. 95% accuracy). The finally trained SSD model is used as the vehicle card punching recognition model in this embodiment.

In step S4, the image of the logistics vehicle to be recognized is input into the vehicle punch recognition model, and the category of the image is output.

The DSS digital monitoring system is provided with a screenshot function and can intercept images of the vehicles entering the station or leaving the station from the monitoring video, so that the images can be intercepted by the DSS monitoring platform, the images are input into a vehicle card punching identification model, and the images are output to belong to the vehicles entering the station (entering the station) or the vehicles leaving the station (leaving the station). The distribution center can reasonably arrange the time of the logistics vehicles getting to and getting away from the card according to the real-time output result of the vehicle card punching identification model, reasonably arrange the loading and unloading time of the vehicles, and avoid the situations of vehicle waiting and redundancy.

Example two

The invention also provides a device for identifying the card punching of the logistics vehicle, and referring to fig. 6, the device comprises:

the data set creating module 1 is used for acquiring historical images of the arrival and departure of the logistics vehicles, performing category marking on the historical images and creating an image data set; the categories include vehicle arrival and vehicle departure;

the model creating module 2 is used for creating an SSD model, and replacing a VGG16 network in the SSD model with a ResNet50 network to obtain an improved SSD model;

the model training module 3 is used for inputting the image data set in the data set creating module 1 into an improved SSD model for training to obtain a vehicle card punching identification model;

and the image recognition module 4 is used for inputting the image of the logistics vehicle to be recognized into the vehicle card punching recognition model and outputting the category of the image.

The data set creating module 1 includes an image acquiring unit and an image labeling unit. The image acquisition unit is used for acquiring the historical image of the physical distribution vehicle card punching and storing the unmarked historical image in a JPEGImages folder. The image Labeling unit is used for Labeling the category of the historical image by adopting a Labeling tool and storing the labeled historical image in an options folder; the names of the history images in the JPEGImages folder correspond one-to-one to the names of the xml files in the indices folder.

The specific contents and implementation methods of the data set creating module 1, the model creating module 2, the model training module 3, and the image recognition module 4 are all as described in the first embodiment, and are not described herein again.

EXAMPLE III

The second embodiment of the present invention describes the identification device for checking the card of the logistics vehicle in detail from the perspective of the modular functional entity, and the following describes the identification device for checking the card of the logistics vehicle in detail from the perspective of hardware processing.

Referring to fig. 7, the identification device 500 for checking a logistics vehicle may have a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 510 (e.g., one or more processors) and a memory 520, one or more storage media 530 (e.g., one or more mass storage devices) storing applications 533 or data 532. Memory 520 and storage media 530 may be, among other things, transient or persistent storage. The program stored in the storage medium 530 may include one or more modules (not shown), each of which may include a series of instruction operations in the identification device 500 for card-punching the logistics vehicle.

Further, the processor 510 may be configured to communicate with the storage medium 530, and execute a series of instruction operations in the storage medium 530 on the identification device 500 for the logistics vehicle card punching.

The identification device 500 for checking a logistics vehicle may further include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input-output interfaces 560, and/or one or more operating systems 531, such as Windows service, Vista, etc.

Those skilled in the art will appreciate that the structure of the identification device for the card punching of the logistics vehicle shown in fig. 7 does not constitute a limitation of the identification device for the card punching of the logistics vehicle, and may include more or less components than those shown, or some components in combination, or a different arrangement of components.

The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and which may also be a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the identification method for physical distribution vehicle card punching in the first embodiment.

The modules in the second embodiment, if implemented in the form of software functional modules and sold or used as independent products, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be substantially or partially implemented in software, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and devices may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, it is still within the scope of the present invention if they fall within the scope of the claims of the present invention and their equivalents.

Claims

1. an identification method for punching in a logistics vehicle, is characterized in that, comprises:

Step S1: obtaining historical images of the logistics vehicles entering and leaving the station, labeling the historical images by category, and establishing an image data set; the categories include vehicle entering and vehicle leaving the station;

Step S2: Create an SSD model, replace the VGG16 network in the SSD model with the ResNet50 network, and obtain an improved SSD model;

Step S3: Input the image data set in the step S1 into the improved SSD model for training to obtain a vehicle punching recognition model;

Step S4: Input the image of the logistics vehicle to be recognized into the vehicle punch-in recognition model, and output the category of the image.

2. The identification method for punching in a logistics vehicle as claimed in claim 1, wherein the step S1 further comprises:

Use Labeling tool to label historical images;

The historical images with category annotations are stored in the format of the voc dataset.

3. the identification method of the logistics vehicle punching card as claimed in claim 2, it is characterised in that the described historical image after the category labelling is stored in the format of the voc data set and further comprises:

Create a voc dataset and save unlabeled historical images in the JPEGImages folder;

Save the marked historical images in the Annotations folder; the names of the historical images in the JPEGImages folder correspond one-to-one with the names of the xml files in the Annotations folder;

Create four txt files in the ImageSets\Main folder of the voc dataset, namely test.txt, train.txt, val.txt and trainval.txt, which are used as the model test set, model training set, model validation set and model in turn. Training + validation set; image data are allocated to four txt files respectively according to preset ratios.

4. The identification method for punching in a logistics vehicle as claimed in claim 1, wherein the step S2 further comprises:

The SSD model includes VGG Backbone, Extra Layers and Multi-box Layers, and the VGG16 neural network in the VGG Backbone is replaced by a ResNet50 neural network;

The Multi-box Layers includes six feature map layers, and deconvolution operations on the output features of the corresponding previous feature map layer are respectively added to the second feature map layer to the sixth feature map layer.

5. the identification method of the logistics vehicle punching card as claimed in claim 4, is characterized in that, described in the second featuremap layer to the sixth feature map layer respectively adds the deconvolution to the output feature of the corresponding upper feature map layer Actions further include:

Deconvolute the output features of the first feature map layer, and then sum the features obtained by deconvolution with the original features input to the first feature map layer, and the obtained results are used as the input features of the second feature map layer;

The input features of the third feature map layer to the sixth feature map layer are processed in the same way as the input features of the second feature map layer.

6. The identification method for punching in a logistics vehicle as claimed in claim 1, wherein the step S3 further comprises:

Divide the image data set into model training set, model validation set and model testing set in turn according to the ratio of 60%, 30% and 10%;

The model training set, the model verification set and the model test set are input into the improved SSD model for training to obtain the vehicle punching recognition model.

7. An identification device for punching in a logistics vehicle, characterized in that, comprising:

The data set creation module is used to obtain historical images of logistics vehicles entering and leaving the station, label the historical images by category, and establish an image data set; the categories include vehicle arrival and vehicle departure;

The model creation module is used to create an SSD model, and replace the VGG16 network in the SSD model with the ResNet50 network to obtain an improved SSD model;

The model training module is used for inputting the image data set in the data set creation module into the improved SSD model for training to obtain the vehicle punch card recognition model;

The image recognition module is used to input the image of the logistics vehicle to be recognized into the vehicle punch-in recognition model, and output the category of the image.

8. The identification device for punching in a logistics vehicle as claimed in claim 7, wherein the data set creation module comprises an image acquisition unit and an image labeling unit;

The image acquisition unit is used to acquire the historical images of the logistics vehicles punched in, and save the unmarked historical images in the JPEGImages folder;

Described image labeling unit is used for adopting Labeling tool to carry out category labeling to historical image, and the historical image after labeling is saved in the Annotations folder; The name of the historical image in the JPEGImages folder is the same as the name of the xml file in the Annotations folder. A correspondence.

9. An identification device for punching in a logistics vehicle, characterized in that, comprising:

a memory and at least one processor with instructions stored in the memory, the memory and the at least one processor interconnected by wires;

The at least one processor invokes the instructions in the memory, so that the identification device for punching a logistics vehicle performs the identification method for punching a logistics vehicle according to any one of claims 1-6.

10. A computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the computer program according to any one of claims 1-6 is implemented. Recognition method of logistics vehicle punch card.