CN112598660B - Automatic detection method for pulp cargo quantity in wharf loading and unloading process - Google Patents
Automatic detection method for pulp cargo quantity in wharf loading and unloading process Download PDFInfo
- Publication number
- CN112598660B CN112598660B CN202011590292.0A CN202011590292A CN112598660B CN 112598660 B CN112598660 B CN 112598660B CN 202011590292 A CN202011590292 A CN 202011590292A CN 112598660 B CN112598660 B CN 112598660B
- Authority
- CN
- China
- Prior art keywords
- pulp
- line segment
- samples
- candidate
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 230000008569 process Effects 0.000 title claims abstract description 18
- 238000001514 detection method Methods 0.000 title abstract description 46
- 238000010586 diagram Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 22
- 229920001131 Pulp (paper) Polymers 0.000 claims abstract description 19
- 238000001914 filtration Methods 0.000 claims abstract description 14
- 238000012549 training Methods 0.000 claims description 11
- 238000012417 linear regression Methods 0.000 claims description 5
- 230000002776 aggregation Effects 0.000 claims description 4
- 238000004220 aggregation Methods 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 230000001629 suppression Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 239000000284 extract Substances 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 229910000831 Steel Inorganic materials 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 229920006395 saturated elastomer Polymers 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30242—Counting objects in image
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an automatic detection method for the quantity of paper pulp cargos in the loading and unloading process of a wharf, which comprises the following steps: s1: extracting a video stream of a transport vehicle loaded with pulp goods in real time at a certain frequency; s2: processing the extracted video image to segment a characteristic diagram of the paper pulp cargo part; s3: processing the characteristic diagram to obtain candidate connection points; s4: extracting a series of candidate line segment samples according to the candidate connecting points; s5: filtering the candidate line segment samples; s6: and counting the number of the candidate line segment samples to obtain the number of the pulp cargos. The invention can automatically detect the quantity of paper pulp goods, reduce the task amount and improve the detection efficiency, has no manual participation, has high intelligent degree and improves the detection accuracy.
Description
Technical Field
The invention relates to the technical field of computer vision and deep learning, in particular to an automatic detection method for the quantity of paper pulp cargos in a wharf loading and unloading process.
Background
Currently, wharves are heavily developed and pulp goods need to be transported by means of transport means (e.g. flat trucks) during loading and unloading at the wharf. The pulp goods have the characteristic of one bundle and one bag, the pulp goods need to be manually checked in the process of loading and unloading the pulp goods, the number of the pulp goods needs to be manually checked by paper records, and then a truck can be released.
The manual counting work has the advantages of large task amount, labor cost increase, low efficiency and wrong counting amount caused by manual fatigue or negligence.
Therefore, an intelligent and automatic method for detecting the quantity of the pulp goods is needed, so that the counting efficiency and the counting accuracy are improved while the task amount is reduced.
Disclosure of Invention
The embodiment of the invention provides an automatic detection method for the quantity of paper pulp cargos in the wharf loading and unloading process.
In order to realize the purpose of the invention, the invention is realized by adopting the following technical scheme:
the application relates to a method for automatically detecting the quantity of paper pulp goods in a wharf loading and unloading process, which is characterized by comprising the following steps of:
s1: extracting a video stream of a transport vehicle loaded with pulp goods in real time at a certain frequency;
s2: processing the extracted video image to segment a characteristic diagram of the paper pulp cargo part;
s3: processing the characteristic diagram to obtain candidate connection points;
s4: extracting a series of candidate line segment samples according to the candidate connecting points;
s5: filtering the candidate line segment samples;
s6: and counting the number of the candidate line segment samples to obtain the number of the pulp cargos.
In the present application, step S2 includes the following:
s21: inputting the extracted video image into a residual error network to obtain a characteristic image;
s22: processing the characteristic image, and extracting a rectangular candidate frame for distinguishing a foreground from a background;
s23: mapping the extracted rectangular candidate frame into the feature image, and unifying the window size of the rectangular candidate frame by using a regional feature aggregation technology;
s24: and segmenting the characteristic image into a characteristic map of the pulp cargo part according to the coordinate information of the rectangular candidate frame.
In the present application, S2 further includes the following steps after S23 and before S24: and S23': and performing boundary frame regression on the rectangular candidate frame after the unified window in the step S23 to correct the coordinate information of the rectangular candidate frame.
In the present application, step S3 includes the steps of:
s31: carrying out mesh division on the feature map to form M mesh units Wx×Hx；
S32: inputting the characteristic diagram into a convolutional layer and a classification layer in sequence for processing to calculate the confidence of each grid unit, and converting the characteristic diagram processed by the convolutional layer into a connection point biasMove characteristic diagram O (x)，
Wherein V represents a set of points and wherein,lirepresenting a certain connection point in the set of points ViIn the grid cellxIn the position (a) of (b),
representing grid cellsxThe center position of (a);
s33: performing threshold value limitation on the calculated confidence of each grid unit to obtain a probability feature map P (x) And classifies whether or not there is a connection point in each grid cell,
s34: using a connection point offset profile O (x) Predicting the relative position of the connection point in the corresponding grid cell;
s35: the relative positions of the connection points in the corresponding grid cells are optimized using linear regression.
In the present application, step S3 further includes the following steps: precise relative position information of the connection points in the grid cells is obtained by non-maxima suppression techniques.
In the present application, step S4 includes:
s41: outputting endpoint coordinate information of a series of line segment samples by adopting a positive and negative sample mixed sampling mechanism;
s42: and performing fixed-length vectorization processing on each line sample according to the end point coordinate information of each line sample to obtain a characteristic vector of each line sample so as to extract a series of line samples.
In the present application, step S5 specifically is: and filtering each line segment sample by using the intersection ratio between the areas of the rectangular frames formed by taking each line segment sample as an intersection line.
In the present application, step S5 specifically includes: and filtering each line segment sample by using the Euclidean distance between each line segment sample.
The automatic detection method for the number of the paper pulp cargos has the following advantages and beneficial effects:
after the pulp goods arrive at port, the pulp goods are loaded on the transport means, the video stream of the transport means loaded with the pulp goods is monitored in real time in a monitoring picture, video images are extracted from the real-time video stream, the part of the pulp goods is detected according to the video images, the characteristic that the pulp goods are bundled into one package is combined, and then line segment detection is carried out according to the detected part of the pulp goods, so that the detection of the quantity of the pulp goods is realized, the whole process does not need manual participation, the manual task quantity is reduced, the intellectualization and the automation are realized, the operation is carried out by means of a computer, the detection speed is high, and the detection efficiency is improved.
Other features and advantages of the present invention will become more apparent from the following detailed description of the invention when taken in conjunction with the accompanying drawings.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow chart of an embodiment of the method for automatically detecting the quantity of pulp goods in a wharf loading and unloading process according to the present invention;
FIG. 2 is an original image of a portion of pulp goods involved in an embodiment of the method for automatically detecting the quantity of pulp goods in a loading and unloading process of a wharf according to the present invention;
FIG. 3 is a diagram of candidate connection points obtained in an embodiment of the method for automatic detection of pulp cargo quantity for a loading and unloading process at a dock according to the present invention;
FIG. 4 is a diagram of a sample of true candidate line segments obtained in an embodiment of the method for automated detection of pulp cargo quantity during loading and unloading at a dock according to the present invention;
FIG. 5 is a final inspection diagram of the mapping of true candidate line segment samples onto partial original images of pulp cargos in an embodiment of the method for automatically detecting the quantity of pulp cargos in the loading and unloading process of a wharf, provided by the invention;
FIG. 6 is a decimated video image;
fig. 7 is a diagram showing the effect of the pulp cargo quantity automatic detection method applied to the video image in fig. 6 on the field detection of the pulp cargo during loading and unloading of the wharf.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
In order to avoid manually counting the number of paper pulp cargo packages in the loading and unloading process of a wharf, the application relates to an automatic detection method for the number of paper pulp cargo. The main task of the automatic detection method for the number of the pulp cargos is to detect the pulp cargo bags and calculate the number of the contained pulp cargos for all vehicles passing through the surveillance video.
The task is to realize the automatic detection of the quantity of the pulp goods on the transport means, and comprehensively use the target detection and the line segment detection.
Target detection has mainly two tasks: the classification and positioning of the target requires separating foreground objects from the background and determining the category and position information of the foreground.
Current target detection algorithms can be divided into candidate region-based and end-to-end based methods, with candidate region-based methods being more dominant in terms of detection accuracy and positioning accuracy. Meanwhile, the paper pulp goods are placed irregularly, so that auxiliary information such as volume, midpoint and the like is difficult to obtain, and the difficulty is increased for further determining the quantity of the paper pulp goods.
The pulp goods have the characteristic of binding one bundle by one bundle, and the quantity information can be effectively extracted by adopting a line segment detection method in combination with the current line segment detection technology.
Referring to fig. 1 to 7, implementation of the pulp goods quantity automatic detection method will be described in detail as follows.
S1: the video stream of the vehicle loaded with pulp goods is extracted in real time at a certain frequency.
After arrival of the pulp goods, they are loaded onto a transport means, such as a transport truck, which is monitored by means of a monitoring picture, and a video image is captured by extracting a certain frequency of frames per second in a monitored video stream, see fig. 2.
It should be noted that each package of pulp goods loaded on the transport vehicle is substantially rectangular and has substantially uniform height and cross-sectional area, and is bound together into a package by straps (e.g., steel wires).
S2: and processing the extracted video image to segment a characteristic diagram of the pulp cargo part.
The task of step S2 is to implement target detection, and is performed in a plurality of sections as follows.
And S21, inputting the video image into a residual error network to obtain a characteristic image.
In order to extract semantic features of the video image and improve the capability of acquiring detailed features of the image, the video image is input into a residual error network to acquire a feature map.
The residual error network is an asymmetric coding and decoding structure fused with expansion convolution.
For the convenience of subsequent processing, the input video images with different sizes are firstly unified into a square of 512 × 512, and then are sequentially input into a plurality of coding and decoding modules.
In the coding and decoding module, the coding part comprises 3 times of convolution operation with step size of 2, each size of convolution is followed by 6 residual connecting blocks, wherein the convolution kernel used in the residual connecting block is processed by the expansion rate of 2 so as to obtain a larger receptive field. The decoding section restores the image to the input size.
The asymmetric coding and decoding structure can fully extract details and fuse a larger receptive field, so that the residual module can provide abundant detail characteristics for subsequent processing.
S22: and processing the characteristic image, and extracting a rectangular candidate frame for distinguishing the foreground from the background.
Firstly, generating 9 rectangular frames containing 3 shapes (the length-width ratio belongs to {1, 2 }), traversing each point in the characteristic image, matching the 9 rectangular frames for each point, judging the rectangular frame belonging to the foreground through a Softmax classifier, and solving a two-classification problem; and meanwhile, the coordinate information of the rectangular frame is corrected by using border regression of the frame to form a more accurate rectangular candidate frame.
Note that the foreground refers to the pulp cargo, and the background refers to a portion other than the pulp cargo portion.
S23: and mapping the extracted rectangular candidate frame into a feature image, and unifying the window size of the rectangular candidate frame by using a regional feature aggregation technology.
The region feature aggregation technique is proposed when used in Mask RCNN to generate a fixed-size feature map from the generated candidate box region pro-apparent map, which is a technique commonly used in the existing example segmentation architecture.
S24: and segmenting the characteristic image into a characteristic image of the pulp cargo part according to the coordinate information of the rectangular candidate frame.
And finding out the rectangular candidate frames belonging to the foreground according to the coordinate information of each rectangular candidate frame, thereby segmenting the characteristic diagram of the pulp cargo part.
In order to ensure the accuracy of the rectangular candidate frame and improve the automatic detection precision of the number of the paper pulp cargos, before the feature map of the paper pulp cargo part is segmented, the rectangular candidate frame obtained in the step S23 is subjected to sequential boundary frame regression, the coordinate information of the rectangular candidate frame is corrected, and the accuracy of the coordinate information is realized.
According to the rectangular candidate frame with accurate coordinate information, the characteristic image is divided into a characteristic diagram of the pulp cargo part according to the coordinate information of the rectangular candidate frame.
And (5) segmenting a characteristic diagram of the pulp cargo part, namely completing target detection.
Then, the line segment detection is needed to be carried out on the characteristic diagram of the pulp cargo part so as to detect the quantity of the pulp cargo in the video image.
The specific line segment detection section is described below with reference to fig. 1 to 7.
S3: and processing the characteristic graph to obtain candidate connection points.
The characteristic diagram here refers to the characteristic diagram of the pulp cargo portion divided in S24.
S31: the feature map is subjected to grid division to formMA grid cellWx×Hx。
Performing gridding treatment on the feature mapW×HIs divided intoMA grid cell having a grid area ofWx×HxWhere V represents a set of points.
In a certain grid cellxIn the method, it is required to predict whether a candidate connection point exists, and if a connection point exists, predict that the connection point exists in the grid cellxRelative position in (2).
S32: and sequentially inputting the feature maps into a convolutional layer and a classification layer for processing so as to calculate the confidence coefficient of each grid unit, and converting the feature maps processed by the convolutional layer into a connection point offset feature map O (x).
Specifically, the feature maps are processed using a network comprising 1 x 1 convolutional layers and classification layers in which confidence is calculated by a softmax classification function as to whether there is a connection point in each grid cell.
And converting the characteristic diagram into a characteristic diagram O (with offset of connection points) by using a network containing 1 × 1 convolution layersx) The following:
wherein,lirepresenting a connection point in the set of points ViIn the grid cellxIn the position (a) of (b),
representing grid cellsxThe center position of (a).
S33: performing threshold value limitation on the calculated confidence of each grid unit to obtain a probability feature map P (x) And for each grid cell isWhether a connection point exists is classified.
Probability feature map P (x) The following were used:
that is, whether there is a connection point in the grid cell is a two-classification problem.
Limiting the calculated confidence of each grid unit by a threshold value p if the grid unit isxIf the confidence of (2) is greater than the threshold, then P (3) is satisfiedx) =1, consider the grid cellxIn (b) there is a connection point, otherwise P: (b)x) =0, consider the grid cellxThere are no connection points.
If the presence of a connection point in a grid cell is predicted, the relative location of the connection point in the grid cell continues to be predicted (this section is described in detail below).
S34: using a connection point offset profile O (x) The relative position of the connection point in the corresponding grid cell is predicted.
O（x) Arranged as the center point and connection point of the grid celliFor predicting the connection pointiIn the grid cellxRelative position in (2).
S35: the relative positions of the connection points in the corresponding grid cells are optimized using linear regression.
If the grid unit x comprises the connection point i, an L2 linear regression is selected to optimize the relative position of the connection point, and the objective function of the L2 linear regression is as follows:
whereinNvIndicating the number of connection points.
In addition, employing non-maximum suppression techniques further eliminates "non-attachment points" in each network cell, i.e., obtains more accurate relative location information of attachment points in the network cell.
Particularly in the procedure, may be implemented by maximally pooling operations for obtaining more accurate relative position information of connection points in the network element.
After the processing of S31 to S35 as described above, the relative positional relationship of the K candidate connection points with the highest confidence is finally output
Refer to fig. 3.
It should be noted that, before step S3 is actually executed, the whole model in S3 needs to be trained by using the cross entropy loss function, the trained model can be directly used in actual use, and when the feature map is used as an input, the output obtains the candidate connection points as described above.
S4: and extracting a series of candidate line segment samples according to the candidate connecting points acquired in the S3.
The purpose of this step is to select K candidate connection points based on the K candidate connection points obtained in S3
Obtaining T candidate line segment samples
Wherein
And
is shown aszThe coordinates of the endpoints of the sample of candidate line segments.
S41: and acquiring the endpoint coordinate information of the T candidate line segment samples by adopting a positive and negative sample mixed sampling mechanism.
It should be noted that the mixed sampling of positive and negative samples is a preparation work for model training, in the training process, the difference between the number of positive and negative samples of the K candidate connection points is large, the number of positive and negative samples needs to be balanced, and a mixed training mode of positive and negative samples is adopted, wherein the positive samples come from the labeled true line segment samples, and the negative samples are unreal line segment samples generated randomly through heuristic learning.
When there are few accurate positive samples or training is saturated in the extracted K candidate connection points, quantitative positive/negative samples are added to help start training. Moreover, the added positive samples help the prediction points to adjust the positions, and the prediction performance is improved.
S41: and performing fixed-length vectorization processing on each line sample according to the endpoint coordinate information of each line sample to obtain a feature vector of each line sample so as to extract a series of candidate line samples.
Based on some sample of candidate line segments, e.g. ofzTwo endpoint coordinates of candidate line segment sample
And
vectorization processing of fixed length for line segment sample, i.e. calculation by two end point coordinatesN l Uniformly distributing points, and obtaining coordinates of intermediate points on the feature map output in the step S2 through bilinear interpolation:
thus, a feature vector q of line segment samples is extracted, which isC×N l In whichCThe number of channels of the feature map output in step S2.
At this time, a sample of candidate line segments is extracted.
It should be noted that before a series of candidate line segment samples are obtained through the endpoint coordinate information of the line segment samples, model training is required.
In the training process, the feature map output in step S2 and the candidate line segment sample output in step S4 are required as input.
The training process is briefly described as follows:
first, based on a sample of candidate line segments, e.g. the firstzCoordinates of two endpoints of each candidate line segment sample
And
vectorization processing of fixed length for line segment sample, i.e. calculation by two end point coordinatesN l Uniformly distributing the points, and obtaining the coordinates of the intermediate points on the characteristic diagram output in the step S2 through bilinear interpolation:
then, the feature vector q is dimensionality reduced by a one-dimensional maximum pooling operation of step size s to becomeC×N l And/s and is expanded into a one-dimensional feature vector.
Inputting the one-dimensional feature vector into a full-connection layer for convolution processing to obtain a logic value, specifically, after performing full-connection convolution twice on the one-dimensional feature vector, taking a log value and returning to the full-connection layer
。
wherein the true valueyThat is, after the feature values in the feature map output in S2 by the true labels of the line segment samples are convolved, the log is taken and the value is returned.
The penalty is the log of the calculated prediction (i.e., the
) And the error between the log values (namely y) corresponding to the real labels of the line segment samples is used for model training and optimization.
Since repeated detection inevitably occurs during detection, that is, two line segment samples overlap, the line segment samples output in S4 need to be filtered in order to improve the detection accuracy of the line segment samples.
The filtering method is implemented by the step S5, that is, each line segment sample is filtered by using an Intersection-over-unity (IoU) ratio between areas of rectangular frames formed by each line segment sample as an Intersection, or each line segment sample is filtered by using a euclidean distance between line segment samples.
The filtering method is not limited herein as long as the purpose of filtering can be achieved.
The IoU filtering line segment samples are taken as an example for explanation.
Suppose that the coordinates of two coincident line segment samples are L respectively 1 [(x 11 ,y 11 )，(x 12 ,y 12 )]And L 2 [(x 21 ,y 21 )，(x 22 , y 22 )]。
L 1 Length H of rectangular frame R1 formed 1 Width W 1 And area A thereof 1 And L is 2 Length H of rectangular frame R2 formed 2 Width W of 2 And area A thereof 2 Respectively expressed as:
the length H, the width H and the area A of the rectangular frame where the two rectangular frames R1 and R2 intersect are respectively expressed as:
if it is not
Or
IoU =0. Otherwise, the IoU is calculated using the following equation:
and limiting the threshold of the IoU, adjusting the threshold of the IoU, filtering, and finally outputting a filtered line segment sample, which is shown in FIG. 4.
S6: and counting the number of the line segment samples after filtering to obtain the number of the pulp cargos.
The number of line segment samples after filtering is the number of pulp acquisitions.
Fig. 5 is a final detection diagram of the line segment candidate samples in fig. 4 mapped onto the original paper pulp goods image, and it can be seen from the detection effect that the automatic detection of the number of the paper pulp goods is accurate.
Referring to fig. 6 and 7, fig. 6 is a truncated original video image; fig. 7 is a diagram showing the effect of the pulp cargo field inspection applied to the video image in fig. 6 by using the automatic pulp cargo quantity detection method proposed in the present application, which outputs the quantity of line segment samples, i.e., the pulp cargo quantity.
As can be seen from the detection effect graphs of FIG. 5 and FIG. 7, the automatic detection method for the quantity of the paper pulp cargos, which is provided by the application, has high detection accuracy.
The automatic detection method for the quantity of the paper pulp cargos directly extracts the video images from the video stream for detection, is carried out automatically, does not need manual participation, reduces the manual task amount, and is high in detection speed due to intelligent computer calculation, and the detection efficiency is improved.
The above examples are only intended to illustrate the technical solution of the present invention, and not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions.
Claims (7)
1. A method for automatically detecting the quantity of pulp goods in a wharf loading and unloading process is characterized by comprising the following steps:
s1: extracting a video stream of a transport vehicle loaded with pulp goods in real time at a certain frequency;
s2: processing the extracted video image to segment a characteristic diagram of the paper pulp cargo part;
s3: processing the characteristic graph to obtain K candidate connection points, wherein K is greater than 1;
s4: extracting a series of candidate line segment samples according to the candidate connecting points;
s5: filtering the candidate line segment samples;
s6: counting the number of the candidate line segment samples to obtain the number of the pulp cargos;
wherein the step S3 comprises the following steps:
s31: carrying out mesh division on the feature map to form M mesh units Wx×Hx；
S32: sequentially inputting the feature maps into a convolutional layer and a classification layer for processing to calculate the confidence of each grid unit, and converting the feature maps processed by the convolutional layer into a connection point offset feature map O (in the following steps of (1))x)，
Wherein V represents a set of points and wherein,lirepresenting a point of connection in a set of points ViIn the grid cellxIn the position (a) of (b),
representing grid cellsxThe center position of (a);
s33: performing threshold value limitation on the calculated confidence of each grid unit to obtain a probability feature map P (x) And classifies the presence of connection points in each grid cell,
s34: using the connection point offset profile O (x) Predicting the relative position of the connection point in the corresponding grid cell;
s35: the relative positions of the connection points in the corresponding grid cells are optimized using linear regression.
2. The method for automatically detecting the pulp cargo quantity according to claim 1, wherein the step S2 comprises the steps of:
s21: inputting the extracted video image into a residual error network to obtain a characteristic image;
s22: processing the characteristic image, and extracting a rectangular candidate frame for distinguishing a foreground from a background;
s23: mapping the extracted rectangular candidate frame into the feature image, and unifying the window size of the rectangular candidate frame by using a regional feature aggregation technology;
s24: and segmenting the characteristic image into a characteristic map of the pulp cargo part according to the coordinate information of the rectangular candidate frame.
3. The method according to claim 2, wherein S2 further comprises the following steps after S23 and before S24:
and S23': and performing boundary frame regression on the rectangular candidate frame after the unified window in the step S23 to correct the coordinate information of the rectangular candidate frame.
4. The method for automatically detecting the pulp cargo quantity according to claim 1, wherein the step S3 further comprises the following steps:
precise relative position information of the connection points in the grid cells is obtained by non-maxima suppression techniques.
5. The method for automatically detecting the pulp cargo quantity according to claim 1, wherein the step S4 comprises:
s41: outputting endpoint coordinate information of a series of line segment samples in a positive and negative sample mixed sampling mode;
s42: vectorizing each line sample in a fixed length according to the endpoint coordinate information of each line sample to obtain a characteristic vector of each line sample so as to extract a series of line samples;
in the training process, the difference between the number of positive samples and the number of negative samples of K candidate connecting points is large, the number of positive samples and the number of negative samples need to be balanced, and a positive sample and negative sample mixed training mode is adopted, wherein the positive samples are marked real line segment samples, and the negative samples are unreal line segment samples generated through a heuristic learning random.
6. The method for automatically detecting the quantity of pulp goods according to claim 1, wherein the step S5 is specifically:
and filtering each line segment sample by utilizing the intersection ratio of the areas of each rectangular frame formed by taking each line segment sample as an intersection line.
7. The method for automatically detecting the quantity of pulp goods according to claim 1, wherein the step S5 is specifically as follows:
and filtering each line segment sample by using the Euclidean distance between each line segment sample.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011590292.0A CN112598660B (en) | 2020-12-29 | 2020-12-29 | Automatic detection method for pulp cargo quantity in wharf loading and unloading process |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011590292.0A CN112598660B (en) | 2020-12-29 | 2020-12-29 | Automatic detection method for pulp cargo quantity in wharf loading and unloading process |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112598660A CN112598660A (en) | 2021-04-02 |
| CN112598660B true CN112598660B (en) | 2022-10-21 |
Family
ID=75203295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011590292.0A Active CN112598660B (en) | 2020-12-29 | 2020-12-29 | Automatic detection method for pulp cargo quantity in wharf loading and unloading process |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112598660B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113610465B (en) * | 2021-08-03 | 2023-12-19 | 宁波极望信息科技有限公司 | Production manufacturing operation management system based on internet of things technology |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105427275A (en) * | 2015-10-29 | 2016-03-23 | 中国农业大学 | Filed environment wheat head counting method and device |
| CN109850540A (en) * | 2019-04-03 | 2019-06-07 | 合肥泰禾光电科技股份有限公司 | Apparatus for grouping and group technology |
| CN110047215A (en) * | 2019-05-24 | 2019-07-23 | 厦门莒光科技有限公司 | A kind of article intellectual access storage appts and method |
| CN110930087A (en) * | 2019-09-29 | 2020-03-27 | 杭州惠合信息科技有限公司 | Inventory checking method and device |
| CN111754450A (en) * | 2019-07-16 | 2020-10-09 | 北京京东乾石科技有限公司 | Method, device, equipment and computer readable medium for determining number of objects |
| CN112101389A (en) * | 2020-11-17 | 2020-12-18 | 支付宝(杭州)信息技术有限公司 | Method and device for measuring warehoused goods |
-
2020
- 2020-12-29 CN CN202011590292.0A patent/CN112598660B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105427275A (en) * | 2015-10-29 | 2016-03-23 | 中国农业大学 | Filed environment wheat head counting method and device |
| CN109850540A (en) * | 2019-04-03 | 2019-06-07 | 合肥泰禾光电科技股份有限公司 | Apparatus for grouping and group technology |
| CN110047215A (en) * | 2019-05-24 | 2019-07-23 | 厦门莒光科技有限公司 | A kind of article intellectual access storage appts and method |
| CN111754450A (en) * | 2019-07-16 | 2020-10-09 | 北京京东乾石科技有限公司 | Method, device, equipment and computer readable medium for determining number of objects |
| CN110930087A (en) * | 2019-09-29 | 2020-03-27 | 杭州惠合信息科技有限公司 | Inventory checking method and device |
| CN112101389A (en) * | 2020-11-17 | 2020-12-18 | 支付宝(杭州)信息技术有限公司 | Method and device for measuring warehoused goods |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112598660A (en) | 2021-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109902600B (en) | Road area detection method | |
| CN110490205B (en) | Road scene semantic segmentation method based on full-residual-error hole convolutional neural network | |
| CN111814755A (en) | Multi-frame image pedestrian detection method and device for night motion scene | |
| CN113052834A (en) | Pipeline defect detection method based on convolution neural network multi-scale features | |
| CN114627052A (en) | Infrared image air leakage and liquid leakage detection method and system based on deep learning | |
| CN102087790B (en) | Method and system for low-altitude ground vehicle detection and motion analysis | |
| CN111814739B (en) | Method, device, equipment and storage medium for detecting express package volume | |
| CN114708295B (en) | Logistics parcel separation method based on Transformer | |
| CN112633149A (en) | Domain-adaptive foggy-day image target detection method and device | |
| CN112990065A (en) | Optimized YOLOv5 model-based vehicle classification detection method | |
| CN110728640A (en) | Double-channel single-image fine rain removing method | |
| CN111814754A (en) | Single-frame image pedestrian detection method and device for night scene | |
| CN115147418B (en) | Compression training method and device for defect detection model | |
| CN112598660B (en) | Automatic detection method for pulp cargo quantity in wharf loading and unloading process | |
| CN113111875A (en) | Seamless steel rail weld defect identification device and method based on deep learning | |
| CN115457415A (en) | Target detection method and device based on YOLO-X model, electronic equipment and storage medium | |
| CN111738114A (en) | Vehicle target detection method based on anchor-free accurate sampling remote sensing image | |
| CN115861281A (en) | Anchor-frame-free surface defect detection method based on multi-scale features | |
| CN113869433A (en) | Deep learning method for rapidly detecting and classifying concrete damage | |
| CN114299727B (en) | Traffic flow prediction system based on Internet of things and edge computing and cloud platform | |
| CN115937736A (en) | Small target detection method based on attention and context awareness | |
| CN114399657A (en) | Vehicle detection model training method and device, vehicle detection method and electronic equipment | |
| CN114463300A (en) | Steel surface defect detection method, electronic device, and storage medium | |
| CN117132872A (en) | Intelligent collision recognition system for material transport vehicle on production line | |
| Chan et al. | Raw camera data object detectors: an optimisation for automotive processing and transmission |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: No.7 Ganghua Road, Shibei District, Qingdao, Shandong Province Applicant after: Shandong Port Technology Group Qingdao Co., Ltd. Applicant after: QINGDAO OCEAN SHIPPING TALLY Co.,Ltd. Applicant after: QINGDAO University Address before: No.7 Ganghua Road, Shibei District, Qingdao, Shandong Province Applicant before: PORT OF QINGDAO TECHNOLOGY Co.,Ltd. Applicant before: QINGDAO OCEAN SHIPPING TALLY Co.,Ltd. Applicant before: QINGDAO University |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |