CN114155494B - Belt conveyor deviation monitoring method based on deep learning - Google Patents

Belt conveyor deviation monitoring method based on deep learning Download PDF

Info

Publication number
CN114155494B
CN114155494B CN202210123099.9A CN202210123099A CN114155494B CN 114155494 B CN114155494 B CN 114155494B CN 202210123099 A CN202210123099 A CN 202210123099A CN 114155494 B CN114155494 B CN 114155494B
Authority
CN
China
Prior art keywords
belt
edge
conveying belt
deviation
point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210123099.9A
Other languages
Chinese (zh)
Other versions
CN114155494A (en
Inventor
张梦超
张媛
周满山
于岩
郝妮妮
曹越帅
周丹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University of Science and Technology
Libo Heavy Machine Technology Co Ltd
Original Assignee
Shandong University of Science and Technology
Libo Heavy Machine Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University of Science and Technology, Libo Heavy Machine Technology Co Ltd filed Critical Shandong University of Science and Technology
Priority to CN202210123099.9A priority Critical patent/CN114155494B/en
Publication of CN114155494A publication Critical patent/CN114155494A/en
Application granted granted Critical
Publication of CN114155494B publication Critical patent/CN114155494B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Abstract

The invention provides a belt conveyor deviation monitoring method based on deep learning, which utilizes a general target detection network to detect the edge straight line characteristics of a conveyor belt and realize effective judgment of deviation state. The invention adopts a labeling method with specific labeling requirements to label the edge area of the conveying belt to obtain a conveying belt deviation data set, further trains a general target detection network by using the data set, then uses the trained network for predicting the edge area of the conveying belt, obtains the diagonal position and the equation of the prediction frame of the edge area of the conveying belt by calculating the coordinate positions of four vertexes of the prediction frame, and represents the straight line of the edge of the conveying belt. The effective monitoring of the deviation state of the conveying belt is realized by comparing the distances from the left and right edge straight lines of the conveying belt to the visual field boundary of the camera. The invention simplifies the straight line extraction process of the edge of the conveying belt in a complex environment and powerfully ensures the safe and efficient operation of the belt conveyor.

Description

Belt conveyor deviation monitoring method based on deep learning
Technical Field
The invention relates to the technical field of coal mine equipment intellectualization, in particular to a belt conveyor deviation monitoring method based on deep learning.
Background
The belt conveyor is a preferred device for continuously conveying bulk materials, is a main transportation device for coal mine underground and open-pit coal mining, and is also widely applied to the fields of mines, docks, ports, chemical industry and the like. The conveying belt is an important component of the belt conveyor and plays an important role in bearing materials and transmitting power. Because of the influence of machining precision, installation precision, unbalanced material loading and the like, the conveying belt is always off tracking in the running process. Off tracking is one of the most common faults of a belt conveyor, and can cause a plurality of accidents, such as the scattering of bearing materials; the edge of the conveying belt is abraded or degummed, and the service life of the conveying belt is shortened; the running resistance coefficient of the conveyor is increased, and the energy consumption of the conveyor is increased; at the same time, it is also an important cause of tearing of the conveyor belt. The generation and the expansion of the deviation accident seriously affect the safe and efficient transportation and the green sustainable development of the coal mine.
Document CN113772364A discloses a method for detecting deviation of a belt conveyor and automatically adjusting the deviation of the belt conveyor, which includes a visual detection module and a carrier roller control module. Firstly, a visual detection module arranged in the middle of a belt conveyor is used for respectively acquiring a middle belt running image, a middle front and back running image and middle coal flow information when the belt conveyor works, gamma and fractional order are used for image enhancement processing, Canny edge detection is carried out on the processed image to acquire carrier roller and belt edge information, a deviation angle is calculated through carrier rollers in front and back of the middle of the belt and the belt edge straight line, then the transverse deviation amount is detected according to the information obtained by the middle edge detection, and the algorithm is complex. At present, a coal mine belt deviation monitoring mode mainly comprises a deviation switch sensor, the deviation sensor determines deviation amount and processes deviation, however, the actual coal mine production environment is complex, more deviation switch sensors are required to be installed on two sides of the running belt, the detection mode is in the running process of the belt, magnetic interference of adjacent sensors mainly exists, output is in an unstable state, data interpretation is influenced, the belt deviation switch and a protection device further need to be stopped regularly for calibration and testing, an underground conveying belt is long, manpower and time are required to be input for detection, and the underground coal mine production efficiency can be influenced.
Disclosure of Invention
In order to solve the technical problems, the invention provides a belt deviation monitoring method of a belt conveyor based on deep learning, which comprises the steps of carrying out data marking on an edge area of the belt conveyor by adopting a marking method with specific marking requirements to obtain a belt deviation data set, further training a general target detection network by utilizing the data set, then using the trained network for predicting the edge area of the belt conveyor, calculating coordinate positions of four vertexes of a prediction frame to obtain a diagonal position and an equation of the prediction frame of the edge area of the belt conveyor, and representing a straight line of the edge of the belt conveyor according to the diagonal position and the equation. The effective monitoring of the deviation state of the conveying belt is realized by comparing the distances from the left and right edge straight lines of the conveying belt to the visual field boundary of the camera. The invention utilizes the general target detection network to detect the edge straight line characteristics of the conveying belt and realize effective judgment of the deviation state, simplifies the edge straight line extraction process of the conveying belt in a complex environment and powerfully ensures the safe and efficient operation of the belt conveyor.
The method in the invention comprises the following steps: step 1, arranging a camera along a belt conveyor, and acquiring running images of the belt conveyor in real time to establish a conveying belt deviation image data set; step 2, extracting and labeling the edge characteristics of the conveyer belt by utilizing a deep learning technology such as LabelImg and the like based on the image data set established in the step 1, labeling a labeling frame on each of the left edge and the right edge of the conveyer belt in each image, aligning the edge of the conveyer belt with one diagonal line of the corresponding labeling frame, and dividing the labeled data set into a training set, a verification set and a test set for the subsequent deep learning target detection network training and calling; step 3, building a target detection network model based on deep learning, setting the edge area of the conveying belt as a detection target, inputting the training set and the verification set in the data set built in the step 2 into the built target detection network model for training until the network converges, and obtaining a training weight and a detection model; step 4, based on the prediction of the target detection network which takes the edge area of the conveying belt as the target in the step 3, generating a prediction frame, outputting the parameters of the prediction frame, and continuously improving the output of the prediction frame through the target detection network so as to enable the prediction frame to have the capability of detecting straight lines; step 5, on the basis of the straight line of the edge of the conveying belt detected in the step 4, establishing a virtual reference line in the visual field of the camera, wherein the virtual reference line is parallel to the x axis, the distance between the virtual reference line and the x axis is marked as delta y, the virtual reference line is respectively intersected with the left and the right of the boundary of the visual field of the camera to form points L and R, is respectively intersected with the left and the right edges of the conveying belt to form points M and N, the LM is the line segment distance between the L point and the M point, and the RN is the line segment distance between the R point and the N point; step 6, calculating the lengths of LM and NR respectively, determining the position of the conveying belt in the visual field range of the camera, calculating the absolute difference value of LM and NR in real time through an algorithm, and comparing the absolute difference value with a threshold value in real time, so that the determination of the deviation state and the deviation amount can be realized; and 7, deploying the improved target detection to an industrial field, and realizing the linkage control and adjustment of the deviation.
Further, the network model based on step 3 is a YOLO network model or a common target detection model such as an RCNN network model or an SSD network model.
Further, the parameters in the step 4 are the center point coordinate of the prediction frame, the length of the prediction frame and the width value, and based on the center point coordinate of the prediction frame, the length of the prediction frame and the width value, four vertex coordinates of the prediction frame can be obtained, so that both a linear equation of a straight line where a diagonal line of a left labeling frame on the left edge of the conveyor belt is located and a linear equation of a straight line where a diagonal line of a right labeling frame on the right edge of the conveyor belt is located can be obtained through calculation, namely the straight lines where the left edge and the right edge of the conveyor belt are located.
Further, the equation of the straight line where the edge of the conveying belt is located is calculated as follows, and the coordinate of the point a is marked as (x) in the left marking frame of the left edge of the conveying belta，ya) The b point coordinate is (x)b，yb) The coordinate of the point c is marked as (x) in the right marking frame at the right edge of the conveyer beltc，yc) D coordinates of point are (x)d，yd) Then, then
The equation for the diagonal ab coinciding with the left conveyor edge in the left label box is:
Figure 586239DEST_PATH_IMAGE001
the equation for the diagonal cd coinciding with the right conveyor edge in the right label box is:
Figure 732049DEST_PATH_IMAGE002
。
further, based on the line segment distance between the L point and the M point described in step 5, the line segment distance between the R point and the N point is calculated as
Figure 842088DEST_PATH_IMAGE003
I.e. the distance of the left and right edges of the conveyor belt from the view boundary of the view camera, and W is the width of the camera view.
Further, the algorithm based on step 6 is
Figure 465967DEST_PATH_IMAGE004
Judging that the conveying belt is not off tracking;
when in use
Figure 374755DEST_PATH_IMAGE005
I.e. to determine the deviation of the conveyor belt, wherein DLMIs the distance between the points L and M, DNRτ is a threshold value for off-tracking determination, and is a distance between the R point and the N point.
After adopting the technical scheme, compared with the prior art, the invention has the following beneficial effects: 1. the invention provides a belt conveyor deviation monitoring method based on deep learning.
2. The method designed by the invention provides a new method for rapidly and accurately detecting the edge of the conveying belt in a complex scene.
3. The algorithm in the method designed by the invention has strong expansibility, can be migrated to other scenes only by training a small amount of data sets, and has strong adaptability.
4. The method greatly simplifies the process of linear feature extraction in a complex environment, is beneficial to realizing the intelligentization of the belt conveyor and the unmanned and energy-saving level of a coal mine, and realizes green energy-saving sustainable development while ensuring the safe and efficient transportation of the belt conveyor.
The technical scheme provided by the invention provides a linear detection algorithm based on a general target detection network aiming at the problem of complex operation of visual-based extraction of the edge characteristics of the conveying belt under a complex environment, and a deviation judgment method is planned and designed again, so that the rapid and accurate extraction of the edge characteristics of the conveying belt is realized, the deviation detection accuracy of the conveying belt of the belt conveyor is improved, the adaptability is strong, the problems of rapid characteristic extraction and deviation judgment of the edge of the conveying belt of the belt conveyor under a complex background are solved, and the visualization of the deviation state detection of the conveying belt is realized.
Drawings
The above and other objects and advantages of the present invention will become more fully apparent from the following detailed description taken in conjunction with the accompanying drawings.
Fig. 1 is a schematic view of a normal operation state of a conveyor belt.
Fig. 2 is a schematic diagram of a rightward deviation state of the conveyor belt.
Fig. 3 is a schematic diagram of the left deviation state of the conveying belt.
Fig. 4 is a schematic diagram of determining deviation of the conveyor belt.
Reference numerals: 1-camera view, 2-virtual reference line, 3-conveyor actual position, 4-conveyor reference position, 5-prediction box.
Detailed Description
The present invention will be described in further detail with reference to examples. The advantages and features of the present invention will become more apparent as the description proceeds. These examples are illustrative only and do not limit the scope of the present invention in any way. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention, and that such changes and modifications may be made without departing from the spirit and scope of the invention.
Terms such as "comprising" and "comprises" mean that, in addition to having components which are directly and explicitly stated in the description and claims, the solution of the invention does not exclude other components which are not directly or explicitly stated. In the description herein, directional terms such as "upper", "lower", "front", "rear", etc. are used, it being understood that these directional terms are relative concepts that are used for relative positional description and clarification, and that the corresponding specific orientation may vary accordingly as the orientation of the camera varies.
The invention provides a belt conveyer belt deviation monitoring method based on deep learning, which comprises the steps of carrying out data marking on a belt edge area by adopting a marking method with specific marking requirements to obtain a belt deviation data set, further utilizing the data set to train a general target detection network, then using the trained network for the prediction of the belt edge area, obtaining a diagonal position and an equation of a belt edge area prediction frame 5 by calculating four vertex coordinate positions of the prediction frame 5, and representing a belt edge straight line. The effective monitoring of the deviation state of the conveying belt is realized by comparing the distances from the straight lines at the left edge and the right edge of the conveying belt to the boundary of the camera visual field 1 respectively. The invention utilizes the general target detection network to detect the edge straight line characteristics of the conveying belt and realize effective judgment of the deviation state, simplifies the edge straight line extraction process of the conveying belt in a complex environment and powerfully ensures the safe and efficient operation of the belt conveyor.
The method in the invention comprises the following steps: step 1, arranging a camera along a belt conveyor, and acquiring running images of the belt conveyor in real time to establish a conveying belt deviation image data set;
step 2, extracting and labeling the edge characteristics of the conveying belt by using a deep learning technology based on the image data set established in the step 1, labeling a labeling frame on the left edge and the right edge of the conveying belt in each image respectively, aligning the edge of the conveying belt with a diagonal line of the corresponding labeling frame, and dividing the labeled data set into a training set, a verification set and a test set for training and calling of a subsequent deep learning target detection network;
step 3, building a target detection network model based on deep learning, setting the edge area of the conveying belt as a monitoring target, inputting the training set and the verification set in the data set built in the step 2 into the built target detection network model for training until the network converges, and obtaining a training weight and a detection model; the network model based on the step 3 is a general target detection network such as a YOLO network model, an RCNN network model or an SSD network model.
Step 4, based on the prediction of the target detection network with the edge area of the conveying belt as the target in the step 3, generating a prediction frame 5 as shown in fig. 2, outputting parameters of the prediction frame 5, and continuously improving the output of the target detection network through the target detection network to enable the target detection network to have the capability of detecting straight lines; the parameters are the center point coordinate of the prediction frame 5, the length of the prediction frame 5 and the width value, and based on the center point coordinate of the prediction frame 5, the length of the prediction frame 5 and the width value, four vertex coordinates of the prediction frame 5 can be obtained, as shown in fig. 4, a straight line equation of a straight line where a left marking frame diagonal line of the left edge of the conveyer belt is located and a straight line equation of a straight line where a right marking frame diagonal line of the right edge of the conveyer belt is located can be obtained through calculation, namely the straight lines where the left edge and the right edge of the conveyer belt are located; the equation of the straight line where the edge of the conveying belt is located is calculated as follows, and the coordinate of the point a is recorded in a left marking frame of the left edge of the conveying belt as (x)a，ya) The b point coordinate is (x)b，yb) The coordinate of the point c is marked as (x) in the right marking frame at the right edge of the conveyer beltc，yc) D coordinates of point are (x)d，yd) Then, the first step is executed,
the equation for the diagonal ab coinciding with the left conveyor edge in the left label box is:
Figure 629150DEST_PATH_IMAGE006
the equation for the diagonal cd coinciding with the right conveyor edge in the right label box is:
Figure 492064DEST_PATH_IMAGE007
。
the size of the prediction box 5 does not affect the detection result because the equation of the straight line of the edge of the conveyor belt is determined by any two points on the straight line.
Step 5, on the basis of the straight line of the edge of the conveying belt detected in the step 4, as shown in fig. 3, establishing a virtual reference line 2 in the camera view 1, wherein the virtual reference line 2 is parallel to the x axis, the distance between the virtual reference line 2 and the x axis is marked as Δ y, the virtual reference line 2 intersects the left and right sides of the boundary of the camera view 1 respectively as points L and R, intersects the left and right edges of the conveying belt respectively as points M and N, LM is the line segment distance between the point L and the point M, and RN is the line segment distance between the point R and the point N; wherein the line segment distance between the L point and the M point and the line segment distance between the R point and the N point are calculated as
Figure 654055DEST_PATH_IMAGE008
I.e. the distance of the left and right edges of the conveyor belt from the view boundary of the view camera, and W is the width of the camera view.
Step 6, calculating the lengths of LM and NR respectively, determining the position of the conveying belt in the visual field range of the camera, calculating the absolute difference value of LM and NR in real time through an algorithm, and comparing the absolute difference value with a threshold value in real time, so that the determination of the deviation state and the deviation amount can be realized;
when in use
Figure 154700DEST_PATH_IMAGE009
Judging that the conveying belt is not off tracking;
when in use
Figure 642313DEST_PATH_IMAGE010
I.e. to determine the deviation of the conveyor belt, wherein DLMIs the distance between the points L and M, DNRτ is a threshold value for off-tracking determination, and is a distance between the R point and the N point.
And 7, deploying the improved target detection to an industrial field, and realizing the linkage control and adjustment of the deviation.
In the embodiment, as shown in fig. 1, fig. 2 and fig. 3, three common operation conditions of the conveyor belt are shown, namely normal operation, right deviation and left deviation. In fig. 1, a coordinate system as shown in fig. 2 is established in the upper left corner of the field of view 1 of the camera, and a virtual reference line 2 is established, which is parallel to the x-axis and has a distance Δ y from the x-axis. The virtual reference line 2 and the camera view 1 border have left and right intersections L and R, respectively, and the virtual reference line and the conveyor belt edge have intersections M and N, respectively. On the basis of ensuring that the camera is installed in a centered manner and ignoring the vibration of the frame, the state of the deviation can be judged according to the following rules:
when in use
Figure 992523DEST_PATH_IMAGE009
Judging that the conveying belt is not off tracking;
when in use
Figure 660002DEST_PATH_IMAGE010
Immediately judging the deviation of the conveyor belt, wherein DLMIs the distance between the points L and M, DNRτ is a threshold value for off-tracking determination, and is a distance between the R point and the N point.
The off-tracking determination threshold is related to the bandwidth and the image resolution, and the off-tracking determination threshold is set related to the installation position of the camera, the visual field range of the camera and the resolution. In the embodiment, a scene that the camera is arranged in the middle of the conveyor is shown, the threshold value for tau deviation judgment is set to be 80 pixels, and the distance between the left side edge and the right side edge of the conveyor belt and the left edge and the right edge of the image are equal, so that the difference value of the distance between the two edges of the conveyor belt and the edge of the visual field is accurately given while the two edges of the conveyor belt are well identified, and whether deviation occurs or not is judged. In some scenarios, the camera cannot be placed in the middle of the conveying belt in a centered manner, and needs to be installed on one side of the conveying belt, and the threshold setting needs to be set according to actual conditions.
The invention is based on the Yolo target detection network, and realizes the detection of the edge area of the conveying belt by correspondingly modifying the output prediction result of the network, so that the conveying belt has the capability of detecting straight lines. Yolo takes CSPDarknet _ SPP as a backbone feature extraction network. The input image is sliced through the Focus module, and the width and height information of the image is integrated to a channel space through extraction and recombination of pixel values, so that down-sampling operation is realized in a phase-changing manner. The Focus module is used for reducing the calculation amount of the network under the condition that no information is lost; by adopting the cross-stage local network CSPDarknet, the problem of repeated network optimization gradient information in the process of extracting the main features of the large convolutional neural network can be effectively solved, and the change of the gradient is integrated into the feature diagram from beginning to end, so that the parameter quantity and the calculated quantity of the model are reduced, the reasoning speed and the accuracy are ensured, and the size of the model is reduced; an SPP + FPN + PAN module is used as a feature enhancement part, so that the diversity and robustness of feature information are improved; meanwhile, the models with different complexities are flexibly configured by using the network channel number and the network layer number adjusting factor similar to the EfficientNet, and the models with the sequentially improved S-M-L-X complexities are obtained.
The Loss function Loss of the Yolo target detection network consists of three parts, namely positioning Loss LIoUSorting loss LclsAnd a confidence loss LobjThe calculation formula is shown as the following formula:
Loss=λ1×LIoU+λ2×Lcls+λ3×Lobj
λ1-3∈（0，1 ]in the present invention, λ is a distribution coefficient1=λ2=λ3=1。
Loss of positioning LIoUThe method is also called a bounding box regression loss and is used for characterizing the distance between the prediction box 5 and the real box, and is realized by calculating the ratio of the intersection area of the two bounding boxes to the phase-parallel area (intersection ratio IoU), wherein the larger the intersection ratio is, the higher the coincidence degree between the prediction box 5 and the real box is, the smaller the distance between the two boxes is, and the smaller the loss is.
The GIoU function is used in Yolo to calculate the localization loss, which is an upgraded version of IoU, IoU =1, L when the prediction box 5 completely coincides with the real boxGIoU=0, IoU =0 and L is greater when the prediction box 5 is completely misaligned with the real box and is further awayGIoU=2, this solves the problem of the gradient disappearance present in the function.
Performing transfer learning by using a pre-training weight of Yolo on a COCO2017 data set, setting the batch size to be 4, and training 100 epochs; by adopting a Moscaic data enhancement mode, 4 images are randomly selected, randomly zoomed and randomly spliced, so that a data set is greatly enriched.
The embodiments in the above embodiments can be further combined or replaced, and the embodiments are only used for describing the preferred embodiments of the present invention, and do not limit the concept and scope of the present invention, and various changes and modifications made to the technical solution of the present invention by those skilled in the art without departing from the design idea of the present invention belong to the protection scope of the present invention.

Claims (5)

1. A belt conveyor deviation monitoring method based on deep learning is characterized by comprising the following steps:
step 1, arranging a camera along a belt conveyor, and acquiring running images of the belt conveyor in real time to establish a conveying belt deviation image data set;
step 2, extracting and labeling the edge characteristics of the conveying belt by using data labeling software based on the image data acquired in the step 1, labeling a labeling frame on the left edge and the right edge of the conveying belt in each image respectively, aligning the edge of the conveying belt with a diagonal line of the corresponding labeling frame, and dividing a labeled data set into a training set, a verification set and a test set for training and calling of a subsequent deep learning target detection network;
step 3, a target detection network model based on deep learning is built, so that the detection of the edge area of the conveying belt is realized, the edge area of the conveying belt is set as a detection target, the training set and the verification set in the data set built in the step 2 are input into the built target detection network model for training until the network is converged, and the training weight and the detection model are obtained;
step 4, based on the prediction of the target detection network taking the edge area of the conveying belt as a target in the step 3, generating a prediction frame, outputting parameters of the prediction frame, and improving the output result of the target detection network to enable the target detection network to have the capability of detecting straight lines;
step 5, on the basis of the straight line of the edge of the conveying belt detected in the step 4, establishing a virtual reference line in the visual field of the camera, wherein the virtual reference line is parallel to the x axis, the distance between the virtual reference line and the x axis is marked as delta y, the virtual reference line is respectively intersected with the left and the right of the boundary of the visual field of the camera to form points L and R, is respectively intersected with the left and the right edges of the conveying belt to form points M and N, the LM is the line segment distance between the L point and the M point, and the RN is the line segment distance between the R point and the N point;
step 6, respectively calculating the lengths of LM and NR, determining the position of the conveying belt in the visual field range of the camera, calculating the absolute difference value of LM and NR in real time through an algorithm, and comparing the absolute difference value with a threshold value in real time to determine the deviation state and the deviation amount, wherein the deviation determination threshold value is set to be related to the installation position of the camera, the visual field range of the camera and the image resolution;
step 7, deploying the improved target detection to an industrial field to realize the linkage control and adjustment of deviation;
the parameters in the step 4 are the center point coordinate of the prediction frame, the length of the prediction frame and the width value, and based on the center point coordinate of the prediction frame, the length of the prediction frame and the width value, the four vertex coordinates of the prediction frame can be obtained, so that the linear equation of the straight line of the diagonal line of the left marking frame at the left edge of the conveyer belt and the linear equation of the straight line of the diagonal line of the right marking frame at the right edge of the conveyer belt can be obtained through calculation, namely the straight lines of the left edge and the right edge of the conveyer belt.
2. The belt conveyor belt deviation monitoring method based on deep learning of claim 1, wherein the method comprises the following steps: the network model based on the step 3 is a general target detection network of a YOLO network model or an RCNN network model or an SSD network model.
3. The belt conveyor belt deviation monitoring method based on deep learning of claim 1, wherein the method comprises the following steps: the equation of the straight line where the edge of the conveying belt is located is calculated as follows, and the coordinate of the point a is recorded in a left marking frame of the left edge of the conveying belt as (x)a，ya) The b point coordinate is (x)b，yb），The coordinate of the point c is marked as (x) in the right marking frame at the right edge of the conveyer beltc，yc) D coordinates of point are (x)d，yd) Then, then
The equation for the diagonal ab coinciding with the left conveyor edge in the left label box is:
Figure 708152DEST_PATH_IMAGE001
the equation for the diagonal cd coinciding with the right conveyor edge in the right label box is:
Figure 902242DEST_PATH_IMAGE002
。
4. the belt conveyor belt deviation monitoring method based on deep learning of claim 3, wherein the method comprises the following steps: based on the line segment distance between the L point and the M point in step 5, the line segment distance between the R point and the N point is calculated as
Figure 317043DEST_PATH_IMAGE003
I.e. the distance of the left and right edges of the conveyor belt from the view boundary of the view camera, where W is the width of the camera view.
5. The belt deviation monitoring method of the belt conveyor based on the deep learning of claim 4, wherein the method comprises the following steps: the algorithm based on the step 6 is
When in use
Figure 259591DEST_PATH_IMAGE004
Namely, the deviation of the conveying belt is judged,
when in use
Figure 510575DEST_PATH_IMAGE005
I.e. to determine the deviation of the conveyor belt, wherein DLMIs the distance between the points L and M, DNRIs a point RAnd a distance between the N points, wherein tau is a threshold value for deviation judgment.
CN202210123099.9A 2022-02-10 2022-02-10 Belt conveyor deviation monitoring method based on deep learning Active CN114155494B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210123099.9A CN114155494B (en) 2022-02-10 2022-02-10 Belt conveyor deviation monitoring method based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210123099.9A CN114155494B (en) 2022-02-10 2022-02-10 Belt conveyor deviation monitoring method based on deep learning

Publications (2)

Publication Number Publication Date
CN114155494A CN114155494A (en) 2022-03-08
CN114155494B true CN114155494B (en) 2022-05-17

Family

ID=80450794

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210123099.9A Active CN114155494B (en) 2022-02-10 2022-02-10 Belt conveyor deviation monitoring method based on deep learning

Country Status (1)

Country Link
CN (1) CN114155494B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114655655A (en) * 2022-03-09 2022-06-24 南京北路软件技术有限公司 Conveyor belt deviation detection method based on UNet network
CN116167991B (en) * 2023-02-15 2023-09-08 中科微至科技股份有限公司 DeepLabv3+ based belt edge line detection method
CN116309565B (en) * 2023-05-17 2023-08-04 山东晨光胶带有限公司 High-strength conveyor belt deviation detection method based on computer vision
CN117670916B (en) * 2024-01-31 2024-04-12 南京华视智能科技股份有限公司 Coating edge detection method based on deep learning

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112257793A (en) * 2020-10-26 2021-01-22 大连理工大学 Remote traffic sign detection method based on improved YOLO v3 algorithm
CN113112485A (en) * 2021-04-20 2021-07-13 中冶赛迪重庆信息技术有限公司 Belt conveyor deviation detection method, system, equipment and medium based on image processing

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10083496B2 (en) * 2012-05-22 2018-09-25 Cognex Corporation Machine vision systems and methods with predictive motion control
CN110697373B (en) * 2019-07-31 2021-04-06 湖北凯瑞知行智能装备有限公司 Conveying belt deviation fault detection method based on image recognition technology
CN113283344B (en) * 2021-05-27 2024-03-12 中国矿业大学 Mining conveyor belt deviation detection method based on semantic segmentation network
CN113378952A (en) * 2021-06-22 2021-09-10 中冶赛迪重庆信息技术有限公司 Method, system, medium and terminal for detecting deviation of belt conveyor
CN113658136B (en) * 2021-08-17 2023-10-13 燕山大学 Deep learning-based conveyor belt defect detection method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112257793A (en) * 2020-10-26 2021-01-22 大连理工大学 Remote traffic sign detection method based on improved YOLO v3 algorithm
CN113112485A (en) * 2021-04-20 2021-07-13 中冶赛迪重庆信息技术有限公司 Belt conveyor deviation detection method, system, equipment and medium based on image processing

Also Published As

Publication number Publication date
CN114155494A (en) 2022-03-08

Similar Documents

Publication Publication Date Title
CN114155494B (en) Belt conveyor deviation monitoring method based on deep learning
CN109230351B (en) Method for identifying abnormal operation of belt conveyor
CN110697373B (en) Conveying belt deviation fault detection method based on image recognition technology
Yang et al. On-line conveyor belts inspection based on machine vision
Brilakis et al. Automated vision tracking of project related entities
CN110065790B (en) Method for detecting blockage of coal mine belt transfer machine head based on visual algorithm
CN109305534A (en) Coal wharf's belt conveyor self-adaptation control method based on computer vision
CN110163852B (en) Conveying belt real-time deviation detection method based on lightweight convolutional neural network
Dang et al. Automatic tunnel lining crack evaluation and measurement using deep learning
CN110147748B (en) Mobile robot obstacle identification method based on road edge detection
CN109765570A (en) A kind of Auto-rectification Device for Belt Conveyer detection method based on laser radar
CN112862806B (en) Belt conveyor slip detection method, system, medium and electronic terminal
Zhu et al. An accurate approach for obtaining spatiotemporal information of vehicle loads on bridges based on 3D bounding box reconstruction with computer vision
CN105957107A (en) Pedestrian detecting and tracking method and device
Zhang et al. A deep learning-based method for deviation status detection in intelligent conveyor belt system
CN113306991A (en) Coal conveyor monitoring and management system based on stereoscopic vision
CN114926733A (en) Conveyor belt tearing detection method for improved regional convolutional neural network
CN115171051A (en) Online detection method and system for tearing of edge of conveying belt
CN111539927A (en) Detection process and algorithm of automobile plastic assembly fastening buckle lack-assembly detection device
Zhang et al. A new paradigm for intelligent status detection of belt conveyors based on deep learning
Fan et al. Long-awaited next-generation road damage detection and localization system is finally here
Xu et al. Measuring bulk material flow—Incorporating RFID and point cloud data processing
Pandey et al. Autopilot control unmanned aerial vehicle system for sewage defect detection using deep learning
CN116152730A (en) Method and device for detecting multiple running states of belt conveyor based on deep learning
CN111062968A (en) Robot path skeleton extraction method based on edge scanning and centerline extraction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant