CN115908272A - Method and system for automatically detecting belt tearing state based on vision technology - Google Patents
Method and system for automatically detecting belt tearing state based on vision technology Download PDFInfo
- Publication number
- CN115908272A CN115908272A CN202211323534.9A CN202211323534A CN115908272A CN 115908272 A CN115908272 A CN 115908272A CN 202211323534 A CN202211323534 A CN 202211323534A CN 115908272 A CN115908272 A CN 115908272A
- Authority
- CN
- China
- Prior art keywords
- belt
- tearing
- data
- detection
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005516 engineering process Methods 0.000 title claims abstract description 61
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000001514 detection method Methods 0.000 claims abstract description 107
- 238000012544 monitoring process Methods 0.000 claims abstract description 22
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 230000002159 abnormal effect Effects 0.000 claims abstract description 5
- 238000013135 deep learning Methods 0.000 claims abstract description 5
- 230000000007 visual effect Effects 0.000 claims description 42
- 230000008569 process Effects 0.000 claims description 9
- 230000006835 compression Effects 0.000 claims description 8
- 238000007906 compression Methods 0.000 claims description 8
- 238000004873 anchoring Methods 0.000 claims description 6
- 238000004519 manufacturing process Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 238000013499 data model Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000003672 processing method Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000007689 inspection Methods 0.000 claims 1
- 238000002372 labelling Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 239000003245 coal Substances 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 229910000831 Steel Inorganic materials 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
- B65G43/00—Control devices, e.g. for safety, warning or fault-correcting
- B65G43/06—Control devices, e.g. for safety, warning or fault-correcting interrupting the drive in case of driving element breakage; Braking or stopping loose load-carriers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Control Of Conveyors (AREA)
Abstract
The invention discloses a method and a system for automatically detecting a belt tearing state based on a vision technology, which comprises the steps of constructing a daily belt state data set on a belt conveyor, and storing a belt monitoring picture in 8 months in the data set; analyzing and comparing the conventional daily state data set of the belt, preprocessing and marking the image frame of the torn belt and establishing a Yolov3 target detection model; and (3) putting the picture shot in real time into a YOLOv3 target detection model, monitoring the real-time monitoring belt state picture through a vision technology, and performing abnormal recognition alarm when the real-time monitoring belt state is the same as the data mark state in the data set. The management of the snapshot picture information under the abnormal condition is identified through an intelligent vision technology, the two symmetrical edges of the belt are detected by adopting the computer vision technology for edge tearing, and whether tearing exists is judged according to the distance between the two detected edges; the non-edge tearing adopts a deep learning technology, and the belt tearing detection is realized by positioning the cracks of the belt.
Description
Technical Field
The invention relates to the technical field of visual recognition, in particular to a method and a system for automatically detecting a belt tearing state based on a visual technology.
Background
In the process of belt operation, the tearing of the belt is always a main fault source in the process of mechanical operation, but the tearing of the belt is always generated based on various reasons and is difficult to realize fixed detection or prediction, so that the conventional method can only adjust and repair the deviated belt according to camera monitoring in the process of belt operation, timely adjustment and short-time repair are difficult to realize, and the loss of mechanical capacity and equipment maintenance is caused.
Therefore, the automatic detection of the belt edge and the non-edge is realized based on the computer vision technology, the normal area of the belt edge is drawn according to the actual requirement of a user, and the tearing is judged if the detected belt edge is not in the normal area. The belt tearing automatic detection with different tolerance degrees is realized, and the belt tearing state online intelligent monitoring is finally realized.
Disclosure of Invention
This section is for the purpose of summarizing some aspects of embodiments of the invention and to briefly introduce some preferred embodiments. In this section, as well as in the abstract and title of the application, simplifications or omissions may be made to avoid obscuring the purpose of the section, the abstract and the title, and such simplifications or omissions are not intended to limit the scope of the invention.
The present invention has been made in view of the above and/or other problems associated with the prior art method for automatically detecting a torn state of a belt based on a visual technique.
Therefore, the problem to be solved by the present invention is how to provide a method for automatically detecting a torn state of a belt based on a visual technology.
In order to solve the technical problems, the invention provides the following technical scheme: a method for automatically detecting the tearing state of a belt based on a visual technology comprises the following steps,
constructing a daily belt state data set on the belt conveyor, and storing belt monitoring pictures in 8 months in the data set;
analyzing and comparing the conventional daily state data set of the belt, preprocessing and marking the image frame of the torn belt and establishing a Yolov3 target detection model;
and (3) putting the picture shot in real time into a YOLOv3 target detection model, monitoring the real-time monitoring belt state picture through a vision technology, and performing abnormal recognition alarm when the real-time monitoring belt state is the same as the data mark state in the data set.
As a preferable scheme of the method for automatically detecting the tearing state of the belt based on the visual technology, the method comprises the following steps: the belt state is that the belt tearing frequency of the belt machine with the belt machine in 8 months is known, and the belt machine is processed by a common processing method and collected on site;
the belt state data set is expressed in the form of a picture after being shot by a camera.
As a preferable scheme of the method for automatically detecting the tearing state of the belt based on the visual technology, the method comprises the following steps: the data marking process is to carry out preprocessing on the data set;
the preprocessing is to record the tearing state of the belt in the data set picture, wherein the tearing state of the belt is divided into edge tearing and non-edge tearing;
the edge tearing adopts a computer vision technology to detect two symmetrical edges of the belt, and whether tearing exists is judged according to the distance between the two detected edges;
the non-edge tearing adopts a deep learning technology, and the belt tearing detection is realized by positioning the crack of the belt;
and the data marking is to mark the belt offset position by using label tool labelImg software.
As a preferable scheme of the method for automatically detecting the tearing state of the belt based on the visual technology, the method comprises the following steps: the YOLOv3 target detection model divides an image into 7x7 grids during detection, the network where the center of an object to be detected falls is responsible for predicting the belt tearing condition, each grid predicts 2 boundary frames, 98 boundary frames are predicted in total, a boundary frame higher than the actual boundary frame IOU of the object is selected from the two predicted boundary frames of each grid as output during training, and each predicted boundary frame consists of an offset value of a center coordinate relative to the upper left corner of the predicted grid and the width, height and confidence coefficient after normalization;
the confidence coefficient is calculated by the formula
A＝P r (o)*IOU
Where A is the confidence, P r (o) is whether there is a target in the grid, when there is a target P r (o) is 1, P is absent r (o) is 0, IOU is the mesh overlap, and the calculation method is
Wherein GT refers to the prediction grid, PB refers to the current detection grid, and based on this, the coordinate transformation between the predicted value and the actual value of the grid is
Wherein c is x ，c y ，c w ，c h Is the center coordinate and width and height of the prediction box, b x ，b y Is the offset of the central coordinate of the prediction frame relative to the coordinate of the upper left corner of the prediction grid, and the delta function is b x ，b y Offset normalized to 1-0, b w ，b h Is the wide and high scaling size of the prediction box compared to the anchor grid, d x And d y Is the width and height of the anchoring grid;
the anchoring grid is used as a frame in the network model for detecting each target in the target detection, and is also called a prior frame;
the YOLOv3 target detection model divides an image into a 7x7 grid division mode during detection, because the tearing state can be quickly embodied through the 7x7 grid division mode no matter the belt is torn at the edge or is not torn at the edge, the 8 x 8 grid division mode is opposite to the belt edge division mode, the importance of the position, which is easy to tear and protrudes, of the belt edge in data concentration is easily reduced, and the position offset of the edge tearing and the non-edge tearing in the specific implementation situation is difficult to find through the 6x 6 grid division mode.
As a preferable scheme of the method for automatically detecting the tearing state of the belt based on the visual technology, the method comprises the following steps: the real-time picture detection result in the data model stores related information into a database every 8 hours, and for image frames with belt tearing, the detailed information of the image frames is stored into the database to be used as a data source for belt tearing statistics except for being displayed on a system homepage as a warning signal, so that production is guided.
The present invention has been made in view of the above and/or other problems with the prior art systems for automatic belt tear condition detection based on visual technology.
Therefore, the problem to be solved by the present invention is how to provide a system for automatic detection of belt tearing state based on visual technology.
In order to solve the technical problems, the invention provides the following technical scheme: a system for automatically detecting the tearing state of a belt based on a visual technology comprises,
the device comprises a data input module, a data detection module, a belt adjusting module and a belt running module;
the data input module is used for shooting the belt conveyor, and shooting data are put into the data detection module in a mode of forming pictures frame by frame after shooting;
the data detection module is used for preprocessing data entering the module, then establishing a Yolov3 target detection model, detecting belt tearing conditions of the preprocessed pictures, and sending corresponding signals to the belt adjustment module when the tearing conditions are found;
the belt adjusting module is composed of a PLC structure and is used for sending a corresponding adjusting signal after the data monitoring module monitors that the belt is torn, and the belt adjusting module transcribes the signal into an instruction through PLC programming and sends the instruction to the belt operating module;
the belt operation module is used for belt conveyor operation, and when the PLC sends a belt conveyor operation adjustment operation instruction, the corresponding adjustment is carried out on the operation represented by the specific code;
the arrangement sequence of the system for automatically detecting the belt tearing state based on the vision technology is that data enters from the data input module, the data is detected by the data detection module and then is sent out by the belt adjusting module to be adjusted, and the belt operation module carries out specific belt conveyor operation based on the instruction.
As a preferable scheme of the system for automatically detecting the tearing state of the belt based on the visual technology, the system comprises: the data input module is used for arranging the network camera above the belt conveyor to be adjusted to a proper angle according to field conditions, and connecting the camera with a network to obtain the transportation condition on the belt conveyor in real time.
As a preferable scheme of the system for automatically detecting the tearing state of the belt based on the visual technology, the system comprises: the data detection module establishes a YOLOv3 target detection model based on a data set, when a belt tearing condition occurs on the belt conveyor, the data detection model inputs an image frame of belt tearing into the target detection model, and if the belt tearing condition is detected, a control instruction is sent to the lower computer PLC through the Ethernet, the corresponding belt conveyor audible and visual alarm is controlled to give an alarm, and related workers are prompted to timely take the belt tearing condition.
As a preferable solution of the system for automatically detecting a torn state of a belt based on a visual technology, the system of the invention comprises: the data detection module is used for detecting a visual technology based on a YOLOv3 target detection model, the conditions of large-size belt tearing, medium-size belt tearing and small-size belt tearing are respectively detected on three scales of 13x13, 26x26 and 52x52 by utilizing the characteristics of a low layer and a high layer, a network is subjected to 5 times of down-sampling with the step length of 2 in the characteristic extraction process, a 13x13 characteristic extracted picture is obtained from an 416x416 original picture, each grid has a larger receptive field, a target with a larger size is detected on the scale of 13x13, and similarly, each grid on the scale of 52x52 corresponds to a smaller range of an original image and is used for detecting a target with a smaller scale;
the receptive field is the size of an area mapped by pixel points on a characteristic graph output by each layer of the convolutional neural network on an input graph, and the interpretation of the popular points is that one point on the characteristic graph corresponds to the area on the input graph;
the image enhancement comprises image size compression, image turning and image rotation;
the image size compression is to compress the length of the picture to a specified length, and the width is compressed in equal proportion with the length of the picture;
the image turning is to obtain a picture symmetrical to the picture in a mirror image turning mode;
the image rotation is that when the center of the image is rotated for a certain angle by taking the original point as an original point, part of the corner part of the picture exceeds the range of the original picture, and the other part of the corner part of the picture is contracted into the picture to leave a blank, and the blank part of the text is filled with a black background;
the image enhancement effect is to increase the expression of the data set of the area where the belt tearing is likely to occur, reduce the time required for detection and increase the detection effect.
As a preferable solution of the system for automatically detecting a torn state of a belt based on a visual technology, the system of the invention comprises: the belt conveyor running module can perform three specific operations according to a PLC instruction, namely normal running, half-speed running and stopping running;
the belt conveyor controls the corresponding belt conveyor audible and visual alarm to give out an alarm by the PLC programming module according to the light-on alarm and the alarm after the belt is torn, the alarm is distinguished in a light-on mode and is divided into three conditions, namely, the normal operation of a green light, the half-speed operation of a yellow light and the stop operation of a red light;
when the data detection module detects that the belt is not torn through a vision technology, the bright green light normally runs;
when the data detection module detects that the belt is torn at a non-edge position through a vision technology, the bright yellow lamp runs at half speed, and the operator on duty replaces the torn non-edge belt in the belt conveyor running at half speed;
when the data detection module detects that the edge of the belt is torn through a vision technology, the red light is turned on to stop running, and the operator on duty can replace the torn edge belt in the belt conveyor which stops running.
The invention has the advantages that the management of the snapshot picture information under the abnormal condition is identified through the intelligent vision technology, the two symmetrical edges of the belt are detected by the edge tearing through the computer vision technology, and whether the tearing exists is judged according to the distance between the two detected edges; the non-edge tearing adopts a deep learning technology, and the belt tearing detection is realized by positioning the cracks of the belt.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise. Wherein:
fig. 1 is a system interface diagram of a method for automatically detecting a torn belt state based on a visual technology in embodiment 1.
FIG. 2 is a flowchart of a system for automatic detection of a torn belt status based on a visual technique according to embodiment 2.
Fig. 3 is a system configuration diagram of a system for automatic belt tearing state detection based on a visual technique according to embodiment 2.
Fig. 4 is a diagram of a camera model of a system for automatically detecting a belt tearing state based on a vision technique in embodiment 3.
Fig. 5 is a PLC type diagram of a system for automatically detecting a torn state of a belt according to embodiment 3.
FIG. 6 is a monitoring-only system diagram of a system for automatic belt tearing state detection based on visual technology in embodiment 3.
FIG. 7 is a belt tearing monitoring configuration diagram of a system for automatic belt tearing state detection based on visual technology according to embodiment 3.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention more comprehensible, embodiments accompanying figures of the present invention are described in detail below.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described and will be readily apparent to those of ordinary skill in the art without departing from the spirit of the present invention, and therefore the present invention is not limited to the specific embodiments disclosed below.
Furthermore, reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one implementation of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.
Example 1
Referring to FIG. 1, a first embodiment of the present invention provides a method for automatic belt tearing state detection based on visual technology, comprising
Constructing a daily belt state data set on the belt conveyor, and storing belt monitoring pictures in 8 months in the data set;
analyzing and comparing the conventional daily state data set of the belt, preprocessing and marking the image frame of the torn belt and establishing a Yolov3 target detection model;
and (3) putting the picture shot in real time into a YOLOv3 target detection model, monitoring the picture of the real-time monitoring belt state through a vision technology, and performing abnormity identification and alarm when the belt state monitored in real time is the same as the state of a data mark in a data set.
The belt state is that the belt tearing frequency of the belt machine with the inner belt in 8 months is known, and the belt state is processed by a common processing method and collected on site;
the belt state data set is expressed in the form of a photograph after being taken by a camera.
The data marking process is to carry out preprocessing on the data set;
the preprocessing is to record the tearing state of the belt in the data set picture, wherein the tearing state of the belt is divided into edge tearing and non-edge tearing;
the edge tearing adopts a computer vision technology to detect two symmetrical edges of the belt, and whether the tearing exists is judged according to the distance between the two detected edges;
the non-edge tearing adopts a deep learning technology, and the belt tearing detection is realized by positioning the crack of the belt;
and the data marking is to mark the belt offset position by using labeling tool labelImg software.
The training of the target detection model belongs to supervised training, the preprocessed pictures need to be labeled with the class information and the position information of objects in the pictures, and then the model learns the characteristics of the pictures continuously according to the labeled information to realize the classification and the positioning of the targets. The invention uses open-source labeling tool, namely, labelImg software to complete the labeling of the belt tearing data set, and the labeling software is widely applied to the data set labeling work of each target detection model, such as YOLO used by the invention. According to the on-site consultation, the common tearing on the belt conveyor is divided into edge tearing and non-edge tearing, the two kinds of pictures torn on the belt conveyor are mainly collected during picture collection, and the labels corresponding to the tearing are shown in table 1.
| Annotating objects | Edge tear | Non edge tearing |
| Label (R) | Off | ON |
Labeling a tearing target in a picture by using labellmg software, wherein a labeling interface is shown in fig. 1, and labeling and storing an imported picture to generate a corresponding xml file, wherein < bndbox > represents coordinate information of the target, and < name > represents an attribute of the target.
The YOLOv3 target detection model divides an image into 7x7 grids during detection, the network where the center of an object to be detected falls is responsible for predicting the belt tearing condition, each grid predicts 2 boundary frames, 98 boundary frames are predicted in total, a boundary frame higher than the actual boundary frame IOU of the object is selected from the two predicted boundary frames of each grid as output during training, and each predicted boundary frame consists of an offset value of a center coordinate relative to the upper left corner of the predicted grid and the width, height and confidence coefficient after normalization;
the confidence coefficient is calculated by
A＝P r (o)*IOU
Where A is the confidence, P r (o) is whether there is a target in the grid, when there is a target P r (o) is 1, P is absent r (o) is 0, IOU is the mesh overlap, and the calculation method is
Wherein GT refers to the prediction grid, PB refers to the current detection grid, and based on this, the coordinate transformation between the predicted value and the actual value of the grid is
Wherein c is x ，c y ，c w ，c h Is the center coordinate and width and height of the prediction box, b x ，b y Is the offset of the central coordinate of the prediction frame relative to the coordinate of the upper left corner of the prediction grid, and the delta function is b x ，b y Offset normalized to 1-0, b w ，b h Is the wide and high scaling size of the prediction box compared to the anchor grid, d x And d y Is the width and height of the anchoring grid;
the anchoring grid is used as a frame in the network model for detecting each target in the target detection, and is also called a prior frame;
the YOLOv3 target detection model divides an image into a 7x7 grid division mode during detection, because the tearing state can be quickly embodied through the 7x7 grid division mode no matter the belt is torn at the edge or not, the 8 x 8 grid division mode is opposite to the belt tearing mode, the importance of the position, which is easy to tear and protrudes, of the belt edge in data set is easily reduced, and the position offset of the edge tearing mode and the position offset of the non-edge tearing mode in the specific implementation situation is difficult to find through the 6x 6 grid division mode.
The real-time picture detection result in the data model stores relevant information into a database every 8 hours, and for the image frame with belt tearing, the detailed information of the image frame is stored into the database to be used as a data source of belt tearing statistics except for being displayed on a system homepage as a warning signal, so as to guide production.
Example 2
Referring to fig. 2, a second embodiment of the present invention, which is different from the first embodiment, is: also include for
The data input module 100 sets the network camera above the belt conveyor to adjust to a proper angle according to the field conditions, and connects the camera to the network so that the transportation condition on the belt conveyor can be obtained in real time.
The data detection module 200 constructs a YOLOv3 target detection model based on the data set, when the belt tearing condition occurs on the belt conveyor, the data detection model inputs an image frame of belt tearing into the target detection model, and if the belt tearing condition is detected, a control instruction is sent to the lower computer PLC through the Ethernet, the corresponding belt conveyor audible and visual alarm is controlled to give an alarm, and related workers are prompted about the belt tearing condition in time.
The data detection module 200 is used for performing visual technology detection based on a YOLOv3 target detection model, and uses the low-level and high-level features to detect the large-size, medium-size and small-size belt tearing conditions on three dimensions 13x13, 26x26 and 52x52 respectively, in the feature extraction process, the network undergoes 5 times of down-sampling with the step length of 2, the 13x13 feature-extracted picture is obtained from the 416x416 original picture, each grid has a larger receptive field, the larger-size target is detected on the 13x13 dimension, similarly, each grid on the 52x52 dimension corresponds to a smaller range of the original image and is used for detecting the smaller-size target, and the edge tearing detection process is shown in fig. 2.
The receptive field is the size of the area mapped by the pixel points on the characteristic graph output by each layer of the convolutional neural network on the input graph, and the explanation of the popular points is that one point on the characteristic graph corresponds to the area on the input graph.
The belt conveyor running module 400 can perform three specific operations according to the PLC instruction, namely normal running, half-speed running and stop running;
the belt conveyor controls the corresponding belt conveyor audible and visual alarm to give an alarm according to the fact that the belt is torn and then the light-up alarm and the alarm are both controlled by the PLC programming module 300, the alarm is distinguished in a light-up mode and is divided into three conditions, namely, the green light is in normal operation, the yellow light is in half-speed operation, and the red light is in stop operation;
when the data detection module 200 detects that the belt is not torn through the vision technology, the green light is turned on to normally operate;
when the data detection module 200 detects that the belt is torn at a non-edge position through a vision technology, the bright yellow lamp runs at a half speed, and an operator on duty replaces the torn non-edge belt in the belt conveyor running at the half speed;
when the data detection module 200 detects that the edge of the belt is torn through a vision technology, the red light is turned on to stop running, and the operator on duty replaces the torn edge belt in the belt conveyor which stops running.
The image enhancement mode of the invention comprises image size compression, picture turning and picture rotation.
(1) Image size compression
The method is characterized in that a Canon camera is used for collecting common foreign matters on a belt conveyor, pictures are high-definition pictures, the resolution is 6000x4000, samples detected as targets are too huge, the resolution of the pictures needs to be compressed, the length of the pictures is compressed to a specified scale so that the pictures cannot be deformed and distorted during picture compression, and the width of the pictures is compressed according to the length compression ratio.
(2) Picture turnover
The image turning is a simpler method for expanding the image in image enhancement, and the image can be rearranged by mirror turning to obtain an image symmetrical to the original image. The picture turning comprises turning up and down, turning left and right or turning up and down and turning left and right simultaneously.
(3) Picture rotation
When the picture rotates by a certain angle by taking the center of the image as an original point, a part of the corner part of the picture exceeds the range of the original picture, and a part of the corner part of the picture is retracted into the picture to leave a blank, and the blank part is filled with a black background.
Example 3
Referring to fig. 4 to 6, a third embodiment of the present invention, which is different from the first two embodiments, is:
the image acquisition equipment system selects a Haokangwei video network camera Canon camera as video acquisition equipment, the model of the camera is DS-2CD3T25-I36mmD, and the video stream is acquired in a rtsp mode. According to the field conditions, the network camera is arranged above the belt conveyor and adjusted to a proper angle, the camera is connected with the network, the coal transportation condition on the belt conveyor can be obtained in real time, and the image acquisition equipment selected by the scheme is as shown in fig. 4:
the rtsp format of the camera is shown in table 1, and the real-time video recording of the camera can be obtained through the interface as the input of the model.
TABLE 1 rstp Address Format
The PLC can be a programmable logic controller, the controller consists of units such as a CPU, an I/0 interface, a power supply, a memory and the like, not only has a logic control function, but also can perform PLC control of a time sequence control, multi-machine communication and the like, and has the advantages of high reliability, strong logic function, good flexibility and the like, so the PLC controller is widely used in the actual production environment, and famous PLC manufacturers have Siemens, mitsubishi and the like. Taking a coal conveying production line of a certain steel mill as an example, siemens S7-400 is selected for belt conveyor control, as shown in FIG. 5.
In the structure of the detection system of fig. 6, when the button of the camera is clicked to open, the original picture of the camera is displayed in the original image area of the camera. And after the detection is started, the detection result is displayed in a detection result area in real time. When the belt is torn in the detection process, the system can automatically intercept the belt tearing picture as warning information to be displayed and list the detailed information of the foreign matters.
The original monitoring video can be stored in the actual production process, and after a user logs in through a browser, the user can select a time period to view or download the video in a historical video viewing menu.
As shown in FIG. 7, the belt tearing condition can be quickly detected by the system during the detection process, and the belt tearing position and the tearing strength can be accurately displayed.
It should be noted that the above-mentioned embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, which should be covered by the claims of the present invention.
Claims (10)
1. A method for automatically detecting a belt tearing state based on a visual technology is characterized by comprising the following steps:
constructing a daily belt state data set on the belt conveyor, and storing belt monitoring pictures in 8 months in the data set;
analyzing and comparing the conventional daily state data set of the belt, preprocessing and marking the image frame of the torn belt and establishing a Yolov3 target detection model;
and (3) putting the picture shot in real time into a YOLOv3 target detection model, monitoring the real-time monitoring belt state picture through a vision technology, and performing abnormal recognition alarm when the real-time monitoring belt state is the same as the data mark state in the data set.
2. The method for the automatic detection of the tearing state of the belt based on the visual technology as claimed in claim 1, characterized in that: the belt state is that the belt tearing frequency of the belt machine with the belt machine in 8 months is known, and the belt machine is processed by a common processing method and collected on site;
the belt state data set is expressed in the form of a picture after being shot by a camera.
3. A method for automatic belt tearing state detection based on visual technology as claimed in claim 1 or 2, characterized in that: the data marking process is to carry out preprocessing on the data set;
the preprocessing is to record the state of the torn belt in the data set picture, and the torn belt state is divided into edge tearing and non-edge tearing;
the edge tearing adopts a computer vision technology to detect two symmetrical edges of the belt, and whether tearing exists is judged according to the distance between the two detected edges;
the non-edge tearing adopts a deep learning technology, and the belt tearing detection is realized by positioning the crack of the belt;
and the data marking is to mark the belt offset position by using label tool labelImg software.
4. A method for automatic belt tearing state detection based on visual technology as claimed in claim 3, characterized in that: the YOLOv3 target detection model divides an image into 7x7 grids during detection, the network where the center of an object to be detected falls is responsible for predicting the belt tearing condition, each grid predicts 2 boundary frames, 98 boundary frames are predicted in total, a boundary frame higher than the actual boundary frame IOU of the object is selected from the two predicted boundary frames of each grid as output during training, and each predicted boundary frame consists of an offset value of a center coordinate relative to the upper left corner of the predicted grid and the width, height and confidence coefficient after normalization;
the confidence coefficient is calculated by
A＝P r (o)*IOU
Where A is the confidence, P r (o) is whether there is a target in the grid, when there is a target P r (o) is 1, P is absent r (o) is 0, IOU is the mesh overlap, and the calculation method is
Wherein GT refers to the prediction grid, PB refers to the current detection grid, and based on this, the coordinate transformation between the predicted value and the actual value of the grid is
Wherein c is x ，c y ，c w ，c h Is the center coordinate and width and height of the prediction box, b x ，b y Is the offset of the center coordinate of the prediction frame relative to the coordinate of the upper left corner of the prediction grid, and the delta function is b x ，b y Offset normalized to 1-0, b w ，b h Is the wide and high scaling size of the prediction box compared to the anchor grid, d x And d y Is the width and height of the anchoring grid;
the anchoring grid is used as a description frame in target detection and is applied to a network model of each target detection, and the description frame is also called a prior frame;
the YOLOv3 target detection model divides an image into a 7x7 grid division mode during detection, because the tearing state can be quickly embodied through the 7x7 grid division mode no matter the belt is torn at the edge or is not torn at the edge, the 8 x 8 grid division mode is opposite to the belt edge division mode, the importance of the position, which is easy to tear and protrudes, of the belt edge in data concentration is easily reduced, and the position offset of the edge tearing and the non-edge tearing in the specific implementation situation is difficult to find through the 6x 6 grid division mode.
5. A method for automatic detection of a torn belt state based on visual technology, as claimed in any one of claims 1, 2 and 4, characterized in that: the real-time picture detection result in the data model stores related information into a database every 8 hours, and for image frames with belt tearing, the detailed information of the image frames is stored into the database to be used as a data source for belt tearing statistics except for being displayed on a system homepage as a warning signal, so that production is guided.
6. The utility model provides a belt tearing state automated inspection's system based on vision technique which characterized in that: the belt regulation system comprises a data input module (100), a data detection module (200), a belt regulation module (300) and a belt running module (400);
the data input module (100) is used for shooting the belt conveyor, and shooting data are put into the data detection module in a mode of forming pictures frame by frame after shooting;
the data detection module (200) is used for preprocessing the data entering the module, then establishing a Yolov3 target detection model, then detecting the belt tearing condition of the preprocessed picture, and sending a corresponding signal to the belt adjusting module (300) when the tearing condition is found;
the belt adjusting module (300) is composed of a PLC structure and is used for sending out a corresponding adjusting signal after the data monitoring module (200) monitors that the belt is torn, and the belt adjusting module (300) transcribes the signal into an instruction through PLC programming and sends the instruction to the belt operating module (400);
the belt running module (400) is used for running the belt conveyor, and when the PLC sends out a belt conveyor running adjustment operation instruction, the corresponding adjustment is carried out on the operation represented by the specific code;
the system for automatically detecting the tearing state of the belt based on the visual technology is arranged in a sequence that data enters from a data input module (100), a belt adjusting module (300) sends out an adjusting instruction after the data is detected by a data detection module (200), and a belt operating module (400) carries out specific belt conveyor operation based on the instruction.
7. The system for automatic belt tearing state detection based on visual technology of claim 6, wherein: the data input module (100) is used for arranging a network camera above the belt conveyor according to field conditions and adjusting the network camera to a proper angle, and the camera is connected with a network so that the transportation condition of the belt conveyor can be obtained in real time.
8. A system for automatic belt tearing state detection based on visual technology according to claim 6 or 7, characterized in that: the data detection module (200) establishes a YOLOv3 target detection model based on a data set, when a belt tearing condition occurs on the belt conveyor, the data detection model inputs an image frame of belt tearing into the target detection model, and if the belt tearing condition is detected, a control instruction is sent to the lower computer PLC through the Ethernet, so that an audible and visual alarm of the corresponding belt conveyor is controlled to give an alarm, and related workers are prompted to timely take the belt tearing condition.
9. The system for automatic belt tearing state detection based on visual technology of claim 8, wherein: the data detection module (200) is used for detecting a visual technology based on a YOLOv3 target detection model, the large-size, medium-size and small-size belt tearing conditions are detected on three scales of 13x13, 26x26 and 52x52 by utilizing the characteristics of a low layer and a high layer respectively, in the characteristic extraction process, a network is subjected to 5 times of down-sampling with the step length of 2, a 13x13 characteristic extraction picture is obtained from a 416x416 original picture, each grid has a larger receptive field, a target with a larger size is detected on the 13x13 scale, and similarly, each grid on the 52x52 scale corresponds to a smaller range of an original image and is used for detecting a target with a smaller scale;
the receptive field is the size of an area mapped by pixel points on a characteristic graph output by each layer of the convolutional neural network on an input graph, and the interpretation of the popular points is that one point on the characteristic graph corresponds to the area on the input graph;
for the detection of belt tearing, the target with smaller detection size needs to be subjected to image enhancement on a shot picture before further detection;
the image enhancement comprises image size compression, image turning and image rotation;
the image size compression is to compress the length of the picture to a specified length, and the width is compressed in equal proportion with the length of the picture;
the image turning is to obtain a picture symmetrical to the picture in a mirror image turning mode;
the image rotation is that when the center of the image is rotated for a certain angle as an original point, a part of the corner part of the image exceeds the range of the original image, and a part of the corner part of the image is contracted into the image to leave a blank, and the blank part of the image is filled with a black background;
the image enhancement effect is to increase the expression of the data set of the area where the belt tearing is likely to occur, reduce the time required for detection and increase the detection effect.
10. A system for automatic belt tearing state detection based on visual technology according to any one of claims 7 and 9, characterized in that: the belt conveyor running module (400) can carry out three specific operations according to a PLC instruction, namely normal running, half-speed running and stopping running;
the belt conveyor controls the corresponding belt conveyor audible and visual alarm to give out an alarm by the PLC programming module (300) according to the light warning and the alarm warning after the belt is torn, the alarm is distinguished in a light-on mode and is divided into three conditions, namely, the normal operation of a green light, the half-speed operation of a yellow light and the stop operation of a red light;
when the data detection module (200) detects that the belt is not torn through a visual technology, the green light is turned on to normally operate;
when the data detection module (200) detects that the belt is not torn at the edge through the vision technology, the bright yellow lamp runs at half speed, and the operator on duty replaces the torn non-edge belt in the belt conveyor running at half speed;
when the data detection module (200) detects that the belt edge is torn through a vision technology, the red light is turned on to stop running, and the operator on duty replaces the torn edge belt in the belt conveyor which stops running.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211323534.9A CN115908272A (en) | 2022-10-27 | 2022-10-27 | Method and system for automatically detecting belt tearing state based on vision technology |
| PCT/CN2022/138353 WO2024087341A1 (en) | 2022-10-27 | 2022-12-12 | Vision technology-based belt tearing state automatic detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211323534.9A CN115908272A (en) | 2022-10-27 | 2022-10-27 | Method and system for automatically detecting belt tearing state based on vision technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115908272A true CN115908272A (en) | 2023-04-04 |
Family
ID=86487528
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211323534.9A Pending CN115908272A (en) | 2022-10-27 | 2022-10-27 | Method and system for automatically detecting belt tearing state based on vision technology |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115908272A (en) |
| WO (1) | WO2024087341A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118164196A (en) * | 2024-05-14 | 2024-06-11 | 深圳市铁越电气有限公司 | Method and system for monitoring health state of coal conveying belt based on machine vision |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109649091A (en) * | 2018-12-28 | 2019-04-19 | 泉州装备制造研究所 | Monitoring system for tyres of automobile based on computer vision |
| CN109829429B (en) * | 2019-01-31 | 2022-08-09 | 福州大学 | Security sensitive article detection method based on YOLOv3 under monitoring scene |
| CN110980192A (en) * | 2019-12-10 | 2020-04-10 | 安徽银河物联通信技术有限公司 | Belt tearing detection method |
| EP3852054A1 (en) * | 2020-01-16 | 2021-07-21 | Koninklijke Philips N.V. | Method and system for automatically detecting anatomical structures in a medical image |
| CN112949633B (en) * | 2021-03-05 | 2022-10-21 | 中国科学院光电技术研究所 | Improved YOLOv 3-based infrared target detection method |
| CN113682762A (en) * | 2021-08-27 | 2021-11-23 | 中国矿业大学 | Belt tearing detection method and system based on machine vision and deep learning |
| CN115171051B (en) * | 2022-09-06 | 2023-01-10 | 合肥金星智控科技股份有限公司 | Online detection method and system for tearing of edge of conveying belt |
-
2022
- 2022-10-27 CN CN202211323534.9A patent/CN115908272A/en active Pending
- 2022-12-12 WO PCT/CN2022/138353 patent/WO2024087341A1/en unknown
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118164196A (en) * | 2024-05-14 | 2024-06-11 | 深圳市铁越电气有限公司 | Method and system for monitoring health state of coal conveying belt based on machine vision |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024087341A1 (en) | 2024-05-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110390691B (en) | Ore dimension measuring method based on deep learning and application system | |
| CN110084165B (en) | Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation | |
| CN110826538A (en) | Abnormal off-duty identification system for electric power business hall | |
| CN110980192A (en) | Belt tearing detection method | |
| CN111432182A (en) | Safety supervision method and system for oil discharge place of gas station | |
| CN111582235A (en) | Alarm method, system and equipment for monitoring abnormal events in station in real time | |
| CN107220969A (en) | The method of testing and detecting system of product lamp position | |
| CN113297885A (en) | Belt conveyor surface state detection method and device based on convolutional neural network | |
| CN114998234A (en) | Self-supervision spring defect detection method based on data enhancement strategy | |
| CN113408361A (en) | Deep learning-based mining conveyor belt bulk material detection method and system | |
| CN112702570A (en) | Security protection management system based on multi-dimensional behavior recognition | |
| CN116310922A (en) | Petrochemical plant area monitoring video risk identification method, system, electronic equipment and storage medium | |
| CN117094609B (en) | Intelligent management system for aluminum profile production quality based on machine vision | |
| Zhang et al. | Belt deviation detection system based on deep learning under complex working conditions | |
| CN112906593A (en) | Sluice image identification method based on fast RCNN | |
| CN115908272A (en) | Method and system for automatically detecting belt tearing state based on vision technology | |
| CN116994066A (en) | Tail rope detection system based on improved target detection model | |
| CN112232235A (en) | Intelligent factory remote monitoring method and system based on 5G | |
| CN111860429A (en) | Blast furnace tuyere abnormality detection method, device, electronic apparatus, and storage medium | |
| CN117035669A (en) | Enterprise safety production management method and system based on artificial intelligence | |
| CN115661707A (en) | Belt deviation identification algorithm based on inspection robot | |
| CN114387564A (en) | Head-knocking engine-off pumping-stopping detection method based on YOLOv5 | |
| CN114120109A (en) | Belt longitudinal tearing detection method based on neural network | |
| CN109214390B (en) | Fence state detection method and system based on machine vision principle | |
| CN110969081A (en) | Power transmission line external force damage detection method based on KL divergence of multi-module division |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |