CN113763375A - Conveyor belt deviation degree detection method and device, computer equipment and storage medium - Google Patents

Conveyor belt deviation degree detection method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN113763375A
CN113763375A CN202111092682.XA CN202111092682A CN113763375A CN 113763375 A CN113763375 A CN 113763375A CN 202111092682 A CN202111092682 A CN 202111092682A CN 113763375 A CN113763375 A CN 113763375A
Authority
CN
China
Prior art keywords
conveyor belt
edge
angle
determining
picture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111092682.XA
Other languages
Chinese (zh)
Inventor
胡懋成
王秋阳
凤阳
郑博超
何金龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Sunwin Intelligent Co Ltd
Original Assignee
Shenzhen Sunwin Intelligent Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Sunwin Intelligent Co Ltd filed Critical Shenzhen Sunwin Intelligent Co Ltd
Priority to CN202111092682.XA priority Critical patent/CN113763375A/en
Publication of CN113763375A publication Critical patent/CN113763375A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Quality & Reliability (AREA)
  • Geometry (AREA)
  • Control Of Conveyors (AREA)

Abstract

The embodiment of the invention discloses a method and a device for detecting the offset degree of a conveyor belt, computer equipment and a storage medium. The method comprises the following steps: acquiring a picture of a conveyor belt to obtain a picture to be detected; performing binarization semantic segmentation on a picture to be detected to obtain a semantic segmentation picture containing a conveyor belt area and a background; detecting edge key points of a picture to be detected; determining a parameter equation of a straight line; determining an angular bisector of the edge of the conveyor belt; carrying out roller detection on the picture to be detected; determining an angular bisector of a roller reference line; determining an included angle offset angle; determining a global offset area; determining the deviation degree of the conveyor belt according to the deviation angle of the included angle and the global deviation area; and carrying out alarm processing according to the deviation degree of the conveyor belt. By implementing the method provided by the embodiment of the invention, the problem that the detection cannot be carried out due to the instrument deployment angle can be solved, and the condition that a plurality of edges are detected or some edges cannot be detected in the detection of the edges of the conveying belt can be avoided.

Description

Conveyor belt deviation degree detection method and device, computer equipment and storage medium
Technical Field
The present invention relates to a conveyor belt, and more particularly, to a method and apparatus for detecting a degree of deviation of a conveyor belt, a computer device, and a storage medium.
Background
Cargo transport conveyer belt conveyer wide application is in shipment harbour, short distance transportation scenes such as processing factory, its simple structure that has, the conveying capacity is big, use advantages such as scene are extensive make can the application in each trade, but in the transportation process, cargo transport conveyer belt can be because the mounted position deviation, the material is overweight, the skew scheduling problem leads to its position to take place the skew, long-time off tracking more probably can make the conveyer belt break off, the life of conveyer belt has seriously been influenced, and in case also reach whole production process of the problem.
The existing mainstream method for detecting the deviation of the goods transportation conveying belt is mostly in a contact type, a laser infrared type, a Hall magnetic induction type and the like. In this type of detection mode, the displacement of the conveyor belt is limited by the clamping groove, so that the conveyor belt is damaged due to friction on the edge of the conveyor belt. Or the detection is carried out by a very expensive instrument, and the result of the detection system can be seriously influenced once the sensor has a problem.
Chinese patent CN 201910421387.0 utilizes video to detect and track the target of multiple rollers on two sides and track the shielding time of the target, when more than two fixed-point rollers are shielded simultaneously, the time that the target is shielded is tracked, when the time exceeds a certain threshold, the server extracts the images around the shielded roller, and gives them to an image classification algorithm to determine whether the roller is a conveyor belt, and then performs alarm processing. Chinese patent CN201510221251.7 is based on machine vision, mark the conveyer belt axis at the non-bearing surface of the goods transportation conveyer belt, utilize the original point zone of image coordinate system the perpendicular line of the conveyer belt axis that detects describes the amount of deviation and direction of the conveyer belt with the contained angle of the image coordinate system horizontal direction, the advantage of this method does not need to keep the high accuracy through the situation that historical sample data goes fitting, but also has more shortcomings, such as can only carry out the video shooting of vertical visual angle, and need artificially mark the axis on the conveyer belt, some scenes can not suit the vertical visual angle, mark the axis on the conveyer belt, can lead to the axis fuzzy because of dust absorption, or take place deformation after the long-time load of conveyer belt and cause the unstable problem of detection. Some of the prior art also adopt the method that images collected in real time are used, the images transmitted by the conveyor belt are detected frame by Hough line detection, and the straight line at the edge of the conveyor belt is screened out by an SVM classification algorithm; acquiring historical sample data, and calculating to obtain a threshold interval of the conveyor belt offset; the screened edge straight line is compared with a conveyor belt deviation threshold value space, and the conveyor belt deviation degree is calculated, but an SVM (support vector machine) has no general solution to the nonlinear problem, and sometimes a proper kernel function cannot be found. And is sensitive to missing data.
In summary, in the prior art, due to the complex field environment of the port, the existing cargo transportation conveyor belt is worn and slipped after being operated for a long time, so that the final deviation situation occurs, and meanwhile, due to the limitation of the field environment, the detection instrument cannot look right at to carry out conveyor belt shooting, and the prior method only detects the conveyor belt according to the angle of looking right at; the Hough change is basically used for straight line detection in conveyor belt deviation detection, but the method often causes the condition that a plurality of edges are detected or some edges are not detected in conveyor belt edge detection because the conveyor belt edges are uneven; the deviation is classified, so that the deviation degree cannot be specifically quantized, and some deviation degrees can be quantized only to measure the overall deviation degree and cannot measure the local deviation degree, so that the deviation of the conveying belt cannot be early warned; the benchmark of detection often needs to be predetermined by people, which can become extremely complex for field deployment, and debugging is very difficult in the case of freight pipeline work.
Therefore, it is necessary to design a new method to solve the problems of the prior art.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a conveyor belt deviation degree detection method, a conveyor belt deviation degree detection device, computer equipment and a storage medium.
In order to achieve the purpose, the invention adopts the following technical scheme: the conveyor belt deviation degree detection method comprises the following steps:
acquiring a picture of a conveyor belt to obtain a picture to be detected;
performing binarization semantic segmentation on the picture to be detected to obtain a semantic segmentation picture containing a conveyor belt area and a background;
detecting edge key points of the picture to be detected to obtain edge key points;
determining a parameter equation of a straight line according to the edge key points;
determining an angular bisector of the edge of the conveyor belt according to a parameter equation of the straight line;
carrying out roller detection on the picture to be detected to obtain an upper edge roller and a lower edge roller;
determining a roller reference line angle bisector according to the upper edge roller and the lower edge roller;
determining an included angle offset angle according to the conveyor belt edge angle bisector and the roller reference line angle bisector;
determining a global offset area according to the area of a polygonal frame defined by the angle bisector of the edge of the conveyor belt and the angle bisector of the roller reference line and the intersection point set of the edge of the picture to be detected;
determining the deviation degree of the conveyor belt according to the included angle deviation angle and the global deviation area;
and performing alarm processing according to the deviation degree of the conveyor belt.
The further technical scheme is as follows: the binarization semantic segmentation is carried out on the picture to be detected to obtain a semantic segmentation picture containing a conveyor belt area and a background, and the method comprises the following steps of:
performing semantic segmentation on the picture to be detected by adopting a semantic segmentation network to obtain a segmented picture;
and carrying out black-white binary mapping on the segmented picture to obtain a semantic segmented picture containing a conveyor belt area and a background.
The further technical scheme is as follows: the edge key point detection of the picture to be detected to obtain edge key points includes:
and identifying coordinates of the four key points of the picture to be detected by adopting a CPN (compact peripheral network) network so as to obtain edge key points.
The further technical scheme is as follows: the determining of the angular bisector of the edge of the conveyor belt according to the parametric equation of the straight line comprises the following steps:
determining the intersection coordinates of the parametric equation of the straight line on the upper edge of the conveyor belt and the parametric equation of the straight line on the lower edge of the conveyor belt, the slope of the parametric equation of the straight line on the upper edge of the conveyor belt and the slope of the parametric equation of the straight line on the lower edge of the conveyor belt;
and determining the angular bisector of the edge of the conveyor belt according to the coordinates of the intersection point, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt.
The further technical scheme is as follows: the right it detects to wait to detect the picture and carry out the roller bearing to obtain top edge roller bearing and lower limb roller bearing, include:
carrying out target positioning on the rollers in the to-be-detected drawing by adopting a yolov5 target detection model to obtain the image coordinate position of each roller;
calculating the coordinates of the center point of each roller according to the image coordinate position of each roller;
the distance from the center point coordinates to the upper edge conveyor and the distance from the center point coordinates to the lower edge conveyor are calculated to determine the upper edge rollers and the lower edge rollers.
The further technical scheme is as follows: the determining of the global offset area according to the area of a polygonal frame defined by the conveyor belt edge angle bisector and the roller shaft reference line angle bisector and the intersection point set of the edges of the picture to be detected comprises the following steps:
determining a set of intersection points of the angular bisector of the edge of the picture to be detected and the angular bisector of the edge of the conveyor belt and the angular bisector of the roller reference line to obtain a polygonal frame;
and calculating the area of the polygon frame to obtain a global offset area.
The further technical scheme is as follows: determining the deviation degree of the conveyor belt according to the included angle deviation angle and the global deviation area, and the method comprises the following steps:
when the global offset area is larger than an offset area threshold value and the included angle offset angle is smaller than an angle offset threshold value, determining the offset degree of the conveyor belt according to the global offset area;
when the global offset area is larger than an offset area threshold value and the included angle offset angle is larger than an angle offset threshold value, determining the offset degree of the conveyor belt according to the included angle offset angle;
and when the global deviation area is smaller than a deviation area threshold value and the included angle deviation angle is larger than an angle deviation threshold value, determining the deviation degree of the conveyor belt according to the included angle deviation angle.
The present invention also provides a conveyor belt deviation degree detection apparatus, including:
the image acquisition unit is used for acquiring the image of the conveyor belt to obtain an image to be detected;
the segmentation unit is used for carrying out binarization semantic segmentation on the picture to be detected so as to obtain a semantic segmentation picture containing a conveyor belt area and a background;
the key point detection unit is used for detecting edge key points of the picture to be detected to obtain edge key points;
the equation determining unit is used for determining a parameter equation of a straight line according to the edge key points;
the first determining unit is used for determining the angular bisector of the edge of the conveyor belt according to the parameter equation of the straight line;
the roller detection unit is used for carrying out roller detection on the picture to be detected so as to obtain an upper edge roller and a lower edge roller;
the second determining unit is used for determining an angular bisector of a roller reference line according to the upper edge roller and the lower edge roller;
the included angle deviation angle determining unit is used for determining an included angle deviation angle according to the conveyor belt edge angle bisector and the roller reference line angle bisector;
the global offset area determining unit is used for determining a global offset area according to the area of a polygonal frame defined by the intersection point set of the conveyor belt edge angle bisector and the roller shaft reference line angle bisector and the edge of the picture to be detected;
the degree determining unit is used for determining the deviation degree of the conveyor belt according to the included angle deviation angle and the global deviation area;
and the alarm processing unit is used for carrying out alarm processing according to the deviation degree of the conveyor belt.
The invention also provides computer equipment which comprises a memory and a processor, wherein the memory is stored with a computer program, and the processor realizes the method when executing the computer program.
The invention also provides a storage medium storing a computer program which, when executed by a processor, is operable to carry out the method as described above.
Compared with the prior art, the invention has the beneficial effects that: the invention adopts the form of the picture to detect the edge defect, greatly saves the cost and simplifies the deployment steps; the method comprises the steps of semantic segmentation, key point detection, linear parameter equation determination, conveyor belt edge angle bisector determination, roller detection, roller reference line angle bisector detection, included angle deviation angle determination, global deviation area determination and conveyor belt deviation detection to determine the conveyor belt deviation degree, conveyor belt deviation of any angle can be detected based on semantic segmentation, target detection and a linear detection algorithm, the problem that detection cannot be performed due to the problem of instrument deployment angle is solved, conveyor belt deviation evaluation is performed through a local angle deviation strategy and a global area deviation strategy, no artificial definition of reference is needed, early warning is performed on conveyor belt deviation, and the condition that a plurality of edges are detected or some edges cannot be detected in conveyor belt edge detection can be avoided.
The invention is further described below with reference to the accompanying drawings and specific embodiments.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic view of an application scenario of a conveyor belt deviation degree detection method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart illustrating a method for detecting a degree of belt deviation according to an embodiment of the present invention;
FIG. 3 is a schematic sub-flowchart of a method for detecting a degree of belt deviation according to an embodiment of the present invention;
FIG. 4 is a schematic sub-flowchart of a method for detecting a degree of belt deviation according to an embodiment of the present invention;
FIG. 5 is a schematic sub-flowchart of a method for detecting a degree of belt deviation according to an embodiment of the present invention;
FIG. 6 is a schematic sub-flowchart of a method for detecting a degree of belt deviation according to an embodiment of the present invention;
fig. 7 is a schematic block diagram of a conveyor belt deviation degree detection apparatus provided in the embodiment of the present invention;
fig. 8 is a schematic block diagram of a dividing unit of the conveyor belt deviation degree detecting apparatus provided in the embodiment of the present invention;
fig. 9 is a schematic block diagram of a first determining unit of the conveyor belt deviation degree detecting apparatus provided by the embodiment of the present invention;
fig. 10 is a schematic block diagram of a roller detecting unit of the conveyor belt deviation degree detecting apparatus according to the embodiment of the present invention;
fig. 11 is a schematic block diagram of a global offset area determination unit of the conveyor belt offset degree detection apparatus provided by the embodiment of the present invention;
FIG. 12 is a schematic block diagram of a computer device provided by an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Referring to fig. 1 and fig. 2, fig. 1 is a schematic view of an application scenario of a method for detecting a degree of conveyor belt deviation according to an embodiment of the present invention. Fig. 2 is a schematic flow chart of a method for detecting a degree of belt deviation according to an embodiment of the present invention. The conveyor belt deviation degree detection method is applied to a server. The server performs data interaction with a terminal and a camera, wherein the camera is used for shooting pictures of a transmission belt, and the server performs semantic segmentation, edge key point detection, parameter equation for determining edge straight lines, determination of angular bisectors of the edge of the transmission belt, roller detection, detection of angular bisectors of roller reference lines, determination of included angle offset angles, determination of global offset areas and transmission belt offset detection so as to determine the offset degree of the transmission belt and perform alarm processing.
Fig. 2 is a schematic flow chart of a method for detecting a degree of belt deviation according to an embodiment of the present invention. As shown in fig. 2, the method includes the following steps S110 to S210.
And S110, acquiring the picture of the conveyor belt to obtain the picture to be detected.
In this embodiment, the picture to be detected is a picture formed by collecting picture data at a fixed position set by the cargo transportation conveyor belt, and a fixed preset value is given to the focal length and the height shot by the pan-tilt in the collection process.
The dependence on hardware such as a sensor is eliminated, the edge abrasion detection of the goods transportation conveyor belt can be carried out only according to the image, the cost is greatly saved, and the deployment steps are simplified. The conveyer belt skew of arbitrary angle can be detected, the difficult problem that can't do the detection because of instrument deployment angle problem is solved.
And S120, performing binarization semantic segmentation on the picture to be detected to obtain a semantic segmentation picture containing a conveyor belt area and a background.
In this embodiment, the semantically segmented picture refers to a picture including a conveyor belt region and a background corresponding mask.
In an embodiment, referring to fig. 3, the step S120 may include steps S121 to S122.
And S121, performing semantic segmentation on the picture to be detected by adopting a semantic segmentation network to obtain a segmented picture.
In this embodiment, the segmented pictures refer to two semantic segmentation pictures of the carousel mask and the background mask.
Specifically, the semantic segmentation network is formed by adopting a resnet50 model as a skeleton network, performing feature fusion through a feature fusion module by combining a context path and a space path mechanism through the resnet50 model, selecting an attention refinement module before the feature fusion module to output and add two features as an auxiliary loss function, and adding the Dice loss function and the auxiliary loss function according to a proportion to form a final loss function of the semantic segmentation network.
Specifically, resnet50 is used as backbone network for semantic segmentation of cargo transportation conveyor belt, the model combines context path and spatial path mechanisms and performs Feature Fusion through Feature Fusion module to obtain semantic segmentation network, and the loss function used here is changed into dice loss function l1. An ARM (Attention refining Module) before Feature Fusion is selected to output and add two characteristics as an auxiliary loss function l2And l3Finally, they are added in a ratio of 1:1:1 as the final Loss function Loss ═ l1+l2+l3。
In order to ensure the accuracy of the prediction of the conveyor belt mask, firstly, the picture to be detected is used as the input of the network according to the pixels with the original width of 1920 and the original height of 1080, and the output types of the semantic segmentation network are two semantic segmentation graphs of the conveyor belt mask and the background mask respectively.
And S122, performing black-white binarization mapping on the segmented picture to obtain a semantic segmented picture containing a conveyor belt area and a background.
Performing black-white binarization mapping on the segmentation picture, setting three channels of pixels at the conveyor belt part as pixel points with the value of 255, and setting three channels of pixels at the background part as pixel points with the value of 0, thereby forming a semantic segmentation picture containing a conveyor belt region and a background, and defining the conveyor belt region as s hereConveyor belt。
S130, detecting edge key points of the picture to be detected to obtain edge key points.
In the present embodiment, the edge key points refer to four points, i.e., the start and end points of the upper edge, the start and end points of the lower edge of the conveyor belt.
Specifically, a CPN network is adopted to identify coordinates of four key points of the picture to be detected so as to obtain edge key points.
The confidence coefficient of the key points of the CPN network is 0.5-0.7, k-means clustering is carried out on the coordinates of the four key points in space, and the central point of the cluster is selected as the edge key point.
The straight line detection effect based on Hough change is not good when the edge of the conveyor belt is detected because the unevenness of the edge of the conveyor belt is considered, and the detection is carried out based on key points of the conveyor belt, and points of the starting end and the end point of the upper edge and the lower edge of the conveyor belt are respectively identified. The model used here is the identification of four keypoints by the CPN (Cascaded Pyramid Network) Network. And detecting vertex coordinates of two ends of the edge of the conveyor belt based on a key point detection algorithm, and obtaining a parameter equation of an edge straight line according to the coordinates of the two ends of the conveyor belt.
Two key points of the starting end and the terminal end of the upper edge can be obtained through the neural network. Upper edge point is ps,top，pe,topLower edge point is ps,bottom，pe,bottom. In order to prevent the key points from being detected due to edge redundancy and the key points from missing detection, slightly reducing the confidence coefficient of the key points to 0.6, simultaneously performing k-means clustering on the coordinates of the key points in the space on the basis of improving the recall rate of the key points, setting k to be 4, and selecting the central point of the clustering cluster as the representative point.
And S140, determining a parameter equation of the straight line according to the edge key points.
In this embodiment, the parametric equation of the straight line refers to the parametric equation of the straight line of the edge of the conveyor belt.
The parametric equation for the straight line of the upper and lower conveyor edges can be determined from the upper edge point, i.e. the two key points of the upper conveyor edge, and the lower edge point, i.e. the two key points of the lower conveyor edge.
And S150, determining the angular bisector of the edge of the conveyor belt according to the parameter equation of the straight line.
In this embodiment, the conveyor belt edge angle bisector refers to an angle bisector of the conveyor belt upper edge and the conveyor belt lower edge.
In an embodiment, referring to fig. 4, the step S150 may include steps S151 to S152.
S151, determining coordinates of an intersection of the parametric equation of the straight line at the upper edge of the conveyor belt and the parametric equation of the straight line at the lower edge of the conveyor belt, a slope of the parametric equation of the straight line at the upper edge of the conveyor belt, and a slope of the parametric equation of the straight line at the lower edge of the conveyor belt.
Specifically, the edge points of the corresponding straight line intersected with the picture to be detected are obtained based on the parameter equation of the straight line, and two edge line segments l of the conveyor belt are made in sequenceConveyor topAnd lDown conveyer belt。
S152, determining the angular bisector of the edge of the conveyor belt according to the coordinates of the intersection point, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt.
Specifically, the above-mentioned lConveyor topAnd lDown conveyer beltCoordinates (x) of intersection of two conveyor belt line equationsIntersection point,yIntersection point) And the slope of two straight lines into a half-angle formula
Figure BDA0003268136350000091
From the slope k of the equation of the parameter of the straight line at the upper edge of the conveyor beltConveyor topThe included angle A between the straight line and the coordinate system can be obtained, and similarly, the included angle A is lower than that of the lower edge of the conveyor beltSlope k of the parametric equation of a straight lineDown conveyer beltThe included angle B is obtained and is substituted into the right side of the half-angle formula to obtain the slope k of the angular bisector, namely
Figure BDA0003268136350000092
Further obtain the slope k of the angular bisector of the two edge linesConveying beltThen bring it into the intersection coordinates (x)Intersection point,yIntersection point) In (3), a linear equation f (x) of the angular bisector is obtainedAngular bisector＝k*x+b。
In the embodiment, the Feature extraction part is optimized for the key point detection, the FPN (Feature Pyramid network) network and the transform mechanism are combined together to perform model Feature extraction, and Feature point identification is performed by combining GlobalNet and RefineNet. In order to prevent the key points from being detected due to edge redundancy and the key points from missing detection, the confidence coefficient of the key points is reduced, and the coordinates of the key points are subjected to k-means clustering in space, wherein k is set to be 4, and the central point of the clustering cluster is selected as the representative point. And an OHEM strategy is also adopted in the training process, only the key point of top3 with the largest loss is returned, and the identification difficulty of individual key points is higher due to dust such as light, so that after two corresponding conveyor belt edge lines are obtained, the conveyor belt edge datum plane is further determined.
And S160, carrying out roller detection on the picture to be detected to obtain an upper edge roller and a lower edge roller.
In this embodiment, the rollers of the conveyor belt include an upper edge roller and a lower edge roller.
In an embodiment, referring to fig. 5, the step S160 may include steps S161 to S163.
S161, carrying out target positioning on the rollers in the to-be-detected drawing by adopting a yolov5 target detection model, and obtaining the image coordinate position of each roller.
In the present embodiment, the image coordinate position of each roller refers to the coordinates of the upper edge roller and the lower edge roller within the image.
In particular, the model is detected through yolov5 targetThe roller is positioned to obtain the image coordinate position (x) of each rollermin，ymin，xmax，ymax). The detection confidence of the yolov5 target detection model was chosen to be 0.6. In order to ensure the accuracy of the coordinates of the top end of the roller, a structure of yolo5x is selected for detection, CIOU loss is selected as a loss function corresponding to the loss function of the regression frame coordinates, and the front background cross entropy loss and the classification cross entropy loss are combined to form an integral loss function.
The detection of the roller needs to be noted here that if a roller is located at the edge of the image, the roller is not completely positioned easily, so that the end point value is not accurate, the position of the roller is respectively judged based on the condition, and the end point coordinate of the roller is not considered if any one of the following conditions is met.
(1)xmin<α；(2)width-xmax<α；(3)ymin<α；(4)height-ymax<α; here, width is the width of the picture, height is the height of the picture, and α is set to 10.
And S162, calculating the center point coordinate of each roller according to the image coordinate position of each roller.
In the present embodiment, the coordinates of the center point of each reel frame are calculated separately
Figure BDA0003268136350000101
Figure BDA0003268136350000102
S163, calculating the distance from the center point coordinates to the upper edge conveyor and the distance from the center point coordinates to the lower edge conveyor to determine the upper edge roller and the lower edge roller.
In the present embodiment, the center point coordinates (c) are calculatedx，cy) To the upper edge lConveyor beltDistance d ofUpper edgeAnd to the lower edge lConveyor beltDistance d ofLower edge. If d isLower edge<dUpper edgeThen the roller is determined as the lower edgeThe rollers, conversely referred to as upper edge rollers.
And S170, determining a roller reference line angle bisector according to the upper edge roller and the lower edge roller.
In this embodiment, the roller reference line angular bisector refers to an angular bisector of two reference lines defined by the upper edge roller and the lower edge roller.
Specifically, the end point coordinates of the upper edge roller and the lower edge roller are calculated; determining an upper reference line and a lower reference line according to the endpoint coordinates; determining an angular bisector of the roller reference line according to the upper reference line, the lower reference line and the intersection point of the two reference lines; if the slope k of the conveyor beltConveyor belt>0, for the upper edge rollers, selecting the coordinate of the upper right corner of each upper edge roller as the endpoint coordinate pi(k>0,top)(xmax，ymin) The lower edge rollers select the coordinate of the upper left corner of each lower edge roller as the endpoint coordinate pi(k>0,bottom)(xmin，ymin). If the slope k of the conveyor beltConveyor belt<0, for the upper edge roller, selecting the coordinate of the upper left corner of the roller as the endpoint coordinate pi(k<0,top)(xmin，ymin) The lower edge roller selects the upper right corner coordinate of the roller as the endpoint coordinate pi(k<0,top)(xmax，ymin) Finally, fitting two datum lines l through the end pointsDatum. Angular bisector l of roller reference line obtained through intersection points of reference linesReference angular bisector。
And S180, determining an included angle deviation angle according to the conveyor belt edge angle bisector and the roller reference line angle bisector.
In this embodiment, the included angle deviation angle refers to the included angle between the angular bisector of the edge of the conveyor belt and the angular bisector of the reference line of the roller.
Deriving a conveyor belt edge angle bisector l based on the aboveAngular bisector of conveyor beltAnd the resulting angular bisector l of the roller reference lineReference angular bisectorCalculating the corresponding included angle offset angle thetaOffset of。
And S190, determining a global offset area according to the area of a polygonal frame formed by the intersection point set of the conveyor belt edge angle bisector and the roller shaft reference line angle bisector and the edge of the picture to be detected.
In this embodiment, the global offset area is an area of a polygon formed by an intersection set of the conveyor belt edge angle bisector and the roller reference line angle bisector and the edge of the picture to be detected.
In an embodiment, referring to fig. 6, the step S190 may include steps S191 to S192.
S191, determining a set of intersection points of the angular bisector of the edge of the picture to be detected and the angular bisector of the edge of the conveyor belt and the angular bisector of the roller reference line to obtain a polygonal frame;
and S192, calculating the area of the polygon frame to obtain a global offset area.
Since only partial deviations can be reflected in consideration of the angular deviations, but if the conveyor belt deviates as a whole, the angular deviations are not changed, so that the belt edge angle bisector l can be passedAngular bisector of conveyor beltBisector l of angle with reference line of rolling shaftReference angular bisectorSolving the global offset area s according to the area of a polygonal frame formed by the intersection point set of the edge of the picture to be detectedOffset of. The local offset area is a closed polygon frame formed by the intersection point set of two angle bisectors and an image edge, and the main idea is to divide a polygon into a plurality of triangles and then add the areas of all the triangles, wherein the areas of the triangles are calculated by using the cross product of vectors. Namely, it is
Figure BDA0003268136350000121
Figure BDA0003268136350000122
Wherein the content of the first and second substances,
Figure BDA0003268136350000123
the vector can be obtained by coordinates of points A and B; | AB | is
Figure BDA0003268136350000124
The mold of (4); sin (BAC) is the sin value of < BAC. The method has the advantages that the existing point coordinates can be directly used for converting into vectors, and the method supports the area calculation of not only convex polygons but also concave polygons.
S200, determining the deviation degree of the conveyor belt according to the included angle deviation angle and the global deviation area.
Specifically, when the global offset area is larger than an offset area threshold and the included angle offset angle is smaller than an angle offset threshold, determining the offset degree of the conveyor belt according to the global offset area; when the global offset area is larger than an offset area threshold value and the included angle offset angle is larger than an angle offset threshold value, determining the offset degree of the conveyor belt according to the included angle offset angle; and when the global deviation area is smaller than a deviation area threshold value and the included angle deviation angle is larger than an angle deviation threshold value, determining the deviation degree of the conveyor belt according to the included angle deviation angle.
In the present embodiment, the offset area threshold s is setthresholdAnd an angular offset threshold θthreshold. The degree of belt deflection σ; when s isOffset of>sthresholdAnd thetaOffset of<θthresholdThen, then
Figure BDA0003268136350000125
Set lambda to
Figure BDA0003268136350000126
sConveyor beltIs the area of the conveyor belt; when s isOffset of>sthresholdAnd thetaOffset of>θthresholdThen, then
Figure BDA0003268136350000127
Figure BDA0003268136350000128
Is provided with
Figure BDA0003268136350000129
When s isOffset of<sthresholdAnd thetaOffset of>θthresholdThen, then
Figure BDA00032681363500001210
Is provided with
Figure BDA00032681363500001211
And S210, performing alarm processing according to the conveyor belt deviation degree.
And setting an alarm threshold gamma, wherein tau is more than gamma, and triggering an alarm mechanism. Where γ is set to 0.2.
The present embodiment gives a measure of the local and global offsets, respectively, for this case. The conveyer belt offset evaluation is carried out through the local angle offset strategy and the global area offset strategy respectively, automatic local angle offset and global area offset values can be given out to measure the conveyer belt offset condition, and the reference does not need to be defined artificially.
According to the conveyor belt deviation degree detection method, the edge defects are detected in a picture mode, so that the cost is greatly saved, and the deployment steps are simplified; the method comprises the steps of semantic segmentation, key point detection, linear parameter equation determination, conveyor belt edge angle bisector determination, roller detection, roller reference line angle bisector detection, included angle deviation angle determination, global deviation area determination and conveyor belt deviation detection to determine the conveyor belt deviation degree, conveyor belt deviation of any angle can be detected based on semantic segmentation, target detection and a linear detection algorithm, the problem that detection cannot be performed due to the problem of instrument deployment angle is solved, conveyor belt deviation evaluation is performed through a local angle deviation strategy and a global area deviation strategy, no artificial definition of reference is needed, early warning is performed on conveyor belt deviation, and the condition that a plurality of edges are detected or some edges cannot be detected in conveyor belt edge detection can be avoided.
Fig. 7 is a schematic block diagram of a conveyor belt deviation degree detection apparatus 300 according to an embodiment of the present invention. As shown in fig. 7, the present invention also provides a conveyor belt deviation degree detecting device 300 corresponding to the above conveyor belt deviation degree detecting method. The conveyor belt deviation degree detection apparatus 300 includes a unit for performing the above-described conveyor belt deviation degree detection method, and may be configured in a server. Specifically, referring to fig. 7, the conveyor belt deviation degree detection apparatus 300 includes a picture obtaining unit 301, a dividing unit 302, a key point detecting unit 303, an equation determining unit 304, a first determining unit 305, a roller detecting unit 306, a second determining unit 307, an included angle deviation angle determining unit 308, a global deviation area determining unit 309, a degree determining unit 310, and an alarm processing unit 311.
The picture acquiring unit 301 is configured to acquire a picture of the conveyor belt to obtain a picture to be detected; a segmentation unit 302, configured to perform binarization semantic segmentation on the to-be-detected picture to obtain a semantic segmentation picture containing a conveyor belt region and a background; a key point detecting unit 303, configured to perform edge key point detection on the picture to be detected to obtain edge key points; an equation determining unit 304, configured to determine a parameter equation of the straight line according to the edge key point; a first determining unit 305 for determining a conveyor belt edge angle bisector according to a parametric equation of the straight line; a roller detection unit 306, configured to perform roller detection on the picture to be detected to obtain an upper edge roller and a lower edge roller; a second determining unit 307 for determining a roller reference line angle bisector from the upper edge roller and the lower edge roller; an included angle offset angle determining unit 308, configured to determine an included angle offset angle according to the conveyor belt edge angle bisector and the roller reference line angle bisector; a global offset area determining unit 309, configured to determine a global offset area according to an area of a polygon frame defined by an intersection set of the conveyor belt edge angle bisector and the roller reference line angle bisector with the edge of the to-be-detected picture; a degree determining unit 310, configured to determine a conveyor belt deviation degree according to the included angle deviation angle and the global deviation area; and the alarm processing unit 311 is configured to perform alarm processing according to the conveyor belt deviation degree.
In one embodiment, as shown in fig. 8, the segmentation unit 302 includes a semantic segmentation subunit 3021 and a mapping subunit 3022.
A semantic segmentation subunit 3021, configured to perform semantic segmentation on the picture to be detected by using a semantic segmentation network to obtain a segmented picture; a mapping subunit 3022, configured to perform black-and-white binarization mapping on the segmented picture to obtain a semantic segmented picture containing a conveyor belt region and a background.
In an embodiment, the key point detecting unit 303 is configured to identify coordinates of four key points of the to-be-detected picture by using a CPN network, so as to obtain edge key points.
In an embodiment, as shown in fig. 9, the first determining unit 305 comprises a slope determining subunit 3051 and a conveyor edge bisector determining subunit 3052.
A slope determination subunit 3051, configured to determine intersection coordinates of the parametric equation of the straight line at the upper edge of the conveyor belt and the parametric equation of the straight line at the lower edge of the conveyor belt, a slope of the parametric equation of the straight line at the upper edge of the conveyor belt, and a slope of the parametric equation of the straight line at the lower edge of the conveyor belt; and the conveyor belt edge angle bisector determining subunit 3052, configured to determine a conveyor belt edge angle bisector according to the intersection coordinates, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt, and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt.
In one embodiment, as shown in fig. 10, the roller detecting unit 306 includes a target positioning subunit 3061, a center point coordinate calculating subunit 3062, and a distance calculating subunit 3063.
The target positioning subunit 3061 is configured to perform target positioning on the rollers in the to-be-detected figure by using a yolov5 target detection model, so as to obtain image coordinate positions of each roller; a central point coordinate calculation subunit 3062, for calculating the central point coordinate of each roller according to the image coordinate position of each roller; a distance calculation subunit 3063, for calculating the distance of the center point coordinates to the upper edge conveyor and the distance of the center point coordinates to the lower edge conveyor to determine the upper edge roller and the lower edge roller.
In an embodiment, as shown in fig. 11, the global offset area determination unit 309 includes an intersection set determination subunit 3091 and an area calculation subunit 3092.
The intersection set determining subunit 3091 is configured to determine an intersection set between the angular bisector of the conveyor belt edge and the angular bisector of the roller reference line and the edge of the picture to be detected, so as to obtain a polygonal frame; an area calculating subunit 3092, configured to calculate an area of the polygon frame to obtain a global offset area.
In an embodiment, the degree determining unit 310 is configured to determine the degree of belt deviation according to the global deviation area when the global deviation area is greater than a deviation area threshold and the included angle deviation angle is smaller than an angle deviation threshold; when the global offset area is larger than an offset area threshold value and the included angle offset angle is larger than an angle offset threshold value, determining the offset degree of the conveyor belt according to the included angle offset angle; and when the global deviation area is smaller than a deviation area threshold value and the included angle deviation angle is larger than an angle deviation threshold value, determining the deviation degree of the conveyor belt according to the included angle deviation angle.
It should be noted that, as will be clear to those skilled in the art, the specific implementation process of the conveyor belt deviation degree detection apparatus 300 and each unit may refer to the corresponding description in the foregoing method embodiment, and for convenience and brevity of description, no further description is provided herein.
The conveyor belt deviation degree detection apparatus 300 described above may be implemented in the form of a computer program that can be run on a computer device as shown in fig. 12.
Referring to fig. 12, fig. 12 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of a plurality of servers.
Referring to fig. 12, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.
The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 comprises program instructions that, when executed, cause the processor 502 to perform a method of conveyor belt offset detection.
The processor 502 is used to provide computing and control capabilities to support the operation of the overall computer device 500.
The internal memory 504 provides an environment for running the computer program 5032 in the non-volatile storage medium 503, and when the computer program 5032 is executed by the processor 502, the processor 502 may be caused to execute a conveyor belt deviation degree detection method.
The network interface 505 is used for network communication with other devices. Those skilled in the art will appreciate that the configuration shown in fig. 12 is a block diagram of only a portion of the configuration associated with the present application and does not constitute a limitation of the computer device 500 to which the present application may be applied, and that a particular computer device 500 may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
Wherein the processor 502 is configured to run the computer program 5032 stored in the memory to implement the following steps:
acquiring a picture of a conveyor belt to obtain a picture to be detected; performing binarization semantic segmentation on the picture to be detected to obtain a semantic segmentation picture containing a conveyor belt area and a background; detecting edge key points of the picture to be detected to obtain edge key points; determining a parameter equation of a straight line according to the edge key points; determining an angular bisector of the edge of the conveyor belt according to a parameter equation of the straight line; carrying out roller detection on the picture to be detected to obtain an upper edge roller and a lower edge roller; determining a roller reference line angle bisector according to the upper edge roller and the lower edge roller; determining an included angle offset angle according to the conveyor belt edge angle bisector and the roller reference line angle bisector; determining a global offset area according to the area of a polygonal frame defined by the angle bisector of the edge of the conveyor belt and the angle bisector of the roller reference line and the intersection point set of the edge of the picture to be detected; determining the deviation degree of the conveyor belt according to the included angle deviation angle and the global deviation area; and performing alarm processing according to the deviation degree of the conveyor belt.
In an embodiment, when implementing the step of performing binarization semantic segmentation on the to-be-detected picture to obtain a semantic segmented picture containing a conveyor belt region and a background, the processor 502 specifically implements the following steps:
performing semantic segmentation on the picture to be detected by adopting a semantic segmentation network to obtain a segmented picture; and carrying out black-white binary mapping on the segmented picture to obtain a semantic segmented picture containing a conveyor belt area and a background.
In an embodiment, when the processor 502 implements the step of performing edge key point detection on the picture to be detected to obtain edge key points, the following steps are specifically implemented:
and identifying coordinates of the four key points of the picture to be detected by adopting a CPN (compact peripheral network) network so as to obtain edge key points.
In an embodiment, when the processor 502 implements the step of determining the bisector of the edge angle of the conveyor belt according to the parametric equation of the straight line, the following steps are specifically implemented:
determining the intersection coordinates of the parametric equation of the straight line on the upper edge of the conveyor belt and the parametric equation of the straight line on the lower edge of the conveyor belt, the slope of the parametric equation of the straight line on the upper edge of the conveyor belt and the slope of the parametric equation of the straight line on the lower edge of the conveyor belt; and determining the angular bisector of the edge of the conveyor belt according to the coordinates of the intersection point, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt.
In an embodiment, when the processor 502 performs the roller detection on the to-be-detected picture to obtain the upper edge roller and the lower edge roller, the following steps are specifically performed:
carrying out target positioning on the rollers in the to-be-detected drawing by adopting a yolov5 target detection model to obtain the image coordinate position of each roller; calculating the coordinates of the center point of each roller according to the image coordinate position of each roller; the distance from the center point coordinates to the upper edge conveyor and the distance from the center point coordinates to the lower edge conveyor are calculated to determine the upper edge rollers and the lower edge rollers.
In an embodiment, when the processor 502 determines the global offset area according to the area of the polygonal frame defined by the intersection set of the conveyor belt edge angle bisector and the roller reference line angle bisector and the edge of the picture to be detected, the following steps are specifically implemented:
determining a set of intersection points of the angular bisector of the edge of the picture to be detected and the angular bisector of the edge of the conveyor belt and the angular bisector of the roller reference line to obtain a polygonal frame; and calculating the area of the polygon frame to obtain a global offset area.
In an embodiment, when the processor 502 determines the conveyor belt offset degree according to the included angle offset angle and the global offset area, the following steps are specifically implemented:
when the global offset area is larger than an offset area threshold value and the included angle offset angle is smaller than an angle offset threshold value, determining the offset degree of the conveyor belt according to the global offset area; when the global offset area is larger than an offset area threshold value and the included angle offset angle is larger than an angle offset threshold value, determining the offset degree of the conveyor belt according to the included angle offset angle; and when the global deviation area is smaller than a deviation area threshold value and the included angle deviation angle is larger than an angle deviation threshold value, determining the deviation degree of the conveyor belt according to the included angle deviation angle.
It should be understood that in the embodiment of the present Application, the Processor 502 may be a Central Processing Unit (CPU), and the Processor 502 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It will be understood by those skilled in the art that all or part of the flow of the method implementing the above embodiments may be implemented by a computer program instructing associated hardware. The computer program includes program instructions, and the computer program may be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the flow steps of the embodiments of the method described above.
Accordingly, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program, when executed by a processor, causes the processor to perform the steps of:
acquiring a picture of a conveyor belt to obtain a picture to be detected; performing binarization semantic segmentation on the picture to be detected to obtain a semantic segmentation picture containing a conveyor belt area and a background; detecting edge key points of the picture to be detected to obtain edge key points; determining a parameter equation of a straight line according to the edge key points; determining an angular bisector of the edge of the conveyor belt according to a parameter equation of the straight line; carrying out roller detection on the picture to be detected to obtain an upper edge roller and a lower edge roller; determining a roller reference line angle bisector according to the upper edge roller and the lower edge roller; determining an included angle offset angle according to the conveyor belt edge angle bisector and the roller reference line angle bisector; determining a global offset area according to the area of a polygonal frame defined by the angle bisector of the edge of the conveyor belt and the angle bisector of the roller reference line and the intersection point set of the edge of the picture to be detected; determining the deviation degree of the conveyor belt according to the included angle deviation angle and the global deviation area; and performing alarm processing according to the deviation degree of the conveyor belt.
In an embodiment, when the processor executes the computer program to implement the step of performing binarization semantic segmentation on the picture to be detected to obtain a semantic segmented picture containing a conveyor belt region and a background, the following steps are specifically implemented:
performing semantic segmentation on the picture to be detected by adopting a semantic segmentation network to obtain a segmented picture; and carrying out black-white binary mapping on the segmented picture to obtain a semantic segmented picture containing a conveyor belt area and a background.
In an embodiment, when the processor executes the computer program to implement the step of performing edge keypoint detection on the picture to be detected to obtain edge keypoints, the following steps are specifically implemented:
and identifying coordinates of the four key points of the picture to be detected by adopting a CPN (compact peripheral network) network so as to obtain edge key points.
In an embodiment, the processor executes the computer program to recognize coordinates of four key points of the picture to be detected by using the CPN network, so as to obtain edge key points. When the steps are carried out, the following steps are concretely realized:
determining the intersection coordinates of the parametric equation of the straight line on the upper edge of the conveyor belt and the parametric equation of the straight line on the lower edge of the conveyor belt, the slope of the parametric equation of the straight line on the upper edge of the conveyor belt and the slope of the parametric equation of the straight line on the lower edge of the conveyor belt; and determining the angular bisector of the edge of the conveyor belt according to the coordinates of the intersection point, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt.
In an embodiment, when the processor executes the computer program to implement the step of performing the roller detection on the picture to be detected to obtain the upper edge roller and the lower edge roller, the following steps are specifically implemented:
carrying out target positioning on the rollers in the to-be-detected drawing by adopting a yolov5 target detection model to obtain the image coordinate position of each roller; calculating the coordinates of the center point of each roller according to the image coordinate position of each roller; the distance from the center point coordinates to the upper edge conveyor and the distance from the center point coordinates to the lower edge conveyor are calculated to determine the upper edge rollers and the lower edge rollers.
In an embodiment, when the processor executes the computer program to implement the step of determining a global offset area according to an area of a polygon frame defined by an intersection set of the conveyor belt edge angular bisector and the roller reference line angular bisector and the edge of the picture to be detected, the following steps are specifically implemented:
determining a set of intersection points of the angular bisector of the edge of the picture to be detected and the angular bisector of the edge of the conveyor belt and the angular bisector of the roller reference line to obtain a polygonal frame; and calculating the area of the polygon frame to obtain a global offset area.
In an embodiment, when the processor executes the computer program to implement the step of determining the degree of conveyor belt offset according to the included angle offset angle and the global offset area, the following steps are specifically implemented:
when the global offset area is larger than an offset area threshold value and the included angle offset angle is smaller than an angle offset threshold value, determining the offset degree of the conveyor belt according to the global offset area; when the global offset area is larger than an offset area threshold value and the included angle offset angle is larger than an angle offset threshold value, determining the offset degree of the conveyor belt according to the included angle offset angle; and when the global deviation area is smaller than a deviation area threshold value and the included angle deviation angle is larger than an angle deviation threshold value, determining the deviation degree of the conveyor belt according to the included angle deviation angle.
The storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk, which can store various computer readable storage media.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative. For example, the division of each unit is only one logic function division, and there may be another division manner in actual implementation. For example, various elements or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented.
The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs. The units in the device of the embodiment of the invention can be merged, divided and deleted according to actual needs. In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a terminal, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (10)

1. The method for detecting the degree of belt deviation is characterized by comprising the following steps:
acquiring a picture of a conveyor belt to obtain a picture to be detected;
performing binarization semantic segmentation on the picture to be detected to obtain a semantic segmentation picture containing a conveyor belt area and a background;
detecting edge key points of the picture to be detected to obtain edge key points;
determining a parameter equation of a straight line according to the edge key points;
determining an angular bisector of the edge of the conveyor belt according to a parameter equation of the straight line;
carrying out roller detection on the picture to be detected to obtain an upper edge roller and a lower edge roller;
determining a roller reference line angle bisector according to the upper edge roller and the lower edge roller;
determining an included angle offset angle according to the conveyor belt edge angle bisector and the roller reference line angle bisector;
determining a global offset area according to the area of a polygonal frame defined by the angle bisector of the edge of the conveyor belt and the angle bisector of the roller reference line and the intersection point set of the edge of the picture to be detected;
determining the deviation degree of the conveyor belt according to the included angle deviation angle and the global deviation area;
and performing alarm processing according to the deviation degree of the conveyor belt.
2. The conveyor belt deviation degree detection method according to claim 1, wherein the performing binarization semantic segmentation on the picture to be detected to obtain a semantic segmented picture containing a conveyor belt region and a background comprises:
performing semantic segmentation on the picture to be detected by adopting a semantic segmentation network to obtain a segmented picture;
and carrying out black-white binary mapping on the segmented picture to obtain a semantic segmented picture containing a conveyor belt area and a background.
3. The method for detecting the degree of belt deviation according to claim 1, wherein the detecting the edge key points of the image to be detected to obtain the edge key points comprises:
and identifying coordinates of the four key points of the picture to be detected by adopting a CPN (compact peripheral network) network so as to obtain edge key points.
4. The method for detecting the degree of belt deviation according to claim 3, wherein said determining the bisector of the belt edge angle according to the parametric equation of the straight line comprises:
determining the intersection coordinates of the parametric equation of the straight line on the upper edge of the conveyor belt and the parametric equation of the straight line on the lower edge of the conveyor belt, the slope of the parametric equation of the straight line on the upper edge of the conveyor belt and the slope of the parametric equation of the straight line on the lower edge of the conveyor belt;
and determining the angular bisector of the edge of the conveyor belt according to the coordinates of the intersection point, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt.
5. The method as claimed in claim 1, wherein the roller detection of the to-be-detected picture to obtain an upper edge roller and a lower edge roller comprises:
carrying out target positioning on the rollers in the to-be-detected drawing by adopting a yolov5 target detection model to obtain the image coordinate position of each roller;
calculating the coordinates of the center point of each roller according to the image coordinate position of each roller;
the distance from the center point coordinates to the upper edge conveyor and the distance from the center point coordinates to the lower edge conveyor are calculated to determine the upper edge rollers and the lower edge rollers.
6. The method for detecting the degree of belt deviation according to claim 1, wherein the determining of the global deviation area according to the area of a polygonal frame defined by the intersection point set of the belt edge angle bisector and the roller reference line angle bisector and the edge of the picture to be detected comprises:
determining a set of intersection points of the angular bisector of the edge of the picture to be detected and the angular bisector of the edge of the conveyor belt and the angular bisector of the roller reference line to obtain a polygonal frame;
and calculating the area of the polygon frame to obtain a global offset area.
7. The method for detecting the degree of belt deviation according to claim 1, wherein the determining the degree of belt deviation according to the included angle deviation angle and the global deviation area includes:
when the global offset area is larger than an offset area threshold value and the included angle offset angle is smaller than an angle offset threshold value, determining the offset degree of the conveyor belt according to the global offset area;
when the global offset area is larger than an offset area threshold value and the included angle offset angle is larger than an angle offset threshold value, determining the offset degree of the conveyor belt according to the included angle offset angle;
and when the global deviation area is smaller than a deviation area threshold value and the included angle deviation angle is larger than an angle deviation threshold value, determining the deviation degree of the conveyor belt according to the included angle deviation angle.
8. Conveyer belt skew degree detection device, its characterized in that includes:
the image acquisition unit is used for acquiring the image of the conveyor belt to obtain an image to be detected;
the segmentation unit is used for carrying out binarization semantic segmentation on the picture to be detected so as to obtain a semantic segmentation picture containing a conveyor belt area and a background;
the key point detection unit is used for detecting edge key points of the picture to be detected to obtain edge key points;
the equation determining unit is used for determining a parameter equation of a straight line according to the edge key points;
the first determining unit is used for determining the angular bisector of the edge of the conveyor belt according to the parameter equation of the straight line;
the roller detection unit is used for carrying out roller detection on the picture to be detected so as to obtain an upper edge roller and a lower edge roller;
the second determining unit is used for determining an angular bisector of a roller reference line according to the upper edge roller and the lower edge roller;
the included angle deviation angle determining unit is used for determining an included angle deviation angle according to the conveyor belt edge angle bisector and the roller reference line angle bisector;
the global offset area determining unit is used for determining a global offset area according to the area of a polygonal frame defined by the intersection point set of the conveyor belt edge angle bisector and the roller shaft reference line angle bisector and the edge of the picture to be detected;
the degree determining unit is used for determining the deviation degree of the conveyor belt according to the included angle deviation angle and the global deviation area;
and the alarm processing unit is used for carrying out alarm processing according to the deviation degree of the conveyor belt.
9. A computer device, characterized in that the computer device comprises a memory, on which a computer program is stored, and a processor, which when executing the computer program implements the method according to any of claims 1 to 7.
10. A storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.
CN202111092682.XA 2021-09-17 2021-09-17 Conveyor belt deviation degree detection method and device, computer equipment and storage medium Pending CN113763375A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111092682.XA CN113763375A (en) 2021-09-17 2021-09-17 Conveyor belt deviation degree detection method and device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111092682.XA CN113763375A (en) 2021-09-17 2021-09-17 Conveyor belt deviation degree detection method and device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113763375A true CN113763375A (en) 2021-12-07

Family

ID=78796259

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111092682.XA Pending CN113763375A (en) 2021-09-17 2021-09-17 Conveyor belt deviation degree detection method and device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113763375A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114313883A (en) * 2022-02-08 2022-04-12 深圳市铁越电气有限公司 Belt deviation automatic detection method and system based on image processing technology
CN116523908A (en) * 2023-06-30 2023-08-01 湖南隆深氢能科技有限公司 Safe production method, system, equipment and medium based on coil coating production line
CN117495858A (en) * 2023-12-29 2024-02-02 合肥金星智控科技股份有限公司 Belt offset detection method, system, equipment and medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060056732A1 (en) * 2004-08-28 2006-03-16 David Holmes Method and apparatus for determining offsets of a part from a digital image
CN110838097A (en) * 2019-11-22 2020-02-25 江苏东方赛云电子科技有限公司 Conveyor belt offset measurement method based on machine vision
CN111597904A (en) * 2020-04-17 2020-08-28 许继集团有限公司 Identification method for inclination of tunnel cable support
CN112330703A (en) * 2020-11-09 2021-02-05 昆山易分蓝电子科技有限公司 Method for positioning center of facial tissue in hardcover gift box production system
CN113283344A (en) * 2021-05-27 2021-08-20 中国矿业大学 Mining conveying belt deviation detection method based on semantic segmentation network
CN113379743A (en) * 2021-08-12 2021-09-10 山东中都机器有限公司 Conveyor abnormity detection method and system based on computer vision

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060056732A1 (en) * 2004-08-28 2006-03-16 David Holmes Method and apparatus for determining offsets of a part from a digital image
CN110838097A (en) * 2019-11-22 2020-02-25 江苏东方赛云电子科技有限公司 Conveyor belt offset measurement method based on machine vision
CN111597904A (en) * 2020-04-17 2020-08-28 许继集团有限公司 Identification method for inclination of tunnel cable support
CN112330703A (en) * 2020-11-09 2021-02-05 昆山易分蓝电子科技有限公司 Method for positioning center of facial tissue in hardcover gift box production system
CN113283344A (en) * 2021-05-27 2021-08-20 中国矿业大学 Mining conveying belt deviation detection method based on semantic segmentation network
CN113379743A (en) * 2021-08-12 2021-09-10 山东中都机器有限公司 Conveyor abnormity detection method and system based on computer vision

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
李福俊等: "车道线实时检测与偏离预警系统设计与研究", 光电子・激光, no. 03, 15 March 2018 (2018-03-15), pages 74 - 80 *
黄梦涛等: "基于机器视觉的镭射膜跑偏检测", 计算机工程与应用, no. 20, 8 July 2016 (2016-07-08), pages 208 - 212 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114313883A (en) * 2022-02-08 2022-04-12 深圳市铁越电气有限公司 Belt deviation automatic detection method and system based on image processing technology
CN114313883B (en) * 2022-02-08 2024-03-12 深圳市铁越电气有限公司 Automatic detection method and system for belt deviation based on image processing technology
CN116523908A (en) * 2023-06-30 2023-08-01 湖南隆深氢能科技有限公司 Safe production method, system, equipment and medium based on coil coating production line
CN116523908B (en) * 2023-06-30 2023-09-12 湖南隆深氢能科技有限公司 Safe production method, system, equipment and medium based on coil coating production line
CN117495858A (en) * 2023-12-29 2024-02-02 合肥金星智控科技股份有限公司 Belt offset detection method, system, equipment and medium

Similar Documents

Publication Publication Date Title
CN113763375A (en) Conveyor belt deviation degree detection method and device, computer equipment and storage medium
US20200019760A1 (en) Three-dimensional living-body face detection method, face authentication recognition method, and apparatuses
US10373320B2 (en) Method for detecting moving objects in a video having non-stationary background
US7639878B2 (en) Shadow detection in images
US10748294B2 (en) Method, system, and computer-readable recording medium for image object tracking
US10909719B2 (en) Image processing method and apparatus
CN112149458A (en) Obstacle detection method, intelligent driving control method, device, medium, and apparatus
CN113781511B (en) Conveyor belt edge wear detection method, conveyor belt edge wear detection device, computer equipment and storage medium
CN113449606B (en) Target object identification method and device, computer equipment and storage medium
Felipe et al. Vision-based liquid level detection in amber glass bottles using OpenCV
CN117037132A (en) Ship water gauge reading detection and identification method based on machine vision
CN113763376B (en) Conveyor belt offset detection method, conveyor belt offset detection device, computer equipment and storage medium
JP3576654B2 (en) Exposure determination method, figure extraction method, and face area determination method
JP6340795B2 (en) Image processing apparatus, image processing system, image processing method, image processing program, and moving body control apparatus
CN113781523A (en) Football detection tracking method and device, electronic equipment and storage medium
WO2024016715A1 (en) Method and apparatus for testing imaging consistency of system, and computer storage medium
CN116503414A (en) Screen defect detection method, device, computer equipment and storage medium
Ge et al. Detection and localization strategy based on YOLO for robot sorting under complex lighting conditions
CN115113630A (en) Bridge passing method and device for unmanned ship, computer equipment and storage medium
CN113888482A (en) Conveyor belt edge defect detection method and device, computer equipment and storage medium
KR101648562B1 (en) Apparatus for detecting moving object
JP3404214B2 (en) Positioning method for circular objects
JPS59201180A (en) Detecting system of feature of contour
US11257238B2 (en) Unsupervised object sizing method for single camera viewing
Kanno et al. Automatic Reading of Flow Meters Based on Classification Deep Neural Network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination