CN115713511A - Detection method and device for conveyor belt and electronic equipment - Google Patents
Detection method and device for conveyor belt and electronic equipment Download PDFInfo
- Publication number
- CN115713511A CN115713511A CN202211448372.1A CN202211448372A CN115713511A CN 115713511 A CN115713511 A CN 115713511A CN 202211448372 A CN202211448372 A CN 202211448372A CN 115713511 A CN115713511 A CN 115713511A
- Authority
- CN
- China
- Prior art keywords
- image
- conveyor belt
- historical
- area
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The application provides a detection method and device of a conveyor belt and electronic equipment. The method comprises the following steps: acquiring a historical image of a conveyor belt; compressing and restoring the historical image to obtain a historical reconstructed image, and constructing a reconstruction model according to the historical image and the historical reconstructed image; acquiring a current image of the conveyor belt, inputting the current image into a reconstruction model for compression processing and reduction processing to obtain a current reconstruction image; and comparing the similarity of the current reconstructed image and the current image, and determining whether the conveyor belt is abnormal according to the similarity. According to the scheme, the abnormal detection work can be achieved through the acquired images of the normally-working conveyor belt, the abnormal samples do not need to participate in a training model, and the samples of the conveyor belt in the normal state can be acquired and can be achieved, so that the detection accuracy of the conveyor belt can be guaranteed to be high.
Description
Technical Field
The application relates to the field of coal conveyor belt detection, in particular to a conveyor belt detection method and device, a computer readable storage medium and electronic equipment.
Background
The coal industry is an important basic industry in China, and the sustainable development of the coal industry is related to national economic health development and national energy safety. In the process of coal mine collection, faults such as conveyor belt deviation, conveyor belt tearing caused by large foreign matter mixing and the like sometimes occur, safety accidents are easily caused, and great economic loss is caused.
At present, the detection method aiming at the abnormity of the conveyor belt mainly comprises the detection of a human engineering method, the detection of a ray method and the detection of a video image. The manual detection method needs a dedicated worker to pay attention to whether the condition of the conveyor belt is abnormal all the day long, and needs high labor cost; the ray detection method needs professional equipment to identify foreign matters, needs higher cost investment and maintenance, and cannot solve the problem of deviation of the conveyor belt; the video image detection method can monitor the conveyor belt in real time by using the conveyor belt video shot by the camera through computer vision related knowledge, can automatically convey alarm information when an abnormal condition occurs, has low required labor cost and economic investment cost, and is the most widely applied conveyor belt abnormality detection method at present.
The identification method based on video image detection mainly comprises two modes of manually designing image features and automatically extracting the features through deep learning. Because the manual feature extraction has the limitations of complex design and the like, the automatic feature extraction mode of the current deep learning method is widely applied. However, in the coal collection process, a large amount of economic loss is caused by the abnormal condition of the conveyor belt, so that it is not practical to make the conveyor belt in an abnormal condition in order to collect the abnormal sample training model, and therefore, it is difficult to collect the abnormal sample of the conveyor belt in the abnormal condition at present, so that the detection accuracy of the conveyor belt is low.
Disclosure of Invention
The application mainly aims to provide a detection method and device for a conveyor belt, a computer readable storage medium and electronic equipment, so as to solve the problem that the detection accuracy of the conveyor belt is low due to the fact that an abnormal sample of the conveyor belt in an abnormal state is difficult to collect in the prior art.
According to an aspect of an embodiment of the present invention, there is provided a detection method of a conveyor belt, including: acquiring a historical image of the conveyor belt, wherein the historical image is acquired by image acquisition equipment; compressing and restoring the historical image to obtain a historical reconstructed image, and constructing a reconstruction model according to the historical image and the historical reconstructed image; acquiring a current image of the conveyor belt, and inputting the current image into the reconstruction model for compression processing and reduction processing to obtain a current reconstruction image; and comparing the similarity of the current reconstructed image and the current image, and determining whether the conveyor belt is abnormal or not according to the similarity.
Optionally, in the process of constructing a reconstruction model from the historical images and the historical reconstructed images, the method further includes: constructing an encoder module and a decoder module, wherein the encoder module is composed of a first number of convolutional neural network layers, the decoder module is composed of a second number of convolutional neural network layers, the encoder module is used for compressing the history images of a first size into the history images of a second size, and the decoder module is used for restoring the history images of the second size into the history reconstructed images of the first size, wherein the first size is larger than the second size; acquiring first mapping information of the encoder module, and acquiring second mapping information of the decoder module, wherein the first mapping information refers to a mapping relation between the history images of the first size and the history images of the second size, and the second mapping information refers to a mapping relation between the history images of the second size and the history reconstructed images of the first size; and determining a loss function of the reconstruction model according to the historical image, the first mapping information and the second mapping information.
Optionally, after determining a loss function of the reconstructed model according to the historical image, the first mapping information and the second mapping information, the method further comprises: obtaining a plurality of first loss functions, wherein the first loss functions are the loss functions corresponding to the first mapping information, and the plurality of first loss functions are obtained by inputting a third number of historical images serving as training sets into the reconstruction model; obtaining an average value of a plurality of second loss functions, wherein the second loss functions are the loss functions corresponding to the second mapping information, and the plurality of second loss functions are obtained by inputting a fourth number of historical images into the reconstruction model as a verification set; and determining the first loss function with the minimum difference value with the average value as a target loss function, and determining the first mapping information corresponding to the target loss function as target mapping information.
Optionally, determining whether the conveyor belt is abnormal according to the similarity includes: determining that the conveyor belt is normal when the similarity is greater than or equal to a similarity threshold; under the condition that the similarity is smaller than the similarity threshold value, acquiring a first area and a second area in the current image, wherein the first area is an area where the conveyor belt is located, and the second area is an area where the conveyor belt is not located; and determining whether the conveyor belt is abnormal according to the position of a target area, wherein the target area refers to an area of which the similarity between the current image and the current reconstructed image is smaller than the similarity threshold value.
Optionally, the obtaining a first region and a second region in the current image includes: acquiring a plurality of preset coordinate points, wherein the preset coordinate points are position coordinate points used for marking the first area in the current image; determining a region surrounded by a plurality of predetermined coordinate points as the first region; and determining the area except the first area in the current image as the second area.
Optionally, determining whether the conveyor belt is abnormal according to the position of the target area includes: determining that the conveyor belt is abnormal if the target area is within the first area; determining that the conveyor belt is normal if the target area is within the second area; and determining that the conveyor belt is abnormal when the target area is partially in the first area and partially in the second area.
Optionally, after determining whether the conveyor belt is abnormal according to the similarity, the method further includes: and under the condition that the conveyor belt is determined to be abnormal, determining the abnormal reason of the conveyor belt according to the current image and the current reconstructed image.
According to another aspect of the embodiments of the present invention, there is also provided a detection apparatus for a conveyor belt, including: the first acquisition unit is used for acquiring a historical image of the conveyor belt, wherein the historical image is acquired by the image acquisition equipment; the processing unit is used for compressing and restoring the historical image to obtain a historical reconstructed image and constructing a reconstruction model according to the historical image and the historical reconstructed image; the second acquisition unit is used for acquiring a current image of the conveyor belt, inputting the current image into the reconstruction model for compression processing and reduction processing to obtain a current reconstruction image; a detection unit for comparing the similarity between the current reconstructed image and the current image, and determining whether the conveyor belt is abnormal according to the similarity
According to still another aspect of embodiments of the present invention, there is also provided a computer-readable storage medium including a stored program, wherein the program performs any one of the methods.
According to still another aspect of the embodiments of the present invention, there is also provided an electronic device, including: one or more processors, memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described herein.
In the embodiment of the invention, the historical image of the conveyor belt is firstly obtained, then the historical image is compressed and restored to obtain the historical reconstruction image, the reconstruction model is constructed according to the historical image and the historical reconstruction image, then the current image of the conveyor belt is obtained, the current image is input into the reconstruction model to be compressed and restored to obtain the current reconstruction image, finally the similarity of the current reconstruction image and the current image is compared, and whether the conveyor belt is abnormal or not is determined according to the similarity. According to the detection scheme, the abnormal detection work can be realized through the acquired image of the normally-working conveyor belt, the abnormal sample does not need to participate in the training model, and the sample of the conveyor belt in the normal state can be acquired and can be realized, so that the detection accuracy of the conveyor belt can be ensured to be high.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:
FIG. 1 shows a schematic flow diagram of a method of inspecting a conveyor belt according to an embodiment of the application;
FIG. 2 shows a schematic structural diagram of a reconstructed model;
FIG. 3 shows a schematic flow chart of training a reconstructed model;
FIG. 4 shows a flow diagram of a test reconstruction model;
FIG. 5 shows a schematic flow diagram of another test reconstruction model;
fig. 6 shows a schematic structural diagram of a detection device of a conveyor belt according to an embodiment of the application.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It will be understood that when an element such as a layer, film, region, or substrate is referred to as being "on" another element, it can be directly on the other element or intervening elements may also be present. Also, in the specification and claims, when an element is described as being "connected" to another element, the element may be "directly connected" to the other element or "connected" to the other element through a third element.
Because of the limitations of complicated design and the like of manually extracting features, the automatic feature extraction mode of the current deep learning method is widely applied, and the main process can be summarized as follows: 1. the image information in a section of video is sequentially extracted according to the frame number, the conveyor belt images in a normal state are used as normal training samples, the conveyor belt images in an abnormal state are used as heterogeneous training samples and input into a deep learning model for training, and a relevant mapping relation is obtained, wherein the mapping relation can learn different representation information of the normal samples and the abnormal samples. 2. And (3) dividing the learned features in the step (1) into normal class samples and abnormal class samples through a classifier, and realizing the detection of the abnormal samples. However, in the coal collection process, a large amount of economic loss is caused by the abnormal condition of the conveyor belt, so that it is not practical to make the conveyor belt in an abnormal condition in order to collect the abnormal sample training model, and therefore, it is difficult to collect the abnormal sample of the conveyor belt in the abnormal condition at present, so that the detection accuracy of the conveyor belt is low.
As mentioned in the background art, it is difficult to collect an abnormal sample of a conveyor belt in an abnormal state in the prior art, which results in a low detection accuracy of the conveyor belt.
According to an embodiment of the present application, a method of inspecting a conveyor belt is provided.
Fig. 1 is a flowchart of a detection method of a conveyor belt according to an embodiment of the present application. As shown in fig. 1, the method comprises the steps of:
step S101, acquiring a historical image of a conveyor belt, wherein the historical image is acquired by image acquisition equipment;
specifically, historical video information during normal transmission of the conveyor belt (including historical images in the historical video information) can be collected by an image collecting device installed above the conveyor belt, and the historical video information can be collected for a predetermined period of time, wherein the predetermined period of time can be 1 hour or other time periods. And intercepting a sample of the historical image from the historical video information, wherein the intercepting interval can be set according to actual conditions, and is 1 second for example.
Step S102, carrying out compression processing and reduction processing on the historical image to obtain a historical reconstruction image, and constructing a reconstruction model according to the historical image and the historical reconstruction image;
in order to further more efficiently construct a reconstruction model for more efficiently reconstructing an image according to the reconstruction model (the reconstruction includes a compression process and a restoration process), in an embodiment of the present application, in constructing the reconstruction model according to the history image and the history reconstruction image, the method further includes: constructing an encoder module and a decoder module, wherein the encoder module is composed of a first number of convolutional neural network layers, the decoder module is composed of a second number of convolutional neural network layers, the encoder module is configured to compress the history images of a first size into the history images of a second size, and the decoder module is configured to restore the history images of the second size into the history reconstructed images of the first size, wherein the first size is larger than the second size; acquiring first mapping information of the encoder module, the first mapping information indicating a mapping relationship between the history image of the first size and the history image of the second size, and acquiring second mapping information of the decoder module, the second mapping information indicating a mapping relationship between the history image of the second size and the history reconstructed image of the first size; and determining a loss function of the reconstructed model according to the historical image, the first mapping information and the second mapping information.
In one embodiment, the first number and the second number may be the same number.
Optionally, sparsity limitation may be performed on neurons of the neural network model, and sparsity constraint may be applied to the historical image of the first size, or to the historical image of the second size, or to the historical reconstructed image of the first size. The conventional self-encoder is expected to maximize the reconstruction loss of abnormal samples so as to distinguish normal samples from abnormal samples, however, in the conveyor belt abnormality detection task related to the patent, the conventional self-encoder structure has the problem that the reconstruction loss of part of abnormal samples is very small, namely, the abnormal image samples are reconstructed into images which are distributed close to the normal sample data structure.
Specifically, the structure of the reconstruction model is shown in fig. 2, where the encoder module may be formed by a convolutional neural network composed of 3 convolutional neural network layers, and the mapping from the history image to the feature code is implemented by means of convolutional mapping by using RGB image samples of 3 channels. Correspondingly, the decoder module is similar to the encoder module and is also composed of 3 convolutional neural network layers, can map the feature codes to the historical reconstructed images of 3 channels, updates the mapping information (including the first mapping information and the second mapping information) according to the encoder module and the decoder module of the reconstruction model, and can determine the loss function according to the historical images and the historical reconstructed images.
The encoder module and the decoder module can adopt an activation function to optimize, because the swish function can only play a role in a deep network, the Hardswish activation function can be used as the activation function of the self-encoder, the Hardswish activation function has the advantages of good numerical stability, high calculation speed and the like, and the expression of the Hardswish activation function is as follows:
the traditional convolutional neural network generally adopts functions such as Sigmoid, tanh, reLU and the like as an activation function, and the main purpose of the traditional convolutional neural network is to introduce nonlinear characteristics into the neural network, namely to change a linear regression model into a nonlinear model for application so as to solve the nonlinear problem that a plurality of hidden layers exist among layers of the neural network. Among many activation functions, the swish function can be regarded as a smooth function between a linear function and a ReLU function, which is superior to the ReLU function in model effect. However, although the swish nonlinear activation function improves the detection accuracy, the swish nonlinear activation function is suitable in an academic level and is not suitable in an application level, because the swish function has higher calculation cost and more complex derivation, and is slow in calculation during quantification and difficult to deploy in a conveyor belt abnormality detection task.
In contrast, the Hardswinh nonlinear activation function has no obvious difference in accuracy, but has great advantages in practical application. One is that in quantization mode, the hardwinsh function eliminates the potential numerical precision loss due to the different implementations of the approximate Sigmoid shape. Secondly, in practice, the memory access times are reduced through the segmentation function, so that the operation cost of the model can be further reduced.
In order to shorten the distribution distance between the historical reconstructed image of the normal sample and the original image sample (historical image), a gradient descent method can be adopted, firstly, a loss function of a reconstruction model is designed, and the formula is as follows:
wherein Loss represents a Loss function, x represents an input historical image, F (eta)) represents first mapping information, G (eta)) represents second mapping information, lambda represents weight for controlling sparsity, j represents a jth neuron in feature coding, c represents dimension of feature coding, and rho represents a sparsity parameter,
represents the average activation of the feature-encoding neurons,
is rho and
relative entropy of (d); in this patent, ρ is set to 0.07 and λ is set to 0.3, but it may be set to other values according to actual conditions. ρ is a manually set hyper-parameter, and a constant close to 0 may be selected. The relative entropy, also known as KL divergence, can also be expressed as
In particular, an auto-encoder mode can be introduced, and during the training phase of the reconstruction model, the manifold distribution of normal samples in a potential space is learned without supervision. As shown in fig. 3, a training sample (history image) is compressed by an encoder module of the self-encoder, a high-dimensional training sample is compressed into a low-dimensional form to obtain a feature code, and the compressed data is restored by a decoder module of the self-encoder to obtain a history reconstructed image. Subsequently, the mapping information (including the first mapping information and the second mapping information) of the self-encoder may be iteratively updated according to the loss function, so that the historical image and the historical reconstructed image are close to each other, in the test stage of the reconstructed model, as shown in fig. 4, the test sample (the historical image or the current image) obtains the reconstructed image (the historical reconstructed image or the current reconstructed image) of the test sample through the mapping information obtained in the training stage, and then an output result is obtained by comparing the distribution difference (similarity) between the test sample and the reconstructed image, if the distribution difference exceeds a set threshold, the test sample may be determined as an abnormal sample, and if the distribution difference does not exceed the set threshold, the test sample may be determined as a normal sample.
In order to further train the reconstructed model to ensure that the accuracy of the obtained reconstructed model is high, and then the reconstructed model can be reconstructed more efficiently and accurately, in another embodiment of the present application, after determining the loss function of the reconstructed model according to the historical image, the first mapping information, and the second mapping information, the method further includes: obtaining a plurality of first loss functions, wherein the first loss functions are the loss functions corresponding to the first mapping information, and the plurality of first loss functions are obtained by inputting a third number of the historical images into the reconstruction model as a training set; obtaining an average value of a plurality of second loss functions, the second loss functions being the loss functions corresponding to the second mapping information, the plurality of second loss functions being obtained by inputting a fourth number of the history images as a verification set into the reconstruction model; and determining the first loss function with the minimum difference value with the average value as a target loss function, and determining the first mapping information corresponding to the target loss function as target mapping information.
Specifically, the data set of the acquired historical images may be divided into a training set and a verification set according to a predetermined ratio, which may be set according to an actual situation, for example, the ratio of 8. The collected historical images are preprocessed, and because the coal mine conveyor belt mainly relates to brightness change and does not relate to factors such as translation and rotation, the preprocessing can at least comprise brightness change processing.
Specifically, the target mapping information may be determined by an iterative process, and the following specific process is described:
in a first iteration, first initializing first mapping information and second mapping information of a reconstructed model, including: randomly selecting one historical image in a training set, inputting the selected historical image into a reconstruction model, obtaining a historical reconstruction image through initialized first mapping information and initialized second mapping information, calculating a gradient direction by adopting a loss function, updating the first mapping information and the second mapping information through a set learning rate, wherein the learning rate can be set according to the actual situation, such as 0.001, taking the two updated mapping information as the first mapping information and the second mapping information of the initialized reconstruction model, inputting the other historical image in the training set into the reconstruction model, repeating the updating step until all training samples participating in training in the training set obtain the corresponding first mapping information and second mapping information, and storing the corresponding first mapping information and second mapping information;
and inputting all historical images on the verification set into the reconstruction model, and obtaining historical reconstruction images corresponding to the historical images through mapping information obtained through training. Calculating the distribution difference between the historical images of all the verification sets and the corresponding historical reconstruction images to obtain corresponding second loss functions, and solving an average value, wherein the formula is as follows:
where L represents the average, n represents the number of history images in the verification set, x i Representing the ith historical image in the validation set;
and repeating the initialization step and the verification step, wherein in the second iteration, the initialized first mapping information and the initialized second mapping information are the first mapping information and the second mapping information stored in the first iteration, and the iteration can be performed for multiple times subsequently to ensure that the accuracy of the first mapping information and the second mapping information is high, and the iteration times can be set according to the actual situation, for example, 500 times.
Finally, selecting a first loss function corresponding to the jth round with the minimum difference value of the average values as a target loss function, and selecting first mapping information corresponding to the target loss function as target mapping information, wherein L is the iteration number of 500 j ＝min(L 1 ,L 2 ,...L k ) And k represents the number of iterations.
Step S103, acquiring a current image of the conveyor belt, and inputting the current image into the reconstruction model for compression processing and reduction processing to obtain a current reconstruction image;
and step S104, comparing the similarity of the current reconstructed image and the current image, and determining whether the conveyor belt is abnormal according to the similarity.
Specifically, after the reconstruction model is constructed, the reconstruction model may be tested, as shown in fig. 5, in the testing stage, a classifier of a normal sample (a normal historical image) and an abnormal sample (an abnormal historical image) is constructed mainly through trained target mapping information to realize detection of the abnormal sample, and the classifier may be realized by setting a judgment threshold (i.e., a similarity threshold) of the abnormal sample.
For the parameters trained in the training stage of the reconstruction model, the distribution difference between the history images which normally work in an ideal state and the history reconstruction images output by the reconstruction model is smaller (the similarity is larger) or even 0, and the distribution difference between the history images which abnormally work and the history reconstruction images output by the reconstruction model is larger (the similarity is smaller), so that whether the conveyor belt is abnormal or not can be detected by setting a corresponding threshold, if the similarity exceeds the similarity threshold, the history images are determined to be abnormal samples, and if the similarity exceeds the similarity threshold, the history images are determined to be normal samples.
Specifically, the threshold used by the classifier can be calculated by the following formula: Ω = max (L) 1 ,L 2 ,...,L k )+η，L 1 、L 2 And L k The loss value of the image in the verification set and the original image after the image is reconstructed by the self-encoder is verified, wherein eta is a set parameter and can be 0.5 or any other feasible value.
Since the field of view of the image capturing device in reality includes the second area, the distribution difference of the abnormal samples may be caused by non-conveyor belt abnormality, for example, a bird flies through the second area, in which case exceeding the similarity threshold may not need to trigger an alarm, so that it may be further determined whether the conveyor belt is abnormal efficiently and accurately according to the position of the target area, in a specific embodiment of the present application, determining whether the conveyor belt is abnormal according to the similarity includes: determining that the conveyor belt is normal when the similarity is greater than or equal to a similarity threshold; under the condition that the similarity is smaller than the similarity threshold value, acquiring a first area and a second area in the current image, wherein the first area is an area where the conveyor belt is located, and the second area is an area where the conveyor belt is not located; and determining whether the conveyor belt is abnormal according to the position of a target area, wherein the target area is an area of which the similarity between the current image and the current reconstructed image is smaller than the similarity threshold.
In order to further efficiently and accurately determine the first region and the second region in the current image so as to further efficiently and accurately determine whether the conveyor belt is abnormal, in another specific embodiment of the present application, the acquiring the first region and the second region in the current image includes: acquiring a plurality of preset coordinate points, wherein the preset coordinate points are position coordinate points used for marking the first area in the current image; determining a region surrounded by a plurality of the predetermined coordinate points as the first region; and determining the area except the first area in the current image as the second area.
Specifically, the position coordinate point of the first region, which displays a plurality of predetermined coordinate points such as an upper left coordinate point, an upper right coordinate point, a lower left coordinate point, and a lower right coordinate point in the image capturing apparatus, may be manually marked.
Subtracting the gray value of the pixel point corresponding to the historical image of the abnormal sample and the historical image of the previous frame, and then taking the absolute value to obtain a difference image, wherein the formula can be D n (x,y)＝f n (x,y)-f n-1 (x, y) wherein D n (x, y) represents a pixel point corresponding to the differential image, f n (x, y) represents a pixel point corresponding to the current image, f n-1 And (x, y) represents a pixel point corresponding to the historical image of the previous frame of the current image.
The absolute value D of the difference value n (x, y) binarizing to obtain
Where T is a threshold hyperparameter, which may be 25.
The set target mapping information of the model and the reconstruction model of the classifier can be put on a server to operate, the video information of the conveyor belt acquired by the image acquisition equipment is uploaded to the server, the server acquires the obtained video information into a current image in a mode of 20 frames, and the current image is input into the reconstruction model and the classifier as an input image. When the classifier identifies an abnormal sample, the abnormality is further performed by an inter-frame difference method. If the abnormal input image is determined to be abnormal, the abnormal input image is stored in a server, PLC variables are written in through an OPC interface, the conveyor belt is paused through a set PLC control program, an alarm signal is sent, and the warning function is achieved through corresponding alarm equipment. On the other hand, the stored abnormal image and alarm information can be uploaded to the integrated management and control platform, and related personnel are assisted to judge the abnormal information.
In order to further efficiently and accurately determine whether the conveyor belt is abnormal according to the position of the target area of the conveyor belt in the case that the conveyor belt is preliminarily determined to be abnormal according to the similarity threshold, another specific embodiment of the present application, in which determining whether the conveyor belt is abnormal according to the position of the target area includes: determining that the conveyor belt is abnormal when the target area is within the first area; determining that the conveyor belt is normal if the target area is within the second area; and determining that the conveyor belt is abnormal when part of the target area is in the first area and part of the target area is in the second area.
In another embodiment of the present application, after determining whether the conveyor belt is abnormal according to the similarity, the method further includes: and if the conveyor belt is determined to be abnormal, determining the abnormal reason of the conveyor belt according to the current image and the current reconstructed image.
Specifically, in the case where it is determined that the conveyor belt is abnormal, it may be determined whether the conveyor belt is off tracking or not, in the case where the conveyor belt is off tracking, it may be determined whether the conveyor belt is off tracking or not, in the case where the conveyor belt is not off tracking, it may be determined whether the conveyor belt has a foreign object or not, in the case where the foreign object is present, it may be determined that the abnormal cause is the foreign object on the conveyor belt.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than here.
According to the method, firstly, a historical image of the conveyor belt is obtained, then the historical image is subjected to compression processing and reduction processing to obtain a historical reconstruction image, a reconstruction model is constructed according to the historical image and the historical reconstruction image, then a current image of the conveyor belt is obtained, the current image is input into the reconstruction model to be subjected to compression processing and reduction processing to obtain a current reconstruction image, finally, the similarity of the current reconstruction image and the similarity of the current image are compared, and whether the conveyor belt is abnormal or not is determined according to the similarity. According to the detection scheme, the abnormal detection work can be realized through the acquired image of the normally-working conveyor belt, the abnormal sample does not need to participate in the training model, and the sample of the conveyor belt in the normal state can be acquired and can be realized, so that the detection accuracy of the conveyor belt can be ensured to be high.
The embodiment of the present application further provides a detection device for a conveyor belt, and it should be noted that the detection device for a conveyor belt according to the embodiment of the present application may be used to execute the detection method for a conveyor belt according to the embodiment of the present application. The following describes a detection device for a conveyor belt according to an embodiment of the present application.
Fig. 6 is a schematic view of a detection device of a conveyor belt according to an embodiment of the present application. As shown in fig. 6, the apparatus includes:
a first acquisition unit 10, configured to acquire a history image of the conveyor belt, where the history image is acquired by an image acquisition device;
a processing unit 20, configured to perform compression processing and reduction processing on the historical image to obtain a historical reconstructed image, and construct a reconstruction model according to the historical image and the historical reconstructed image;
in order to further more efficiently construct a reconstruction model for more efficiently reconstructing an image according to the reconstruction model (the reconstruction includes a compression process and a restoration process), in an embodiment of the present application, the apparatus further includes a construction unit, a third obtaining unit and a first determining unit, the construction unit is configured to construct an encoder module and a decoder module in the process of constructing the reconstruction model according to the historical image and the historical reconstruction image, wherein the encoder module is composed of a first number of convolutional neural network layers, the decoder module is composed of a second number of convolutional neural network layers, the encoder module is configured to compress the historical image of a first size into the historical image of a second size, and the decoder module is configured to restore the historical image of the second size into the historical reconstruction image of the first size, wherein the first size is larger than the second size; a third acquiring unit configured to acquire first mapping information of the encoder module, the first mapping information being a mapping relationship between the history image of the first size and the history image of the second size, and acquire second mapping information of the decoder module, the second mapping information being a mapping relationship between the history image of the second size and the history reconstructed image of the first size; the first determining unit is configured to determine a loss function of the reconstructed model based on the history image, the first mapping information, and the second mapping information.
In order to further train the reconstructed model to ensure that the accuracy of the obtained reconstructed model is high, and then the reconstructed model can be reconstructed more efficiently and accurately, in another embodiment of the present application, the apparatus further includes a fourth obtaining unit, a fifth obtaining unit, and a second determining unit, where the fourth obtaining unit is configured to obtain a plurality of first loss functions after determining a loss function of the reconstructed model according to the historical images, the first mapping information, and the second mapping information, the first loss functions being the loss functions corresponding to the first mapping information, and the first loss functions being obtained by inputting a third number of the historical images into the reconstructed model as a training set; a fifth acquiring unit configured to acquire an average value of a plurality of second loss functions, the second loss functions being the loss functions corresponding to the second mapping information, the plurality of second loss functions being obtained by inputting a fourth number of the history images as a verification set into the reconstruction model; the second determining unit is configured to determine the first loss function having the smallest difference from the average value as a target loss function, and determine the first mapping information corresponding to the target loss function as target mapping information.
A second obtaining unit 30, configured to obtain a current image of the conveyor belt, and input the current image into the reconstruction model to perform compression processing and restoration processing, so as to obtain a current reconstruction image;
and a detection unit 40, configured to compare similarity between the current reconstructed image and the current image, and determine whether the conveyor belt is abnormal according to the similarity.
Since the visual field range of the image acquisition device in reality shooting will include the second area, the distribution difference of the abnormal samples may be caused by non-conveyor belt abnormality, for example, birds fly through the second area, in which case exceeding the similarity threshold may not need to trigger an alarm, so that whether the conveyor belt is abnormal or not may be further efficiently and accurately determined according to the position of the target area; the acquisition module is used for acquiring a first area and a second area in the current image under the condition that the similarity is smaller than the similarity threshold, wherein the first area is an area where the conveyor belt is located, and the second area is an area where the conveyor belt is not located; the second determining module is used for determining whether the conveyor belt is abnormal according to the position of a target area, wherein the target area is an area where the similarity between the current image and the current reconstructed image is smaller than the similarity threshold.
In order to further efficiently and accurately determine a first area and a second area in a current image so as to further efficiently and accurately determine whether a conveyor belt is abnormal, in another specific embodiment of the present application, the obtaining module includes an obtaining sub-module, a first determining sub-module, and a second determining sub-module, the obtaining sub-module is configured to obtain a plurality of predetermined coordinate points, and the predetermined coordinate points are position coordinate points used for marking the first area in the current image; the first determining submodule is used for determining a region surrounded by a plurality of the preset coordinate points as the first region; the second determining sub-module is configured to determine a region other than the first region in the current image as the second region.
In order to further efficiently and accurately determine whether the conveyor belt is abnormal according to the position of the target area of the conveyor belt when the conveyor belt is preliminarily determined to be abnormal according to the similarity threshold, in another specific embodiment of the present application, the second determining module includes a third determining submodule, a fourth determining submodule and a fifth determining submodule, and the third determining submodule is configured to determine that the conveyor belt is abnormal when the target area is in the first area; the fourth determining submodule is used for determining that the conveyor belt is normal under the condition that the target area is in the second area; the fifth determining submodule is configured to determine that the conveyor belt is abnormal when the target area is partially within the first area and partially within the second area.
In another embodiment of the present application, the apparatus further comprises a third determining unit, configured to determine the abnormality cause of the conveyor belt according to the current image and the current reconstructed image, after determining whether the conveyor belt is abnormal according to the similarity, and in a case where the conveyor belt is determined to be abnormal, determine the abnormality cause of the conveyor belt according to the current image and the current reconstructed image.
In the device, the first acquisition unit is used for acquiring a historical image of the conveyor belt, the processing unit is used for performing compression processing and reduction processing on the historical image to obtain a historical reconstruction image and constructing a reconstruction model according to the historical image and the historical reconstruction image, the second acquisition unit is used for acquiring a current image of the conveyor belt, inputting the current image into the reconstruction model to perform compression processing and reduction processing to obtain a current reconstruction image, and the detection unit is used for comparing the similarity between the current reconstruction image and the current image and determining whether the conveyor belt is abnormal or not according to the similarity. According to the detection scheme, the abnormal detection work can be realized through the acquired image of the normally-working conveyor belt, the abnormal sample does not need to participate in the training model, and the sample of the conveyor belt in the normal state can be acquired and can be realized, so that the detection accuracy of the conveyor belt can be ensured to be high.
The detection device of the conveyor belt comprises a processor and a memory, wherein the first acquisition unit, the processing unit, the second acquisition unit, the detection unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, and the accuracy of the detection of the conveyor belt is improved by adjusting the kernel parameters.
The memory may include volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM), including at least one memory chip.
An embodiment of the present invention provides a computer-readable storage medium on which a program is stored, the program implementing the above-described conveyor belt detection method when executed by a processor.
The embodiment of the invention provides a processor, which is used for running a program, wherein the detection method of the conveyor belt is executed when the program runs.
The present application also provides an electronic device comprising one or more processors, memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing any of the above-described methods.
An embodiment of the present invention provides an apparatus, where the apparatus includes a processor, a memory, and a program that is stored in the memory and is executable on the processor, and when the processor executes the program, at least the following steps are implemented:
step S101, acquiring a historical image of a conveyor belt, wherein the historical image is acquired by image acquisition equipment;
step S102, carrying out compression processing and reduction processing on the historical image to obtain a historical reconstruction image, and constructing a reconstruction model according to the historical image and the historical reconstruction image;
step S103, acquiring a current image of the conveyor belt, and inputting the current image into the reconstruction model for compression processing and reduction processing to obtain a current reconstruction image;
and step S104, comparing the similarity of the current reconstructed image and the current image, and determining whether the conveyor belt is abnormal according to the similarity.
The device herein may be a server, a PC, a PAD, a mobile phone, etc.
The present application further provides a computer program product adapted to perform a program of initializing at least the following method steps when executed on a data processing device:
step S101, acquiring a historical image of a conveyor belt, wherein the historical image is acquired by image acquisition equipment;
step S102, carrying out compression processing and reduction processing on the historical image to obtain a historical reconstruction image, and constructing a reconstruction model according to the historical image and the historical reconstruction image;
step S103, acquiring a current image of the conveyor belt, inputting the current image into the reconstruction model for compression processing and reduction processing to obtain a current reconstruction image;
and step S104, comparing the similarity of the current reconstructed image and the current image, and determining whether the conveyor belt is abnormal according to the similarity.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the above-described division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may also be implemented in the form of a software functional unit.
The integrated unit may be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the above methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, and various media capable of storing program codes.
From the above description, it can be seen that the above-mentioned embodiments of the present application achieve the following technical effects:
1) The detection method of the conveyor belt comprises the steps of firstly obtaining a historical image of the conveyor belt, then conducting compression processing and reduction processing on the historical image to obtain a historical reconstruction image, constructing a reconstruction model according to the historical image and the historical reconstruction image, then obtaining a current image of the conveyor belt, inputting the current image into the reconstruction model to conduct compression processing and reduction processing to obtain a current reconstruction image, finally comparing the similarity of the current reconstruction image and the current image, and determining whether the conveyor belt is abnormal or not according to the similarity. According to the detection scheme, the abnormal detection work can be realized through the acquired image of the normally-working conveyor belt, the abnormal sample does not need to participate in the training model, and the sample of the conveyor belt in the normal state can be acquired and can be realized, so that the detection accuracy of the conveyor belt can be ensured to be high.
2) According to the detection device of the conveyor belt, the first acquisition unit is used for acquiring a historical image of the conveyor belt, the processing unit is used for performing compression processing and reduction processing on the historical image to obtain a historical reconstruction image, a reconstruction model is constructed according to the historical image and the historical reconstruction image, the second acquisition unit is used for acquiring a current image of the conveyor belt, the current image is input into the reconstruction model to be subjected to compression processing and reduction processing to obtain a current reconstruction image, the detection unit is used for comparing the similarity of the current reconstruction image and the current image, and whether the conveyor belt is abnormal or not is determined according to the similarity. According to the detection scheme, the abnormal detection work can be realized through the acquired image of the normally-working conveyor belt, the abnormal sample does not need to participate in the training model, and the sample of the conveyor belt in the normal state can be acquired and can be realized, so that the detection accuracy of the conveyor belt can be ensured to be high.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A method of inspecting a conveyor belt, comprising:
acquiring a historical image of the conveyor belt, wherein the historical image is acquired by image acquisition equipment;
compressing and restoring the historical image to obtain a historical reconstruction image, and constructing a reconstruction model according to the historical image and the historical reconstruction image;
acquiring a current image of the conveyor belt, and inputting the current image into the reconstruction model for compression processing and reduction processing to obtain a current reconstruction image;
and comparing the similarity of the current reconstructed image and the current image, and determining whether the conveyor belt is abnormal according to the similarity.
2. The method of claim 1, wherein in constructing a reconstruction model from the historical images and the historical reconstructed images, the method further comprises:
constructing an encoder module and a decoder module, wherein the encoder module is composed of a first number of convolutional neural network layers, the decoder module is composed of a second number of convolutional neural network layers, the encoder module is used for compressing the historical images of a first size into the historical images of a second size, and the decoder module is used for restoring the historical images of the second size into the historical reconstructed images of the first size, wherein the first size is larger than the second size;
acquiring first mapping information of the encoder module, and acquiring second mapping information of the decoder module, wherein the first mapping information refers to a mapping relation between the history images of the first size and the history images of the second size, and the second mapping information refers to a mapping relation between the history images of the second size and the history reconstructed images of the first size;
and determining a loss function of the reconstruction model according to the historical image, the first mapping information and the second mapping information.
3. The method of claim 2, wherein after determining a loss function for the reconstructed model from the historical image, the first mapping information, and the second mapping information, the method further comprises:
obtaining a plurality of first loss functions, wherein the first loss functions are the loss functions corresponding to the first mapping information, and the plurality of first loss functions are obtained by inputting a third number of historical images serving as training sets into the reconstruction model;
obtaining an average value of a plurality of second loss functions, wherein the second loss functions are the loss functions corresponding to the second mapping information, and the plurality of second loss functions are obtained by inputting a fourth number of historical images into the reconstruction model as a verification set;
and determining the first loss function with the minimum difference value with the average value as a target loss function, and determining the first mapping information corresponding to the target loss function as target mapping information.
4. The method of claim 1, wherein determining whether the conveyor belt is abnormal based on the similarity comprises:
determining that the conveyor belt is normal when the similarity is greater than or equal to a similarity threshold;
under the condition that the similarity is smaller than the similarity threshold value, acquiring a first area and a second area in the current image, wherein the first area is an area where the conveyor belt is located, and the second area is an area where the conveyor belt is not located;
and determining whether the conveyor belt is abnormal according to the position of a target area, wherein the target area refers to an area of which the similarity between the current image and the current reconstructed image is smaller than the similarity threshold value.
5. The method of claim 4, wherein obtaining the first region and the second region in the current image comprises:
acquiring a plurality of preset coordinate points, wherein the preset coordinate points are position coordinate points used for marking the first area in the current image;
determining a region surrounded by a plurality of predetermined coordinate points as the first region;
and determining the area except the first area in the current image as the second area.
6. The method of claim 4, wherein determining whether the conveyor belt is abnormal based on where the target area is located comprises:
determining that the conveyor belt is abnormal if the target area is within the first area;
determining that the conveyor belt is normal if the target area is within the second area;
and determining that the conveyor belt is abnormal when the target area is partially in the first area and partially in the second area.
7. The method of claim 1, wherein after determining whether the conveyor belt is abnormal based on the similarity, the method further comprises:
and under the condition that the conveyor belt is determined to be abnormal, determining the abnormal reason of the conveyor belt according to the current image and the current reconstructed image.
8. A conveyor belt testing apparatus, comprising:
the first acquisition unit is used for acquiring a historical image of the conveyor belt, wherein the historical image is acquired by the image acquisition equipment;
the processing unit is used for compressing and restoring the historical image to obtain a historical reconstruction image and constructing a reconstruction model according to the historical image and the historical reconstruction image;
the second acquisition unit is used for acquiring a current image of the conveyor belt, and inputting the current image into the reconstruction model for compression processing and reduction processing to obtain a current reconstruction image;
and the detection unit is used for comparing the similarity between the current reconstructed image and the current image and determining whether the conveyor belt is abnormal or not according to the similarity.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored program, wherein the program performs the method of any one of claims 1 to 7.
10. An electronic device, comprising: one or more processors, memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211448372.1A CN115713511A (en) | 2022-11-18 | 2022-11-18 | Detection method and device for conveyor belt and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211448372.1A CN115713511A (en) | 2022-11-18 | 2022-11-18 | Detection method and device for conveyor belt and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115713511A true CN115713511A (en) | 2023-02-24 |
Family
ID=85233750
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211448372.1A Pending CN115713511A (en) | 2022-11-18 | 2022-11-18 | Detection method and device for conveyor belt and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115713511A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116758400A (en) * | 2023-08-15 | 2023-09-15 | 安徽容知日新科技股份有限公司 | Method and device for detecting abnormality of conveyor belt and computer readable storage medium |
-
2022
- 2022-11-18 CN CN202211448372.1A patent/CN115713511A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116758400A (en) * | 2023-08-15 | 2023-09-15 | 安徽容知日新科技股份有限公司 | Method and device for detecting abnormality of conveyor belt and computer readable storage medium |
| CN116758400B (en) * | 2023-08-15 | 2023-10-17 | 安徽容知日新科技股份有限公司 | Method and device for detecting abnormality of conveyor belt and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200364842A1 (en) | Surface defect identification method and apparatus | |
| KR101967089B1 (en) | Convergence Neural Network based complete reference image quality evaluation | |
| CN110555819B (en) | Equipment monitoring method, device and equipment based on infrared and visible light image fusion | |
| US9251582B2 (en) | Methods and systems for enhanced automated visual inspection of a physical asset | |
| CN108921203B (en) | Detection and identification method for pointer type water meter | |
| JP2019035626A (en) | Recognition method of tire image and recognition device of tire image | |
| JP2021190105A (en) | Defect detection method and device | |
| CN111091109B (en) | Method, system and equipment for predicting age and gender based on face image | |
| CN109360120B (en) | Intelligent diagnosis method for working condition of electric submersible pump well based on convolutional neural network | |
| CN109086876B (en) | Method and device for detecting running state of equipment, computer equipment and storage medium | |
| CN111868780B (en) | Learning data generation device and method, model generation system, and program | |
| CN110827505A (en) | Smoke segmentation method based on deep learning | |
| CN112966665A (en) | Pavement disease detection model training method and device and computer equipment | |
| CN115713511A (en) | Detection method and device for conveyor belt and electronic equipment | |
| CN115457451B (en) | Constant temperature and humidity test box monitoring method and device based on Internet of things | |
| CN111144168A (en) | Crop growth cycle identification method, equipment and system | |
| CN107977531B (en) | A kind of ground resistance flexible measurement method based on image procossing and mathematical model | |
| CN111310837A (en) | Vehicle refitting recognition method, device, system, medium and equipment | |
| CN112132867A (en) | Remote sensing image transformation detection method and device | |
| CN115423809B (en) | Image quality evaluation method and device, readable storage medium and electronic equipment | |
| Hepburn et al. | Enforcing perceptual consistency on generative adversarial networks by using the normalised laplacian pyramid distance | |
| KR102511967B1 (en) | Method and system for image-based sea level observation | |
| CN111415326A (en) | Method and system for detecting abnormal state of railway contact net bolt | |
| CN116363075A (en) | Photovoltaic module hot spot detection method and system and electronic equipment | |
| CN116091874A (en) | Image verification method, training method, device, medium, equipment and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |