CN115187588A - Foreign matter detection method, foreign matter detection device, storage medium, and electronic apparatus - Google Patents
Foreign matter detection method, foreign matter detection device, storage medium, and electronic apparatus Download PDFInfo
- Publication number
- CN115187588A CN115187588A CN202211088278.XA CN202211088278A CN115187588A CN 115187588 A CN115187588 A CN 115187588A CN 202211088278 A CN202211088278 A CN 202211088278A CN 115187588 A CN115187588 A CN 115187588A
- Authority
- CN
- China
- Prior art keywords
- foreign matter
- image
- foreground
- color
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 78
- 238000003860 storage Methods 0.000 title claims abstract description 11
- 239000000463 material Substances 0.000 claims abstract description 137
- 230000005540 biological transmission Effects 0.000 claims abstract description 52
- 230000002159 abnormal effect Effects 0.000 claims abstract description 26
- 238000000034 method Methods 0.000 claims abstract description 19
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 230000000903 blocking effect Effects 0.000 claims abstract description 7
- 238000012216 screening Methods 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 13
- 239000002245 particle Substances 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000005192 partition Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 6
- 230000000877 morphologic effect Effects 0.000 claims description 5
- 238000004458 analytical method Methods 0.000 description 5
- 239000003086 colorant Substances 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 3
- PCTMTFRHKVHKIS-BMFZQQSSSA-N (1s,3r,4e,6e,8e,10e,12e,14e,16e,18s,19r,20r,21s,25r,27r,30r,31r,33s,35r,37s,38r)-3-[(2r,3s,4s,5s,6r)-4-amino-3,5-dihydroxy-6-methyloxan-2-yl]oxy-19,25,27,30,31,33,35,37-octahydroxy-18,20,21-trimethyl-23-oxo-22,39-dioxabicyclo[33.3.1]nonatriaconta-4,6,8,10 Chemical compound C1C=C2C[C@@H](OS(O)(=O)=O)CC[C@]2(C)[C@@H]2[C@@H]1[C@@H]1CC[C@H]([C@H](C)CCCC(C)C)[C@@]1(C)CC2.O[C@H]1[C@@H](N)[C@H](O)[C@@H](C)O[C@H]1O[C@H]1/C=C/C=C/C=C/C=C/C=C/C=C/C=C/[C@H](C)[C@@H](O)[C@@H](C)[C@H](C)OC(=O)C[C@H](O)C[C@H](O)CC[C@@H](O)[C@H](O)C[C@H](O)C[C@](O)(C[C@H](O)[C@H]2C(O)=O)O[C@H]2C1 PCTMTFRHKVHKIS-BMFZQQSSSA-N 0.000 description 2
- XEEYBQQBJWHFJM-UHFFFAOYSA-N Iron Chemical compound [Fe] XEEYBQQBJWHFJM-UHFFFAOYSA-N 0.000 description 2
- 229910000831 Steel Inorganic materials 0.000 description 2
- 230000002547 anomalous effect Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000010959 steel Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000032258 transport Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000010339 dilation Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 238000005530 etching Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 229910052742 iron Inorganic materials 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 229910052755 nonmetal Inorganic materials 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of foreign matter detection, and discloses a foreign matter detection method, a foreign matter detection device, a storage medium and electronic equipment, wherein the method comprises the following steps: acquiring a video stream of a transmission band area, and preprocessing a plurality of frames of images of the transmission band area in the video stream to obtain a plurality of frames of material foreground images; local features of the material foreground images of the frames are extracted in a blocking mode, abnormal blocks are screened, and foreign matter candidate areas of the material foreground images of the frames are obtained; and obtaining a foreign matter transportation track based on the transportation direction of the conveyor belt and the foreign matter candidate area of each frame of material foreground image, and determining whether the foreign matter is detected according to the foreign matter transportation track. The method has the advantages of low false detection rate, high foreign matter recall rate and strong robustness of the detection algorithm.
Description
Technical Field
The present invention relates to the field of foreign object detection technologies, and in particular, to a method and an apparatus for detecting a foreign object, a storage medium, and an electronic device.
Background
In the industrial field, in the process of material transportation of a belt conveyor, unconventional objects except materials, such as metal or nonmetal objects of steel plates, iron sheets, steel bars and the like, can appear. The foreign matters are doped in the materials, so that the subsequent production process can be influenced, the transportation problems of belt conveyor blockage, belt tearing, feed opening blockage and the like can be caused, and the production safety and the production efficiency are greatly influenced.
The related belt foreign matter detection means mainly comprises a method based on sensor numerical analysis, deep learning and image processing. The method based on sensor numerical analysis cannot detect foreign matters of similar types to materials, cannot perform foreign matter visualization, and is high in deployment difficulty and high in cost. The foreign matter detection algorithm based on deep learning depends on a large number of real foreign matter images of the belt, and is high in actual acquisition difficulty and long in period. The belt foreign matter detection method based on single-frame image processing can only detect specific foreign matters and is easy to generate false detection.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art. Therefore, an object of the present invention is to provide a foreign object detection method, which has the advantages of low false detection rate, high foreign object recall rate, and strong robustness of the detection algorithm.
A second object of the present invention is to provide a foreign object detection apparatus.
A third object of the invention is to propose a computer-readable storage medium.
A fourth object of the invention is to propose an electronic device.
In order to achieve the above object, a foreign object detection method according to an embodiment of a first aspect of the present invention includes: acquiring a video stream of a transmission band area, and preprocessing a plurality of frames of transmission band area images in the video stream to obtain a plurality of frames of material foreground images; extracting local features of the material foreground images of each frame in a blocking mode, and screening abnormal blocks to obtain foreign matter candidate areas of the material foreground images of each frame; and obtaining a foreign matter transportation track based on the transportation direction of the conveyor belt and the foreign matter candidate area of the material foreground image of each frame, and determining whether the foreign matter is detected according to the foreign matter transportation track.
According to the foreign matter detection method provided by the embodiment of the invention, the foreign matter detection is carried out based on the material foreground image, and compared with the existing detection method based on the whole image, the background interference is effectively reduced, and the false detection rate is reduced; foreign matter characteristics of each image block are analyzed through difference characteristics, so that the foreign matters can be identified according to difference information such as colors, textures, edges and shapes between the foreign matters and materials, and the foreign matter detection recall rate is effectively improved; and on the basis of single-frame foreign body detection, a foreign body time sequence trajectory analysis method is introduced, so that the false detection of foreign bodies can be effectively reduced, and the algorithm robustness is improved.
In addition, the foreign object detection method proposed according to the above embodiment of the present invention may further have the following additional technical features:
according to an embodiment of the present invention, preprocessing a plurality of frames of images of a transmission band region in the video stream to obtain a plurality of frames of foreground images of a material includes: acquiring a plurality of frames of the transmission band area images in the video stream at preset time intervals; marking the transmission band area in the transmission band area image of each frame to obtain a target area image; aiming at a current frame target area image, carrying out gray difference processing on the current frame target area image and a next frame target area image, and carrying out binarization by using a preset threshold value to obtain a motion difference image; filling the motion area in the motion difference image by using morphological closed operation, and calculating the connected domain area of the filled motion area to obtain a material mask; and overlapping the current frame target area image and the material mask with operation to obtain a current frame material foreground image.
According to an embodiment of the invention, extracting local features of the current frame material foreground image in blocks, and performing abnormal block screening, comprises: dividing the current frame material foreground image by using pre-divided blocks with preset sizes to obtain M multiplied by N image blocks with the preset sizes; calculating the color distance characteristic of each image block in an RGB color space and the color variance characteristic of each image block in an HSV color space; and determining an abnormal block in the foreground image of the current frame material according to the color distance characteristic and the color variance characteristic of each image block.
According to the inventionOne embodiment determines the size of the pre-partitions based on the material granularity
Wherein
indicates the width of the pre-partition,
indicating the high of the pre-partition,
particle size of larger than material
Wherein, in the process,
a length representing the estimated particle size of the material,
width representing estimated material particle size;
the image block
The expression of (a) is:
wherein,
m =1,2,3,. Multidot., M represents the image block horizontal index, N =1,2,3,. Multidot., N represents the image block longitudinal index, x, y represent the horizontal and vertical coordinates of the vertex at the upper left corner of the image block, respectively, and w, h represent the width and height of the image block, respectively.
According to one embodiment of the present invention, the expression of the color distance characteristic is:
wherein,
the color distance characteristic is represented by a color distance,
in the form of an inverse cosine function,
representing the image block
The color mean vector in the RGB color space,
representing the color mean vector of the material foreground image in an RGB color space, wherein the value range of the color distance characteristic is
；
The expression of the color variance characteristic is as follows:
wherein,
a color variance characteristic is represented by a color variance of the color image,
for the image block
In HSV colorThe color variance of the chrominance component H component in space,
and obtaining the integral color variance of the chromaticity component H component of the material foreground image in the HSV color space.
According to an embodiment of the present invention, the determining an abnormal block in the foreground image of the current frame material according to the color distance characteristic and the color variance characteristic of each image block includes:
judging each image block in the current frame material foreground image
Said color distance characteristic of
Whether greater than a color distance threshold
The color variance characteristic
Whether greater than a color variance threshold
；
When in use
≥
And is made of
≥
Then, the image block is determined
Is the exception block and is noted
Wherein K =1,2, 3., K denotes an index of the foreign object candidate region, and t denotes a current frame material foreground map.
According to one embodiment of the invention, the foreign matter transportation track is obtained based on the transportation direction of the conveyor belt and the foreign matter candidate area of each frame of material foreground image, and the foreign matter transportation track comprises the following steps: obtaining a foreign matter candidate area in the current frame material foreground image
Position information of each of the abnormal blocks; determining a preset position according to the transportation direction and the position information, and judging whether the preset position of the next frame of material foreground image is a foreign matter candidate area
Wherein
representing a foreign matter candidate area of the next frame material foreground image, and t +1 representing the next frame material foreground image; if so, recording the track of the foreign matter candidate area, and continuing to perform track association of the foreign matter candidate area of the subsequent frame; if not, the foreign matter candidate area is cleared, and the track association of the foreign matter candidate area is not carried out.
In order to achieve the above object, a second aspect of the present invention provides a foreign object detection apparatus, including: the acquisition module is used for acquiring a video stream of a transmission belt area, and preprocessing a plurality of frames of images of the transmission belt area in the video stream to obtain a plurality of frames of material foreground images; the screening module is used for extracting local characteristics of the material foreground images of each frame in a blocking mode and screening abnormal blocks to obtain foreign matter candidate areas of the material foreground images of each frame; and the detection module is used for obtaining a foreign matter transportation track based on the transportation direction of the conveyor belt and the foreign matter candidate area of each frame of material foreground image, and determining whether the foreign matter is detected according to the foreign matter transportation track.
To achieve the above object, a third aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, implementing the foreign object detection method according to the first aspect of the present invention.
In order to achieve the above object, a fourth aspect of the present invention provides an electronic device, including a memory and a processor, where the memory stores a computer program, and the computer program, when executed by the processor, implements the foreign object detection method as set forth in the first aspect of the present invention.
Drawings
FIG. 1 is a flow chart of a foreign object detection method according to an embodiment of the present invention;
FIG. 2 is a flow diagram of pre-processing a transmission band region image according to one embodiment of the invention;
FIG. 3 is a schematic diagram of obtaining a map of a target area in accordance with one embodiment of the present invention;
FIG. 4 is a schematic flow chart of obtaining a material foreground map according to an embodiment of the present invention;
FIG. 5 is a flow chart of a foreign object detection method according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating a candidate region for a foreign object in a material foreground diagram according to an embodiment of the present invention;
fig. 7 is a schematic view of a foreign matter detection apparatus according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
The foreign object detection method, apparatus, storage medium, and electronic device according to the embodiments of the present invention will be described in detail with reference to fig. 1 to 7 and specific embodiments of the present invention.
Fig. 1 is a flowchart of a foreign object detection method according to an embodiment of the present invention. As shown in fig. 1, the foreign object detection method may include:
s1, acquiring a video stream of a transmission band area, and preprocessing a plurality of frames of images of the transmission band area in the video stream to obtain a plurality of frames of material foreground images;
according to the embodiment of the invention, when the foreign matters in the materials on the transmission belt are detected, the video stream of the transmission belt area is obtained, the multi-frame transmission belt area image is obtained from the video stream, and the foreign matter detection is carried out on the single-frame transmission belt area image. In order to reduce background interference, the single-frame material foreground image only containing the material area is obtained by preprocessing the single-frame transmission belt area image.
For better effect, the camera equipment for collecting the video stream in the area of the transmission belt is installed over the transmission belt, so that the camera equipment can clearly shoot the transportation condition of materials on the transmission belt, the quality of the obtained images in the area of the transmission belt is improved, and the influence of the image quality on the detection of foreign matters is reduced.
In an embodiment of the present invention, as shown in fig. 2, preprocessing multiple frames of images of a transmission band region in a video stream to obtain a multiple frames of foreground images of a material may include:
and S11, acquiring multi-frame transmission band area images in the video stream at preset time intervals.
In order to avoid missing partial materials on a transmission belt when a multi-frame transmission belt area image is acquired from a video stream at a preset time interval i, the preset time interval i can be set according to the running speed of the transmission belt. And acquiring multi-frame transmission band area images from the video stream according to the set time interval i, preprocessing the single-frame transmission band area images, and detecting foreign matters in the preprocessed images.
In the embodiment of the present invention, since the running speed of the transmission belt is relatively uniform and does not change suddenly, i can take 10s, that is, every 10s, one transmission belt region image is obtained from the video stream.
And S12, labeling the transmission band region in the transmission band region image of each frame to obtain a target region image.
Specifically, as shown in fig. 3, four vertices of the transmission band in the single frame transmission band image are marked clockwise, and are denoted as a, B, C, and D in order. The quadrilateral ABCD contains a transmission band region. And intercepting an external matrix area marked with a quadrilateral ABCD from the image of the transmission belt area, and recording the external matrix area as a matrix A 'B' C 'D'. Pixels other than the transfer band in the matrix a 'B' C 'D' are 0-filled, and a partial image, i.e., a target area map, with respect to only the transfer band is acquired.
And S13, carrying out gray difference processing on the current frame target area image and the next frame target area image, and carrying out binarization by using a preset threshold value to obtain a motion difference image.
Specifically, for the current frame target area map, i.e. the tth frame target area map
And its next frame target area map, i.e. t +1 th frame target area map
Performing gray level processing on the target area image of the transmission belt area image acquired at the t + i second, calculating the gray level value of each pixel of the current frame target area image and the next frame target area image, and subtracting the gray level value of the corresponding pixel point to obtain the gray level difference value of each pixel point, comparing the gray level difference value with a preset threshold value, wherein the gray level difference value is within the range of the preset threshold value, and the gray level value of the difference image at the pixel point is regarded as 0; if the gray difference is outside the preset threshold range, the gray value of the difference image at the pixel point is 255, and a motion difference image is obtained, as shown in fig. 4.
In the embodiment of the present invention, when i =10, the difference processing is performed every 10 s. Wherein, i is 10, and the algorithm speed can be increased.
And S14, filling the motion region in the motion difference image by using morphological closed operation, and calculating the area of the filled motion region to obtain a material mask.
Specifically, as shown in fig. 4, in the morphological processing algorithm, the operation of performing the dilation process first and then performing the erosion process may be referred to as a closed operation, and the closed operation may be used for filling fine holes in the object, connecting neighboring objects, and smoothing the boundary thereof. And performing morphological closed operation processing on the motion difference image after the binarization processing to fill a motion area in the motion difference image, and connecting images formed by high-numerical-value pixel points with relatively close positions together to obtain a plurality of connected domains. And respectively counting the number of pixel points contained in each connected domain in the motion difference image after filling processing, wherein the connected domain with the largest number of the pixel points is the material mask.
It should be noted that the embodiment of the present invention does not limit the number of times of the expansion process and the etching process in the closed-loop operation, and aims to achieve the best processing effect.
And S15, overlapping the current frame target area image and the material mask with operation to obtain a current frame material foreground image.
Target area map of current frame, i.e. target area map of t-th frame
And overlapping with the obtained material mask to obtain a current frame material foreground image
See fig. 4. And carrying out the processing on the images of the transmission band regions of each frame to obtain a material foreground image of the images of the transmission band regions of each frame, and carrying out foreign matter detection on the material foreground image of the images of the transmission band regions of each frame.
And S2, extracting local features of the material foreground images of the frames in a blocking mode, and screening abnormal blocks to obtain foreign matter candidate areas of the material foreground images of the frames.
Specifically, local features in the material foreground image are extracted in a blocking mode by means of preset blocks, abnormal block screening is conducted on the local features of each image block, and abnormal blocks of the material foreground image are detected, wherein each abnormal block in the material foreground image is a foreign matter candidate area of the material foreground image.
In an embodiment of the present invention, as shown in fig. 5, extracting local features of a foreground image of a current frame material in each frame in a block manner, and performing abnormal block screening may include:
segmenting a foreground image of the current frame material by utilizing pre-segmentation blocks with preset sizes to obtain M multiplied by N image blocks with preset sizes;
calculating the color distance characteristic of each image block in an RGB color space and the color variance characteristic of each image block in an HSV color space;
and determining an abnormal block in the foreground image of the current frame material according to the color distance characteristic and the color variance characteristic of each image block.
Referring to fig. 6, a foreground image of a current frame material is segmented by using pre-partitioned blocks of a preset size to obtain M × N image blocks of the preset size, where the size of the pre-partitioned blocks may be determined according to the material granularity. Because in the industrial production process, the conveying belt usually transports the same materials, the colors of the materials are usually close, and the main difference is the granularity and the shade of the materials. The foreign matters usually include misplaced equipment, aged and fallen mechanical parts, belt fences and other objects, and have larger differences from the colors and forms of the materials. Therefore, the color distance characteristic in the RGB color space and the color variance characteristic in the HSV color space of each image block can be calculated. And detecting an abnormal block in the foreground image of the current frame material according to the color distance characteristic and the color variance characteristic of each image block.
In an embodiment of the invention, the size of the pre-portioned blocks is determined based on the material granularity
Wherein, in the process,
indicates the width of the pre-partition,
indicating the high of the pre-partition,
greater than estimated material particle size
Wherein, in the process,
indicates the length of the estimated particle size of the material,
a width representing an estimated material particle size;
image block
The expression is given as:
wherein,
m =1,2, 3., M denotes an image block horizontal index, N =1,2, 3., N denotes an image block vertical index, x, y denote the horizontal and vertical coordinates of the vertex at the upper left corner of the image block, respectively, and w, h denote the width and height of the image block, respectively.
It should be noted that m and n are positive integers, and the value range is determined according to the horizontal number and the vertical number of the image blocks in the material foreground image.
In an embodiment of the invention, the pre-tiling is square, wherein,
，
representing the side length of a preset pre-partition block.
In an embodiment of the present invention, the expression of the color distance characteristic is:
wherein,
the characteristic of the distance of the color is represented,
in the form of an inverse cosine function,
representing image blocks
A color mean vector in the RGB color space,
the color mean value vector of the material foreground image in the RGB color space is represented, and the value range of the color distance characteristic is
；
The expression of the color variance feature is:
wherein,
the color variance characteristic is represented by a color variance characteristic,
for image blocks
The color variance of the chrominance component H component in the HSV color space,
is a material prospect diagram at HThe overall color variance of the chroma component H component in the SV color space.
Specifically, the color distance characteristic of each image block in the RGB color space is calculated by using the expression of the color distance characteristic and the expression of the color variance characteristic
And color variance characteristics in HSV color space
. According to the color distance threshold
And color variance threshold
Color distance characteristics for each image block
And color variance characteristics
And (5) judging to determine an abnormal block in the current frame material foreground image.
In an embodiment of the present invention, as shown in fig. 5, determining an abnormal block in the foreground image of the current frame material according to the color distance characteristic and the color variance characteristic of each image block may include:
judging each image block in the foreground image of the current frame material
Color distance characteristic of
Whether greater than a color distance threshold
Color variance characteristics
Whether greater than a color variance threshold
；
When in use
≥
And is and
≥
determining image blocks
Is an abnormal block and is recorded as
Wherein K =1,2, 3., K denotes an index of the foreign object candidate region, and t denotes a current frame material foreground map.
Specifically, each image block
Color distance characteristic of
Distance from color threshold
Comparing, color variance characteristics
And color variance threshold
A comparison is made. Judging image blocks
Color distance characteristic of
Whether greater than a color distance threshold
Color variance characteristics
Whether greater than a color variance threshold
. Wherein the color distance characteristic
Greater than a color distance threshold
And color variance characteristics
Is also greater than the color variance threshold
Image block of
Is an exception block.
In addition, the image blocks are based on
M and n of (c), the row and column where the image block that is the abnormal block is located, i.e., the location of the abnormal block, can be determined.
And S3, obtaining a foreign matter transportation track based on the transportation direction of the conveyor belt and the foreign matter candidate area of each frame of material foreground image, and determining whether the foreign matter is detected according to the foreign matter transportation track.
In an embodiment of the present invention, as shown in fig. 5, obtaining a foreign object transportation track based on a transportation direction of a conveyor and a foreign object candidate region of a foreground map of each frame of material may include:
obtaining a foreign matter candidate area in a current frame material foreground image
Position information of each anomalous block of (1);
determining a preset position according to the transportation direction and the position information, and judging whether the preset position of the next frame of material foreground image is a foreign matter candidate area
Wherein
representing a foreign matter candidate area of the next frame material foreground image, and t +1 representing the next frame material foreground image;
if so, recording the foreign matter candidate area track, and continuing to perform track association of the foreign matter candidate area of the subsequent frame;
if not, the foreign matter candidate area is removed, and the track association of the foreign matter candidate area is not carried out.
Specifically, a foreign matter candidate area in a current frame material foreground image is obtained
Position information of each anomalous block of (2)
Determining the preset position of the next frame of material foreground map corresponding to the appearance of the foreign object block according to the conveying direction of the conveying belt, the conveying speed, the preset time interval and the current frame of material foreground map, namely the position information of the foreign object block in the t-th frame of material foreground map
In order to prevent the occurrence of other problems such as jamming of the conveyor belt, the predetermined position may further include
And/or
To prevent missed detection. After the preset position is determined, judging whether the preset position of the next frame of material foreground image is a foreign matter candidate area or not
. If the preset position in the next frame of material foreground image is a foreign matter candidate area
Then the foreign object candidate area track is recorded and track association of the foreign object candidate area of the subsequent frame is continued. If the preset position of the next frame of material foreground image is not the foreign matter candidate area
And removing the foreign matter candidate area, and not performing track correlation on the foreign matter candidate area any more so as to improve the accuracy of foreign matter detection and reduce the false detection rate.
Wherein the candidate foreign body track obtained by correlation is recorded as
Wherein id represents the index corresponding to the track.
The following conditions need to be satisfied: the starting position of the track is the starting position of material transportation in the image; the track end position is a material transportation end position in the image.
According to the foreign matter detection method, foreign matter detection is carried out based on the material foreground image, compared with the existing detection method based on the whole image, background interference is effectively reduced, and the false detection rate is reduced; the foreign matter characteristics of each image block are analyzed through the difference characteristics, so that the foreign matters can be identified according to the difference information such as colors, textures, edges and shapes between the foreign matters and materials, and the foreign matter detection recall rate is effectively improved; and on the basis of single-frame foreign matter detection, a foreign matter time sequence trajectory analysis method is introduced, so that the false detection of foreign matters can be effectively reduced, and the algorithm robustness is improved.
The invention also provides a foreign matter detection device.
Fig. 7 is a schematic view of a foreign object detection apparatus according to an embodiment of the present invention. As shown in fig. 7, the foreign object detection apparatus 100 may include an acquisition module 10, a screening module 20, and a detection module 30.
The acquiring module 10 is configured to acquire a video stream of a transmission band region, and pre-process a plurality of frames of images of the transmission band region in the video stream to obtain a plurality of frames of material foreground images; the screening module 20 is configured to extract local features of each frame of material foreground image in a block manner, and perform abnormal block screening to obtain a foreign matter candidate region of each frame of material foreground image; the detection module 30 is configured to obtain a foreign object transportation track based on the transportation direction of the conveyor belt and the foreign object candidate region of each frame of the material foreground map, and determine whether to detect a foreign object according to the foreign object transportation track.
It should be noted that, for other specific embodiments of the foreign object detection apparatus according to the embodiment of the present invention, reference may be made to specific embodiments of the foreign object detection method according to the above-described embodiment of the present invention.
Compared with the existing detection method based on the whole image, the foreign matter detection device based on the material foreground image effectively reduces background interference and lowers false detection rate; foreign matter characteristics of each image block are analyzed through difference characteristics, so that the foreign matters can be identified according to difference information such as colors, textures, edges and shapes between the foreign matters and materials, and the foreign matter detection recall rate is effectively improved; and on the basis of single-frame foreign body detection, a foreign body time sequence trajectory analysis method is introduced, so that the false detection of foreign bodies can be effectively reduced, and the algorithm robustness is improved.
The invention also provides a computer readable storage medium.
In this embodiment, a computer-readable storage medium has stored thereon a computer program, which corresponds to the above-mentioned foreign object detection method, and which, when executed by a processor, implements the foreign object detection method as described above.
The invention also provides the electronic equipment.
In this embodiment, the electronic device comprises a processor, a memory and a computer program stored on the memory, which when executed by the processor implements the foreign object detection method as described above.
The storage medium and the electronic device of the embodiment of the invention utilize the foreign matter detection method to realize the foreign matter detection of the transmission belt.
It should be noted that the logic and/or steps represented in the flowcharts or otherwise described herein, such as an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Further, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
In the description of the present invention, it is to be understood that the terms "central," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," "circumferential," and the like are used in the orientations and positional relationships indicated in the drawings for convenience in describing the invention and to simplify the description, and are not intended to indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and are therefore not to be considered limiting of the invention.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (10)
1. A foreign object detection method, characterized in that the method comprises:
acquiring a video stream of a transmission band area, and preprocessing a plurality of frames of images of the transmission band area in the video stream to obtain a plurality of frames of material foreground images;
extracting local features of the material foreground images of each frame in a blocking manner, and screening abnormal blocks to obtain foreign matter candidate areas of the material foreground images of each frame;
and obtaining a foreign matter transportation track based on the transportation direction of the conveyor belt and the foreign matter candidate area of each frame of the material foreground image, and determining whether the foreign matter is detected according to the foreign matter transportation track.
2. The foreign matter detection method according to claim 1, wherein preprocessing a plurality of frames of images of a transmission band region in the video stream to obtain a plurality of frames of foreground images of the material comprises:
acquiring a plurality of frames of the transmission band area images in the video stream at preset time intervals;
marking the transmission band area in the transmission band area image of each frame to obtain a target area image;
aiming at a current frame target area image, carrying out gray level difference processing on the current frame target area image and a next frame target area image, and carrying out binarization by using a preset threshold value to obtain a motion difference image;
filling the motion area in the motion difference image by using morphological closed operation, and calculating the connected domain area of the filled motion area to obtain a material mask;
and overlapping the current frame target area image and the material mask with operation to obtain a current frame material foreground image.
3. The foreign matter detection method according to claim 2, wherein extracting local features of the material foreground map of each frame in blocks and performing abnormal block screening includes:
dividing the current frame material foreground image by using pre-divided blocks with preset sizes to obtain M multiplied by N image blocks with the preset sizes;
calculating the color distance characteristic of each image block in an RGB color space and the color variance characteristic of each image block in an HSV color space;
and determining an abnormal block in the foreground image of the current frame material according to the color distance characteristic and the color variance characteristic of each image block.
4. The foreign object detection method according to claim 3, wherein the size of the pre-partition is determined based on the particle size of the material
Wherein, in the process,
indicates the width of the pre-partition,
indicating the high of the pre-partition,
particle size of larger than material
Wherein
indicates the length of the estimated particle size of the material,
width representing estimated material particle size;
the figure isPhoto block
The expression of (a) is:
wherein,
m =1,2, 3., M denotes the image block horizontal index, N =1,2, 3., N denotes the image block vertical index, x, y denote the horizontal and vertical coordinates of the vertex at the upper left corner of the image block, respectively, and w, h denote the width and height of the image block, respectively.
5. The foreign object detection method according to claim 4, wherein the expression of the color distance characteristic is:
wherein,
the color distance characteristic is represented by a color distance,
is a function of the inverse cosine of the,
representing the image block
The color mean vector in the RGB color space,
representing the color mean vector of the material foreground image in an RGB color space, wherein the value range of the color distance characteristic is
；
The expression of the color variance characteristic is as follows:
wherein,
a color variance characteristic is represented by a color variance of the color image,
for the image block
The color variance of the chrominance component H component in the HSV color space,
and obtaining the integral color variance of the chromaticity component H component of the material foreground image in the HSV color space.
6. The method according to claim 5, wherein the determining the abnormal block in the foreground map of the current frame material according to the color distance feature and the color variance feature of each image block comprises:
judging each image block in the current frame material foreground image
Said color distance characteristic of
Whether greater than a color distance threshold
The color variance characteristic
Whether greater than a color variance threshold
；
When the temperature is higher than the set temperature
≥
And is and
≥
then, the image block is determined
Is the exception block and is noted
Wherein K =1,2, 3., K denotes an index of the foreign object candidate region, and t denotes a foreground map of the current frame material.
7. The foreign matter detection method according to claim 6, wherein obtaining the foreign matter transport trajectory based on the transport direction of the transport belt and the foreign matter candidate region of the material foreground map of each frame comprises:
obtaining a foreign matter candidate area in the current frame material foreground image
Position information of each of the abnormal blocks;
determining a preset position according to the transportation direction and the position information, and judging whether the preset position of the next frame of material foreground image is a foreign matter candidate area
Wherein
representing a foreign matter candidate area of the next frame material foreground image, and t +1 representing the next frame material foreground image;
if so, recording the track of the foreign matter candidate area, and continuing to perform track association of the foreign matter candidate area of the subsequent frame;
if not, the foreign matter candidate area is cleared, and the track association of the foreign matter candidate area is not carried out.
8. A foreign object detection apparatus, characterized in that the apparatus comprises:
the acquisition module is used for acquiring a video stream of a transmission belt area, and preprocessing a plurality of frames of images of the transmission belt area in the video stream to obtain a plurality of frames of material foreground images;
the screening module is used for extracting local characteristics of the material foreground images of the frames in a blocking mode and screening abnormal blocks to obtain foreign matter candidate areas of the material foreground images of the frames;
and the detection module is used for obtaining a foreign matter transportation track based on the transportation direction of the conveyor belt and the foreign matter candidate area of each frame of material foreground image, and determining whether the foreign matter is detected according to the foreign matter transportation track.
9. A computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, carries out a foreign object detection method according to any one of claims 1 to 7.
10. An electronic device comprising a memory and a processor, the memory having a computer program stored thereon, wherein the computer program, when executed by the processor, implements the foreign object detection method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211088278.XA CN115187588B (en) | 2022-09-07 | 2022-09-07 | Foreign matter detection method, foreign matter detection device, storage medium, and electronic apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211088278.XA CN115187588B (en) | 2022-09-07 | 2022-09-07 | Foreign matter detection method, foreign matter detection device, storage medium, and electronic apparatus |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115187588A true CN115187588A (en) | 2022-10-14 |
| CN115187588B CN115187588B (en) | 2022-12-09 |
Family
ID=83523803
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211088278.XA Active CN115187588B (en) | 2022-09-07 | 2022-09-07 | Foreign matter detection method, foreign matter detection device, storage medium, and electronic apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115187588B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115947066A (en) * | 2023-03-14 | 2023-04-11 | 合肥金星智控科技股份有限公司 | Belt tearing detection method, device and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104700430A (en) * | 2014-10-05 | 2015-06-10 | 安徽工程大学 | Method for detecting movement of airborne displays |
| CN106778595A (en) * | 2016-12-12 | 2017-05-31 | 河北工业大学 | The detection method of abnormal behaviour in crowd based on gauss hybrid models |
| CN110555868A (en) * | 2019-05-31 | 2019-12-10 | 南京航空航天大学 | method for detecting small moving target under complex ground background |
| CN113486714A (en) * | 2021-06-03 | 2021-10-08 | 荣耀终端有限公司 | Image processing method and electronic equipment |
-
2022
- 2022-09-07 CN CN202211088278.XA patent/CN115187588B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104700430A (en) * | 2014-10-05 | 2015-06-10 | 安徽工程大学 | Method for detecting movement of airborne displays |
| CN106778595A (en) * | 2016-12-12 | 2017-05-31 | 河北工业大学 | The detection method of abnormal behaviour in crowd based on gauss hybrid models |
| CN110555868A (en) * | 2019-05-31 | 2019-12-10 | 南京航空航天大学 | method for detecting small moving target under complex ground background |
| CN113486714A (en) * | 2021-06-03 | 2021-10-08 | 荣耀终端有限公司 | Image processing method and electronic equipment |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115947066A (en) * | 2023-03-14 | 2023-04-11 | 合肥金星智控科技股份有限公司 | Belt tearing detection method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115187588B (en) | 2022-12-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9946941B2 (en) | Lane detection | |
| TWI409718B (en) | Method of locating license plate of moving vehicle | |
| CN114937055A (en) | Image self-adaptive segmentation method and system based on artificial intelligence | |
| CN112184744B (en) | Display screen edge defect detection method and device | |
| US7639878B2 (en) | Shadow detection in images | |
| CN109359594A (en) | A kind of mine belt method for detecting abnormality and device based on video processing | |
| CN113221861B (en) | Multi-lane line detection method, device and detection equipment | |
| JP4156084B2 (en) | Moving object tracking device | |
| US10497107B1 (en) | Method, computer program product and computer readable medium for generating a mask for a camera stream | |
| CN115187588B (en) | Foreign matter detection method, foreign matter detection device, storage medium, and electronic apparatus | |
| US8000535B2 (en) | Methods and systems for refining text segmentation results | |
| CN111881837B (en) | Shadow extraction-based video SAR moving target detection method | |
| CN112541486A (en) | Infrared weak and small target detection algorithm based on improved Pixel segmentation | |
| CN114998317B (en) | Lens occlusion detection method and device, camera device and storage medium | |
| CN107610132B (en) | Method for removing stains from ancient book document image | |
| JP2021052238A (en) | Deposit detection device and deposit detection method | |
| Vargas et al. | A shadow removal algorithm for vehicle detection based on reflectance ratio and edge density | |
| CN114037730A (en) | Traffic flow statistical method, device, equipment and storage medium | |
| CN106951831B (en) | Pedestrian detection tracking method based on depth camera | |
| CN105118072A (en) | Method and device for tracking multiple moving targets | |
| US20140285662A1 (en) | Image processing apparatus, and method | |
| Manipriya et al. | Performance analysis of spatial color information for object detection using background subtraction | |
| CN111242051A (en) | Vehicle identification optimization method and device and storage medium | |
| JP4253265B2 (en) | Shadow detection apparatus, shadow detection method and shadow detection program, image processing apparatus using shadow detection apparatus, image processing method using shadow detection method, and image processing program using shadow detection program | |
| CN115100620B (en) | Lane line fitting method based on road color and driving direction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |