CN116883439A - Method and device for detecting slab contour with water stain on surface - Google Patents

Method and device for detecting slab contour with water stain on surface Download PDF

Info

Publication number
CN116883439A
CN116883439A CN202310750776.4A CN202310750776A CN116883439A CN 116883439 A CN116883439 A CN 116883439A CN 202310750776 A CN202310750776 A CN 202310750776A CN 116883439 A CN116883439 A CN 116883439A
Authority
CN
China
Prior art keywords
image
slab
detected
contour
noise
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310750776.4A
Other languages
Chinese (zh)
Inventor
徐冬
闫汇卿
何婉章
何海楠
赵剑威
王晓晨
杨荃
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology Beijing USTB
Original Assignee
University of Science and Technology Beijing USTB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology Beijing USTB filed Critical University of Science and Technology Beijing USTB
Priority to CN202310750776.4A priority Critical patent/CN116883439A/en
Publication of CN116883439A publication Critical patent/CN116883439A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/10Image enhancement or restoration by non-spatial domain filtering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20048Transform domain processing
    • G06T2207/20064Wavelet transform [DWT]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Abstract

The invention discloses a plate blank contour detection method and device with water stains on the surface, wherein the method comprises the following steps: step S1: acquiring an intermediate blank image, constructing a sample data set and marking; step S2: constructing a convolutional neural network model, and training by using the marked data set to obtain an optimal feature extraction model; step S3: preprocessing a slab image to be detected by wavelet transformation, and extracting the characteristics of the preprocessed slab image to be detected by using the trained optimal characteristic extraction model to obtain a characteristic diagram of the slab image to be detected; step S4: and carrying out post-processing on the feature map of the slab image to be detected, completing the functions of removing noise and smoothing edges, and completing the contour detection of the slab image to be detected by the methods of extracting the edges and searching the maximum contour. The method can realize real-time detection of the contour of the intermediate blank, especially can improve the precision of contour detection aiming at the blank image with water stains, and has stronger robustness.

Description

Method and device for detecting slab contour with water stain on surface
Technical Field
The invention relates to the technical field of detection, in particular to a method and a device for detecting the outline of a plate blank with water stains on the surface.
Background
The side bending of the plate blank generated in the rough rolling stage has great influence on production, and excessive edge cutting loss and short length are caused if the plate blank is light, so that the yield and the sizing rate are reduced, and visual defects such as edge wrinkles, edge cracks, holes, scratches and the like are caused, the plate shape and the quality of a finished product are influenced, and the rolling stability in the subsequent finish rolling process is influenced; when the steel is heavy, quick stop and frame scraping occur, steel stacking accidents at the finishing mill group are caused, steel scraps are generated, equipment such as a pushing bed is damaged, and a large amount of production time is delayed. At present, the rough rolling slab contour is detected by a traditional image processing method in the industrial field, the method has low robustness, and the condition of detection errors can occur under complex and changeable working conditions.
Disclosure of Invention
The embodiment of the invention provides a method and a device for detecting the outline of a plate blank with water stains on the surface. The technical scheme is as follows:
in one aspect, a method for detecting a contour of a slab with water stains on a surface is provided, and the method is implemented by electronic equipment, and comprises the following steps:
step S1: acquiring an intermediate blank image, constructing a sample data set and marking;
step S2: constructing a convolutional neural network model, and training by using the marked data set to obtain an optimal feature extraction model;
step S3: preprocessing a slab image to be detected by wavelet transformation, and extracting the characteristics of the preprocessed slab image to be detected by using the trained optimal characteristic extraction model to obtain a characteristic diagram of the slab image to be detected;
step S4: and carrying out post-processing on the feature map of the slab image to be detected, completing the functions of removing noise and smoothing edges, and completing the contour detection of the slab image to be detected by the methods of extracting the edges and searching the maximum contour.
Optionally, the S1 specifically includes:
s1-1, shooting and obtaining an RGB image of an intermediate blank with water stains on the surface by using an industrial CCD camera;
s1-2, intercepting an effective area of an acquired slab image to obtain an effective area image;
s1-3, labeling the effective area image, and selecting label values with appropriate contour labeling to form a sample data set;
s1-4, dividing the sample data set into a training set and a testing set proportionally.
Optionally, the optimal feature extraction model in the step S2 is formed by combining seven edge detection modules with seven up-sampling modules, and as the number of network layers is deepened, deeper feature information in the image is extracted, weak slab contour edge features of the part covered by the water stain are extracted, and finally the image is output by fusing seven sides of the convolutional neural network, so that the obvious slab edge features are obtained, and meanwhile, weak edges influenced by the water stain are also extracted;
the edge detection modules are fully connected through a convolution kernel of 1 multiplied by 1, the side edge output result of the edge detection modules is used as the input of the up-sampling module, and finally the output feature graphs of the seven up-sampling modules are fused to obtain the output result of the optimal feature extraction model;
the first edge detection module consists of 1 convolution layer, the second edge detection module consists of 1 convolution layer and 1 pooling layer, the third edge detection module consists of 2 convolution layers and 1 pooling layer, the fourth edge detection module consists of 3 convolution layers and 1 pooling layer, the fifth edge detection module consists of 3 convolution layers, the sixth edge detection module consists of 3 convolution layers, the seventh edge detection module consists of 2 convolution layers, and the convolution kernel size of all edge detection modules is 3×3;
the convolution layer calculation formula is as follows:
wherein N represents the size of the image output after convolution, W represents the size of the input image, F represents the size of the convolution kernel, P represents the number of pixels filled, and S represents the step size;
the calculation formula of the pooling layer is as follows:
wherein N represents the size of the image output after pooling, W represents the size of the input image, F represents the size of the pooling kernel, P represents the number of filled pixels, S represents the step size, and G represents the element step size in the control window;
the results output by the seven edge detection modules are respectively input into an up-sampling module, and the up-sampling modules up-sample the feature images output by the edge detection modules by adopting a bilinear interpolation method;
known Q 11 =(x 1 ,y 1 ),Q 12 =(x 1 ,y 2 ),Q 21 =(x 2 ,y 1 ),Q 22 =(x 2 ,y 2 ) The pixel value of four points, the calculation formula of bilinear interpolation is:
wherein f (x, y) represents the pixel value of the sought point P (x, y);
and carrying out image fusion on the output feature images of the seven up-sampling modules, and obtaining weak slab contour features influenced by noise by fusing seven side output images with different depths, and simultaneously, more prominently representing obvious contour features of the slab, which are not influenced by noise.
Optionally, using weighted cross entropy as a loss function of the best feature extraction model;
the calculation formula of the weighted cross entropy loss function is as follows:
β=Y - /Y + +Y -
1-β=Y + /Y + +Y -
wherein Y is - And Y + Respectively representing edge and non-edge pixel points marked in the image data set; n represents the number of the edge detection module; w represents all learning parameters in the model; w is a parameter corresponding to n edge detection modules; sigma () is a sigmoid function, calculating the activation value of the pixel point; y is j Indicating whether pixel j is marked as an edge; x represents an input image; delta represents the weight of each scale level; the sigmoid function maps a real number into the interval of (0, 1), and its calculation formula is as follows:
optionally, the step S3 specifically includes:
s3-1, filtering the slab image to be detected by using wavelet transformation, and filtering out partial background noise information, thereby improving the subsequent detection precision;
the slab image to be detected is a signal distributed along the space, is a two-dimensional signal changing along the space axes x and y, and can be regarded as a group of digital arrays;
decomposing the signal into a series of discrete approximate components and detail components by using wavelet transformation, wherein the noise of the signal is mainly concentrated on the detail components of the signal, removing the noise detail components larger than a preset threshold value, and reconstructing the rest detail components and the approximate components by wavelet to obtain a smooth image;
s3-2, inputting the slab image to be detected after wavelet transformation and filtration into the optimal feature extraction model to obtain a feature extraction diagram of the slab image to be detected.
Optionally, as the best feature extraction model fuses seven side output images with different depths, noise edges are generated while weak edge features of deep slab contours are obtained, and the noise edges include: background noise, water stain noise and side guide roller noise, so the post-processing is required to be carried out on the feature map of the slab image to be detected, the effect of accurately extracting the contour features of the slab is achieved, and the post-processing specifically comprises:
s4-1, performing binarization processing on the feature map of the slab image to be detected to obtain a binary image; the calculation formula of the binarization threshold value is as follows:
g=ω 0 ω 1 (u 0 -u 1 ) 2
where g is the binarization threshold, ω 0 For the proportion of the target points to the total image, u 0 Is the average gray value omega of the target points 1 For the proportion of background points to images, u 1 The average gray value of the background points;
s4-2, performing morphological operation on the obtained binary image to ensure that the peripheral outline of the plate blank is separated from the noise of the side guide rollers at two sides of the rolling line, wherein the operation is performed by firstly corroding and then expanding;
the operation step of corrosion is that the core element with the size of 3 multiplied by 3 and the shape of square is utilized to carry out traversal operation on the binary image, and the part traversed by the core element in the binary image is AND-operated with the core element; the expansion operation steps are to use the nuclear element and the binary image to do a parallel operation;
the corrosion calculation formula is:
(f-b)(s,t)=min{f(s,x,t+y)-b(x,y)|(s+x),(t+y)∈Df;(x,y)∈D b }
wherein D is f And D b The definition fields of f and b are respectively, and f-b represents gray scale corrosion of f by b;
the calculation formula of the expansion is:
wherein f (s, t) is an input image, b (x, y) is a structural element, D f And D b The definition fields of f and b respectively,the representation is that b is used for gray scale expansion of f;
s4-3, calculating the area of each connected region of the image after morphological operation;
the area calculation formula of the communication area is as follows:
wherein the closed area D is surrounded by a piecewise smooth curve L, and the functions P (x, y) and Q (x, y) have first-order continuous partial derivatives on D;
s4-4, fully filling the communication area with black pixels, which is smaller than half of the maximum communication area, to remove partial noise area in the image, and obtaining a preliminary denoising image;
s4-5, performing morphological operation on the image subjected to preliminary denoising, and smoothing the edge of the slab to fill hole noise in the slab, so that the whole slab is independent and complete, and a final processed image is obtained;
s4-6, extracting all edges in the image by using the gradient amplitude and the gradient direction of the pixel points; the calculation formula of the gradient amplitude is as follows:
the calculation formula of the gradient direction is as follows:
wherein:
Gx=(f(x-1,y+1)+2f(x,y+1)+f(x+1,y+1))-(f(x-1,y-1)+2f(x,y-1)+f(x+1,y-1))
Gy=(f(x+1,y-1)+2f(x+1,y)+f(x+1,y+1))-(f(x-1,y-1)+2f(x-1,y)+f(x-1,y+1)
s4-7, calculating the perimeter of the outline formed by all edges, and finding the largest outline, wherein the calculation formula of the perimeter of the outline is as follows:
wherein a and b are the ranges of the curves, x (t), and y (t) is the parameter equation of the curves;
and S4-8, finally displaying the largest outline into the original image to finish detection.
Optionally, the specific method of full filling is as follows:
searching a minimum bounding rectangle of a smaller area, and assigning all pixel values in the minimum bounding rectangle to be 0.
In another aspect, there is provided a slab contour detection apparatus with water stain on a surface, the apparatus comprising:
the acquisition module is used for acquiring the intermediate blank image, constructing a sample data set and marking;
the construction module is used for constructing a convolutional neural network model, and training by using the marked data set to obtain an optimal feature extraction model;
the characteristic extraction module is used for preprocessing the slab image to be detected by wavelet transformation, and extracting the characteristics of the preprocessed slab image to be detected by utilizing the trained optimal characteristic extraction model to obtain a characteristic diagram of the slab image to be detected;
and the post-processing module is used for carrying out post-processing on the characteristic diagram of the slab image to be detected, completing the functions of removing noise and smoothing edges, and completing the contour detection of the slab image to be detected by extracting the edges and searching the maximum contour.
In another aspect, an electronic device is provided, the electronic device includes a processor and a memory, the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the slab contour detection method with water stains on the surface.
In another aspect, a computer readable storage medium is provided, in which at least one instruction is stored, the at least one instruction being loaded and executed by a processor to implement the above-described method for detecting a water spot-bearing slab profile.
The technical scheme provided by the embodiment of the invention has the beneficial effects that at least:
the invention extracts the outline features of the slab image based on the optimal feature extraction model constructed by the convolutional neural network, and detects the slab outline with water stain on the surface efficiently and accurately after post-treatment. Firstly, training a constructed convolutional neural network by using an image of a marked industrial site, so that the network is more beneficial to edge information of a slab with water stains on the surface, then, removing a part of background noise by using wavelet transformation before the neural network is input, enabling the features extracted by an optimal feature extraction model to be more focused on the edge features of the slab, then, inputting the features into the trained network to obtain a feature map of the slab to be detected, and then, carrying out post-treatment to extract the contour of the slab to be detected, thus finishing contour detection of the image of the slab to be detected. The traditional image processing method only can complete the detection of the image contour of a few plate blanks due to the singleness of the method, solves the problem of the contour detection of the plate blanks aiming at the condition that the water-stained plate blank images often have detection errors, is also suitable for images under normal conditions, has higher robustness to complex environments, further improves the detection precision, realizes the real-time online high-precision detection of the plate blank contour, and has important significance for the quality detection of rough rolling intermediate blank.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a method for detecting a plate blank contour with water stains on the surface, which is provided by an embodiment of the invention;
FIG. 2 is a schematic diagram of an intermediate blank image with water spots according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an optimal feature extraction model according to an embodiment of the present invention;
FIG. 4 is a schematic illustration of slab image feature extraction provided by an embodiment of the present invention;
fig. 5 is a schematic diagram of a slab image contour detection result to be detected according to an embodiment of the present invention;
FIG. 6 is a block diagram of a slab contour detection device with water stains on the surface, which is provided by the embodiment of the invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages to be solved more apparent, the following detailed description will be given with reference to the accompanying drawings and specific embodiments.
The embodiment of the invention provides a slab contour detection method with water stains on the surface, which can be realized by electronic equipment, wherein the electronic equipment can be a terminal or a server. As shown in fig. 1, a flow chart of a method for detecting a plate blank contour with water stains on the surface, the processing flow of the method may include the following steps:
step S1: acquiring an intermediate blank image, constructing a sample data set and marking;
step S2: constructing a convolutional neural network model, and training by using the marked data set to obtain an optimal feature extraction model;
step S3: preprocessing a slab image to be detected by wavelet transformation, and extracting the characteristics of the preprocessed slab image to be detected by using the trained optimal characteristic extraction model to obtain a characteristic diagram of the slab image to be detected;
step S4: and carrying out post-processing on the feature map of the slab image to be detected, completing the functions of removing noise and smoothing edges, and completing the contour detection of the slab image to be detected by the methods of extracting the edges and searching the maximum contour.
The embodiment of the invention mainly aims at the problem that the surface of the slab has water stains, the slab can be cooled by side water spraying when the temperature is too high in the rough rolling process, and a large amount of water stains are formed on the surface of an intermediate slab after the intermediate slab passes through a rolling mill, wherein part of the water stains can spread to the edge of the slab. In addition, when the rolled slab is wider, the slab is closer to the side guide rollers at the two sides, and the side guide rollers are illuminated by stronger heat radiation due to higher surface temperature of the slab, so that side guide roller noise is formed. I.e. the noise in the embodiment of the invention is mainly divided into background noise, water stain noise and side guide roller noise.
The difference between the gray values of the water stain noise and the background noise is smaller, and the gray values of the whole plate blank, the background and the water stain are larger. Under the condition of no water stain, the gray value difference between the slab and the background is larger, the gray change is obvious, and the edge of the slab is easy to extract; however, in the case of water stains, the difference between the slab covered by the water stains and the background is small, the gray level change is not obvious, and the detection error is caused by the detection according to the traditional method. For example, when the gray level of the noise in the background area is 20, the gray level of the slab is 150, and the gray level of the water spot covered slab is 50, in the conventional method, when the edge is detected by using the threshold value of the single gray level change (gradient) 150-20=130, the gray level change (gradient) 50-20=30 of the water covered portion is too small to be detected, and sometimes even the gray level change (gradient) between the water covered portion of 150-50=100 and the slab is regarded as the edge to cause detection errors. Therefore, the extraction of edges by using a single gray level change threshold in the conventional detection method causes the profile of the part of the slab edge covered by the water stain to be missing.
The optimal feature extraction model constructed based on the convolutional neural network in the embodiment of the invention can be used for extracting features by deepening the network layer number according to different gradient amplitudes and directions, and connecting edge detection modules by utilizing the convolutional kernel of 3*3 and combining the convolutional kernel of 1*1, so that weak gray level change (gradient) can be extracted, and all edge features can be extracted, wherein the edges comprise the edges of a background and a water-stain covered part, the edges of a background and a plate blank, the edges of the water-stain covered part and the plate blank, and the edges of a side guide roller and the background; and then, carrying out post-processing on the feature map of the slab image to be detected, which is output by the optimal feature extraction model, and detecting the edges of the slab and the background of the part covered by the water stain and the edges of the slab, filtering the noise edges of the part covered by the water stain and the slab and the noise edges of the side guide roller and the background, and accurately detecting the contour of the slab.
The following describes in detail a slab contour detection method with water stains on the surface according to an embodiment of the present invention with reference to fig. 2 to 5, including:
step S1: acquiring an intermediate blank image, constructing a sample data set and marking;
optionally, the S1 specifically includes:
s1-1, shooting and obtaining an RGB image of an intermediate blank with water stains on the surface by using an industrial CCD camera;
the data set image used for the experiment is from a rough rolling intermediate blank image of a production line of an industrial site 1580, and an industrial CCD camera is adopted for shooting, so that an RGB image with the resolution of 4096 x 2160 is obtained.
Of course, besides the water stain on the surface, the surface can also be oil stain or other noise.
An image of a waterstained intermediate blank in an embodiment of the invention is shown in fig. 2.
S1-2, intercepting an effective area of an acquired slab image to obtain an effective area image;
in this embodiment, the size of the intercepted image is 1024 x 1024;
s1-3, labeling the effective area image, and selecting label values with appropriate contour labeling to form a sample data set;
the label of the effective area is the real contour information of the slab.
S1-4, dividing the sample data set into a training set and a testing set proportionally.
The embodiment of the invention divides the sample data set into a training set and a testing set according to the proportion of 8:2.
Step S2: constructing a convolutional neural network model, and training by using the marked data set to obtain an optimal feature extraction model;
optionally, as shown in fig. 3, the best feature extraction model in S2 is formed by combining seven edge detection modules with seven up-sampling modules, and as the number of network layers is deepened, deeper feature information in the image is extracted, weak slab contour edge features of the part covered by the water stain are extracted, and finally, the image is output by fusing seven sides of the convolutional neural network, so that the obvious slab edge features are obtained, and meanwhile, weak edges influenced by the water stain are also extracted;
the edge detection modules are fully connected through a convolution kernel of 1 multiplied by 1, the side edge output result of the edge detection modules is used as the input of the up-sampling module, and finally the output feature graphs of the seven up-sampling modules are fused to obtain the output result of the optimal feature extraction model;
the first edge detection module consists of 1 convolution layer, the second edge detection module consists of 1 convolution layer and 1 pooling layer, the third edge detection module consists of 2 convolution layers and 1 pooling layer, the fourth edge detection module consists of 3 convolution layers and 1 pooling layer, the fifth edge detection module consists of 3 convolution layers, the sixth edge detection module consists of 3 convolution layers, the seventh edge detection module consists of 2 convolution layers, and the convolution kernel size of all edge detection modules is 3×3;
the convolution layer calculation formula is as follows:
wherein N represents the size of the image output after convolution, W represents the size of the input image, F represents the size of the convolution kernel, P represents the number of pixels filled, and S represents the step size;
in the embodiment of the present invention, w=1024, f=3, p=1, s=2, so that the size of the image output after the convolution layer is calculated to be 512;
the calculation formula of the pooling layer is as follows:
wherein N represents the size of the image output after pooling, W represents the size of the input image, F represents the size of the pooling kernel, P represents the number of filled pixels, S represents the step size, and G represents the element step size in the control window;
in the embodiment of the present invention, w=1024, f=3, p=1, s=2, and g=1, so that the image size after the output of the pooling layer is 512 can be calculated.
The results output by the seven edge detection modules are respectively input into an up-sampling module, and the up-sampling modules up-sample the feature images output by the edge detection modules by adopting a bilinear interpolation method;
known Q 11 =(x 1 ,y 1 ),Q 12 =(x 1 ,y 2 ),Q 21 =(x 2 ,y 1 ),Q 22 =(x 2 ,y 2 ) Four-point pixelThe calculation formula of bilinear interpolation is:
wherein f (x, y) represents the pixel value of the sought point P (x, y);
in embodiments of the present invention, Q is known to be 11 =(1,1),Q 12 =(1,3),Q 21 =(3,1),Q 22 The pixel values of four points = (3, 3) are 50,100,150,200 respectively, and the calculation formula of the pixel bilinear interpolation of the point to be solved P (2, 2) is:
=125
and carrying out image fusion on the output feature images of the seven up-sampling modules, and obtaining weak slab contour features influenced by noise by fusing seven side output images with different depths, and simultaneously, more prominently representing obvious contour features of the slab, which are not influenced by noise.
Optionally, using weighted cross entropy as a loss function of the best feature extraction model;
the calculation formula of the weighted cross entropy loss function is as follows:
β=|Y - |/|Y + +Y - |
1-β=|Y + |/|Y + +Y - |
wherein Y is - And Y + Respectively representing edge and non-edge pixel points marked in the image data set; n represents an edgeThe number of the detection module; w represents all learning parameters in the model; w is a parameter corresponding to n edge detection modules (each module does not share a weight); sigma () is a sigmoid function, calculating the activation value of the pixel point; y is j Indicating whether pixel j is marked as an edge; x represents an input image; delta represents the weight of each scale level; the sigmoid function maps a real number into the interval of (0, 1), and its calculation formula is as follows:
optionally, the embodiment of the invention optimizes the model by using a grid search method and a cross-validation method; in this embodiment, the set network model training hyper-parameters are as follows: the optimizers were treated with Adam with a learning rate of 0.001, a batch size of 8, and an events of 150 k.
Optionally, the embodiment of the present invention sets a suitable model evaluation index, and in the embodiment of the present invention, the decision coefficient and the mean square error are used for calculation:
wherein R is 2 Represents a decision coefficient, f (x i ) Representing predicted data, y i Representing real data, y representing a real data average;
where MSE denotes mean square error, f (x i ) Representing predicted data, y i Representing real data, n representing the number of data.
According to the embodiment of the invention, the comprehensive optimal model in the model evaluation index is saved, and the optimal feature extraction model is obtained.
Step S3: preprocessing a slab image to be detected by wavelet transformation, and extracting the characteristics of the preprocessed slab image to be detected by using the trained optimal characteristic extraction model to obtain a characteristic diagram of the slab image to be detected;
optionally, the step S3 specifically includes:
s3-1, filtering the slab image to be detected by using wavelet transformation, and filtering out partial background noise information, thereby improving the subsequent detection precision;
the slab image to be detected is a signal distributed along the space, is a two-dimensional signal changing along the space axes x and y, and can be regarded as a group of digital arrays;
decomposing the signal into a series of discrete approximate components and detail components by using wavelet transformation, wherein the noise of the signal is mainly concentrated on the detail components of the signal, removing the noise detail components larger than a preset threshold value, and reconstructing the rest detail components and the approximate components by wavelet to obtain a smooth image;
s3-2, inputting the slab image to be detected after wavelet transformation and filtration into the optimal feature extraction model to obtain a feature extraction diagram of the slab image to be detected.
The slab image feature extraction diagram obtained in the example of the present invention is shown in fig. 4.
Step S4: and carrying out post-processing on the feature map of the slab image to be detected, completing the functions of removing noise and smoothing edges, and completing the contour detection of the slab image to be detected by the methods of extracting the edges and searching the maximum contour.
Optionally, as the best feature extraction model fuses seven side output images with different depths, noise edges are generated while weak edge features of deep slab contours are obtained, and the noise edges include: background noise, water stain noise and side guide roller noise, so the post-processing is required to be carried out on the feature map of the slab image to be detected, the effect of accurately extracting the contour features of the slab is achieved, and the post-processing specifically comprises:
s4-1, performing binarization processing on the feature map of the slab image to be detected to obtain a binary image; the calculation formula of the binarization threshold value is as follows:
g=ω 0 ω 1 (u 0 -u 1 ) 2
where g is the binarization threshold, ω 0 For the proportion of the target points to the total image, u 0 Is the average gray value omega of the target points 1 For the proportion of background points to images, u 1 The average gray value of the background points;
in the embodiment of the invention, the target point is 0.38, the average gray value of the target point is 95, the background point is 0.62 and the average gray value of the background point is 76, so that the binary threshold value of the embodiment of the invention is 85, and the gray value of the pixel larger than the binary threshold value is assigned 255 to represent full white; and the gray value of the pixel smaller than or equal to the binarization threshold value is assigned to be 0, and the pixel is completely black.
S4-2, performing morphological operation on the obtained binary image to ensure that the peripheral outline of the plate blank is separated from the noise of the side guide rollers at two sides of the rolling line, wherein the operation is performed by firstly corroding and then expanding;
the operation step of corrosion is that the core element with the size of 3 multiplied by 3 and the shape of square is utilized to carry out traversal operation on the binary image, the part traversed by the core element in the binary image and the core element are subjected to AND operation (for example, the gray value of a certain point in the binary image is 255, and the gray value of the core element traversed to the point is 0, after corrosion operation, the gray value of the point is 0); the expansion operation step is to use the kernel element and the binary image to do and calculate (for example, the gray value of a certain point in the binary image is 255, and the gray value of the kernel element traversing to the point is 0, after corrosion operation, the gray value of the point is 255);
the corrosion calculation formula is:
(f-b)(s,t)=min{f(s,x,t+y)-b(x,y)|(s+x),(t+y)∈D f ;(x,y)∈D b }
wherein D is f And D b The definition fields of f and b are respectively, and f-b represents gray scale corrosion of f by b;
the calculation formula of the expansion is:
wherein f (s, t) is an input image, b (x, y) is a structural element, D f And D b The definition fields of f and b respectively,the representation is that b is used for gray scale expansion of f;
s4-3, calculating the area of each connected region of the image after morphological operation;
the area calculation formula of the communication area is as follows:
wherein the closed area D is surrounded by a piecewise smooth curve L, and the functions P (x, y) and Q (x, y) have first-order continuous partial derivatives on D;
s4-4, fully filling the communication area with black pixels, which is smaller than half of the maximum communication area, to remove partial noise area in the image, and obtaining a preliminary denoising image;
optionally, the specific method of full filling is as follows:
searching a minimum bounding rectangle of a smaller area, and assigning all pixel values in the minimum bounding rectangle to be 0.
S4-5, performing morphological operation on the image subjected to preliminary denoising, and smoothing the edge of the slab to fill hole noise in the slab, so that the whole slab is independent and complete, and a final processed image is obtained;
s4-6, extracting all edges in the image by using the gradient amplitude and the gradient direction of the pixel points; the calculation formula of the gradient amplitude is as follows:
the calculation formula of the gradient direction is as follows:
wherein:
Gx=(f(x-1,y+1)+2f(x,y+1)+f(x+1,y+1))-(f(x-1,y-1)+2f(x,y-1)+f(x+1,y-1))
Gy=(f(x+1,y-1)+2f(x+1,y)+f(x+1,y+1))-
(f(x-1,y-1)+2f(x-1,y)+f(x-1,y+1)
in the embodiment of the invention, the edge with the gradient amplitude larger than 20 and the gradient direction of (0, 1) is extracted.
S4-7, calculating the perimeter of the outline formed by all edges, and finding the largest outline, wherein the calculation formula of the perimeter of the outline is as follows:
wherein a and b are the ranges of the curves, x (t), and y (t) is the parameter equation of the curves;
and S4-8, finally displaying the largest outline into the original image to finish detection.
In the embodiment of the invention, the contour detection effect of the slab with water stains is shown in fig. 5, wherein the detected contour lines are marked and represented by white lines, the whole method in the embodiment of the invention takes 421ms, the requirement of real-time detection on industrial sites is met, and the influence of the water stains on the surface of the slab and the side guide plates on the contour detection is effectively avoided by the method as can be seen from the figure, and the problem of slab contour detection is accurately and efficiently solved.
As shown in fig. 6, the embodiment of the invention further provides a device for detecting a contour of a slab with water stains on a surface, which comprises:
an acquisition module 610, configured to acquire an intermediate blank image, construct a sample dataset, and mark;
the construction module 620 is configured to construct a convolutional neural network model, and train with the marked data set to obtain an optimal feature extraction model;
the feature extraction module 630 is configured to pre-process the slab image to be detected by using wavelet transformation, and perform feature extraction on the pre-processed slab image to be detected by using the trained optimal feature extraction model, so as to obtain a feature map of the slab image to be detected;
and the post-processing module 640 is used for carrying out post-processing on the feature map of the slab image to be detected, completing the functions of removing noise and smoothing edges, and completing the contour detection of the slab image to be detected by extracting the edges and searching the maximum contour.
The functional structure of the slab contour detection device with water stains on the surface provided by the embodiment of the invention corresponds to the slab contour detection method with water stains on the surface provided by the embodiment of the invention, and is not repeated here.
Fig. 7 is a schematic structural diagram of an electronic device 700 according to an embodiment of the present invention, where the electronic device 700 may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units, CPU) 701 and one or more memories 702, where at least one instruction is stored in the memories 702, and the at least one instruction is loaded and executed by the processors 701 to implement the steps of the above-mentioned method for detecting a water-stained slab contour.
In an exemplary embodiment, a computer readable storage medium, such as a memory comprising instructions executable by a processor in the terminal to perform the above-described method of surface water spot slab contour detection is also provided. For example, the computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims (10)

1. The slab contour detection method with the water stain on the surface is characterized by comprising the following steps of:
step S1: acquiring an intermediate blank image, constructing a sample data set and marking;
step S2: constructing a convolutional neural network model, and training by using the marked data set to obtain an optimal feature extraction model;
step S3: preprocessing a slab image to be detected by wavelet transformation, and extracting the characteristics of the preprocessed slab image to be detected by using the trained optimal characteristic extraction model to obtain a characteristic diagram of the slab image to be detected;
step S4: and carrying out post-processing on the feature map of the slab image to be detected, completing the functions of removing noise and smoothing edges, and completing the contour detection of the slab image to be detected by the methods of extracting the edges and searching the maximum contour.
2. The method according to claim 1, wherein S1 specifically comprises:
s1-1, shooting and obtaining an RGB image of an intermediate blank with water stains on the surface by using an industrial CCD camera;
s1-2, intercepting an effective area of an acquired slab image to obtain an effective area image;
s1-3, labeling the effective area image, and selecting label values with appropriate contour labeling to form a sample data set;
s1-4, dividing the sample data set into a training set and a testing set proportionally.
3. The method according to claim 1, wherein the best feature extraction model in S2 is formed by combining seven edge detection modules with seven up-sampling modules, and as the number of network layers is increased, deeper feature information in the image is extracted, weak slab contour edge features of the water-stain covered part are extracted, and finally the image is output by fusing seven sides of the convolutional neural network, so that the obvious slab edge features are obtained, and weak edges influenced by the water-stain are also extracted;
the edge detection modules are fully connected through a convolution kernel of 1 multiplied by 1, the side edge output result of the edge detection modules is used as the input of the up-sampling module, and finally the output feature graphs of the seven up-sampling modules are fused to obtain the output result of the optimal feature extraction model;
the first edge detection module consists of 1 convolution layer, the second edge detection module consists of 1 convolution layer and 1 pooling layer, the third edge detection module consists of 2 convolution layers and 1 pooling layer, the fourth edge detection module consists of 3 convolution layers and 1 pooling layer, the fifth edge detection module consists of 3 convolution layers, the sixth edge detection module consists of 3 convolution layers, the seventh edge detection module consists of 2 convolution layers, and the convolution kernel size of all edge detection modules is 3×3;
the convolution layer calculation formula is as follows:
wherein N represents the size of the image output after convolution, W represents the size of the input image, F represents the size of the convolution kernel, P represents the number of pixels filled, and S represents the step size;
the calculation formula of the pooling layer is as follows:
wherein N represents the size of the image output after pooling, W represents the size of the input image, F represents the size of the pooling kernel, P represents the number of filled pixels, S represents the step size, and G represents the element step size in the control window;
the results output by the seven edge detection modules are respectively input into an up-sampling module, and the up-sampling modules up-sample the feature images output by the edge detection modules by adopting a bilinear interpolation method;
known Q 11 =(x 1 ,y 1 ),Q 12 =(x 1 ,y 2 ),Q 21 =(x 2 ,y 1 ),Q 22 =(x 2 ,y 2 ) The pixel value of four points, the calculation formula of bilinear interpolation is:
wherein f (x, y) represents the pixel value of the sought point P (x, y);
and carrying out image fusion on the output feature images of the seven up-sampling modules, and obtaining weak slab contour features influenced by noise by fusing seven side output images with different depths, and simultaneously, more prominently representing obvious contour features of the slab, which are not influenced by noise.
4. A method according to claim 3, characterized in that weighted cross entropy is used as a loss function of the best feature extraction model;
the calculation formula of the weighted cross entropy loss function is as follows:
β=|Y - |/|Y + +Y - |
1-β=|Y + |/|Y + +Y - |
wherein Y is - And Y + Respectively representing edge and non-edge pixel points marked in the image data set; n represents the number of the edge detection module; w represents all learning parameters in the model; w is a parameter corresponding to n edge detection modules; sigma () is a sigmoid function, calculating the activation value of the pixel point; y is j Indicating whether pixel j is marked as an edge; x represents an input image; delta represents the weight of each scale level; the sigmoid function maps a real number into the interval of (0, 1), and its calculation formula is as follows:
5. the method according to claim 1, wherein S3 specifically comprises:
s3-1, filtering the slab image to be detected by using wavelet transformation, and filtering out partial background noise information, thereby improving the subsequent detection precision;
the slab image to be detected is a signal distributed along the space, is a two-dimensional signal changing along the space axes x and y, and can be regarded as a group of digital arrays;
decomposing the signal into a series of discrete approximate components and detail components by using wavelet transformation, wherein the noise of the signal is mainly concentrated on the detail components of the signal, removing the noise detail components larger than a preset threshold value, and reconstructing the rest detail components and the approximate components by wavelet to obtain a smooth image;
s3-2, inputting the slab image to be detected after wavelet transformation and filtration into the optimal feature extraction model to obtain a feature extraction diagram of the slab image to be detected.
6. The method of claim 1, wherein, as the best feature extraction model fuses seven side output images of different depths, noise edges are generated while weak edge features of deep slab contours are obtained, and the noise edges include: background noise, water stain noise and side guide roller noise, so the post-processing is required to be carried out on the feature map of the slab image to be detected, the effect of accurately extracting the contour features of the slab is achieved, and the post-processing specifically comprises:
s4-1, performing binarization processing on the feature map of the slab image to be detected to obtain a binary image; the calculation formula of the binarization threshold value is as follows:
g=ω 0 ω 1 (u 0 -u 1 ) 2
where g is the binarization threshold, ω 0 For the proportion of the target points to the total image, u 0 Is the average gray value omega of the target points 1 For the proportion of background points to images, u 1 The average gray value of the background points;
s4-2, performing morphological operation on the obtained binary image to ensure that the peripheral outline of the plate blank is separated from the noise of the side guide rollers at two sides of the rolling line, wherein the operation is performed by firstly corroding and then expanding;
the operation step of corrosion is that the core element with the size of 3 multiplied by 3 and the shape of square is utilized to carry out traversal operation on the binary image, and the part traversed by the core element in the binary image is AND-operated with the core element; the expansion operation steps are to use the nuclear element and the binary image to do a parallel operation;
the corrosion calculation formula is:
(f-b)(s,t)=min{f(s,x,t+y)-b(x,y)|(s+x),(t+y)∈D f ;(x,y)∈D b }
wherein D is f And D b The definition fields of f and b are respectively, and f-b represents gray scale corrosion of f by b;
the calculation formula of the expansion is:
wherein f (s, t) is an input image, b (x, y) is a structural element, D f And D b The definition fields of f and b respectively,the representation is that b is used for gray scale expansion of f;
s4-3, calculating the area of each connected region of the image after morphological operation;
the area calculation formula of the communication area is as follows:
wherein the closed area D is surrounded by a piecewise smooth curve L, and the functions P (x, y) and Q (x, y) have first-order continuous partial derivatives on D;
s4-4, fully filling the communication area with black pixels, which is smaller than half of the maximum communication area, to remove partial noise area in the image, and obtaining a preliminary denoising image;
s4-5, performing morphological operation on the image subjected to preliminary denoising, and smoothing the edge of the slab to fill hole noise in the slab, so that the whole slab is independent and complete, and a final processed image is obtained;
s4-6, extracting all edges in the image by using the gradient amplitude and the gradient direction of the pixel points; the calculation formula of the gradient amplitude is as follows:
the calculation formula of the gradient direction is as follows:
wherein:
Gx=(f(x-1,y+1)+2f(x,y+1)+f(x+1,y+1))-
(f(x-1,y-1)+2f(x,y-1)+f(x+1,y-1))
Gy=(f(x+1,y-1)+2f(x+1,y)+f(x+1,y+1))-
(f(x-1,y-1)+2f(x-1,y)+f(x-1,y+1)
s4-7, calculating the perimeter of the outline formed by all edges, and finding the largest outline, wherein the calculation formula of the perimeter of the outline is as follows:
wherein a and b are the ranges of the curves, x (t), and y (t) is the parameter equation of the curves;
and S4-8, finally displaying the largest outline into the original image to finish detection.
7. The method according to claim 6, wherein the specific method of full filling is:
searching a minimum bounding rectangle of a smaller area, and assigning all pixel values in the minimum bounding rectangle to be 0.
8. A device for detecting the contour of a slab with water stains on the surface, which is characterized by comprising:
the acquisition module is used for acquiring the intermediate blank image, constructing a sample data set and marking;
the construction module is used for constructing a convolutional neural network model, and training by using the marked data set to obtain an optimal feature extraction model;
the characteristic extraction module is used for preprocessing the slab image to be detected by wavelet transformation, and extracting the characteristics of the preprocessed slab image to be detected by utilizing the trained optimal characteristic extraction model to obtain a characteristic diagram of the slab image to be detected;
and the post-processing module is used for carrying out post-processing on the characteristic diagram of the slab image to be detected, completing the functions of removing noise and smoothing edges, and completing the contour detection of the slab image to be detected by extracting the edges and searching the maximum contour.
9. An electronic device comprising a processor and a memory, the memory having at least one instruction stored therein, wherein the at least one instruction is loaded and executed by the processor to implement the method of surface water spot slab contour detection as recited in any one of claims 1-7.
10. A computer readable storage medium having stored therein at least one instruction, wherein the at least one instruction is loaded and executed by a processor to implement a method of slab contour detection with water stains on a surface according to any one of claims 1 to 7.
CN202310750776.4A 2023-06-25 2023-06-25 Method and device for detecting slab contour with water stain on surface Pending CN116883439A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310750776.4A CN116883439A (en) 2023-06-25 2023-06-25 Method and device for detecting slab contour with water stain on surface

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310750776.4A CN116883439A (en) 2023-06-25 2023-06-25 Method and device for detecting slab contour with water stain on surface

Publications (1)

Publication Number Publication Date
CN116883439A true CN116883439A (en) 2023-10-13

Family

ID=88267073

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310750776.4A Pending CN116883439A (en) 2023-06-25 2023-06-25 Method and device for detecting slab contour with water stain on surface

Country Status (1)

Country Link
CN (1) CN116883439A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117576416A (en) * 2024-01-15 2024-02-20 北京阿丘机器人科技有限公司 Workpiece edge area detection method, device and storage medium
CN117828900A (en) * 2024-03-04 2024-04-05 宝鸡核力材料科技有限公司 Impurity removal reminding method, system and medium applied to slab rolling

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117576416A (en) * 2024-01-15 2024-02-20 北京阿丘机器人科技有限公司 Workpiece edge area detection method, device and storage medium
CN117828900A (en) * 2024-03-04 2024-04-05 宝鸡核力材料科技有限公司 Impurity removal reminding method, system and medium applied to slab rolling

Similar Documents

Publication Publication Date Title
CN116883439A (en) Method and device for detecting slab contour with water stain on surface
CN109191459B (en) Automatic identification and rating method for continuous casting billet macrostructure center segregation defect
CN111145161A (en) Method for processing and identifying pavement crack digital image
CN109840483B (en) Landslide crack detection and identification method and device
CN112233067A (en) Hot rolled steel coil end face quality detection method and system
CN115546768B (en) Pavement marking identification method and system based on multi-scale mechanism and attention mechanism
CN113240623B (en) Pavement disease detection method and device
CN111161260A (en) Hot-rolled strip steel surface defect detection method and device based on deep learning
CN111539938A (en) Method, system, medium and electronic terminal for detecting curvature of rolled strip steel strip head
CN116152261B (en) Visual inspection system for quality of printed product
CN114596500A (en) Remote sensing image semantic segmentation method based on channel-space attention and DeeplabV3plus
CN116563262A (en) Building crack detection algorithm based on multiple modes
CN111429437B (en) Image non-reference definition quality detection method for target detection
CN115511775A (en) Light-weight ceramic tile surface defect detection method based on semantic segmentation
CN112950594A (en) Method and device for detecting surface defects of product and storage medium
CN114529715B (en) Image identification method and system based on edge extraction
CN113658180B (en) Surface defect region segmentation method and device based on spatial context guidance
CN115330705A (en) Skin paint surface defect detection method based on adaptive weighting template NCC
CN112116561B (en) Power grid transmission line detection method and device based on image processing fusion network weight
CN115512322A (en) Multidimensional pavement damage data processing method
CN112837329B (en) Tibetan ancient book document image binarization method and system
CN111553874B (en) Light center extraction method for surface line structure of light-reflecting nuclear fuel rod
CN114419317A (en) Light strip center extraction method for light with complex environment line structure
CN112198170A (en) Detection method for identifying water drops in three-dimensional detection of outer surface of seamless steel pipe
CN117475262B (en) Image generation method and device, storage medium and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination