CN112926685A - Industrial steel oxidation zone target detection method, system and equipment - Google Patents
Industrial steel oxidation zone target detection method, system and equipment Download PDFInfo
- Publication number
- CN112926685A CN112926685A CN202110337542.8A CN202110337542A CN112926685A CN 112926685 A CN112926685 A CN 112926685A CN 202110337542 A CN202110337542 A CN 202110337542A CN 112926685 A CN112926685 A CN 112926685A
- Authority
- CN
- China
- Prior art keywords
- oxidation zone
- training
- steel
- network
- tiny
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The application discloses a method, a system and equipment for detecting an oxidation zone target of steel in industry, which are used for collecting oxidation zone sample images of different steel types under different illumination intensities and carrying out image preprocessing to obtain an initial oxidation zone sample data set; dividing the training set, the verification set and the test set into a training set, a verification set and a test set according to a preset proportion; improving the network structure of the steel based on the original YOLOv4-tiny algorithm to obtain a steel oxidation zone target detection algorithm; training the optimized network by using a training set, and selecting a weight file with the optimal mAP; and integrating the trained network in a detection system, and detecting the steel oxidation zone image to obtain the position information of the oxidation zone. The optimized network reduces the calculation parameters, improves the capability of the network for extracting effective characteristics, and can reduce the misjudgment and missing detection in the detection. The compressed model can be deployed at a small embedded device end to realize online detection, the robustness of the detection method to a complex industrial environment is improved, and a technical guarantee is provided for industrial intelligent manufacturing.
Description
Technical Field
The application relates to the technical field of industrial intelligent detection, in particular to a method, a system and equipment for detecting a steel oxidation zone target in industry.
Background
The steel oxidation zone is a steel surface defect formed in the welding process, and is easy to cause air holes, slag inclusion and cracks. In order to ensure the quality of workpieces, the positions of steel oxidation belts are required to be marked in the industrial production process. The traditional technology often adopts the mode of artifical mark, has a lot of problems: the false detection rate is high due to the fact that the manually marked positions are not strict; the steel workpiece has large volume and is not easy to operate manually; industrial production sites are extremely dangerous and may cause operational accidents and the like. Therefore, the manual quality inspection mode greatly reduces the workpiece quality and the production efficiency, and the loss of the production cost is caused.
In the hot tide of industrial intelligent manufacturing, a target detection method based on deep learning is combined with small embedded equipment, so that the target detection method is applied to a steel product production line. The target detection method has higher detection speed and precision, not only solves the problems caused by manual quality inspection, but also can adapt to complex industrial scenes and meet the requirement of production instantaneity. With the continuous development of the target detection method based on deep learning, the detection precision is increased in proportion to the complexity of a network structure, and the requirement of the operating environment on the hardware performance is higher and higher. The small embedded equipment is not as powerful as a large server, and a lightweight target detection method needs to be selected to realize the deployment of the target detection method on the small embedded equipment.
The lightweight target detection method has few parameters and small model, and the detection precision is reduced along with the simplification of the network although the method is suitable for the low computational power condition of hardware. In order to meet the detection requirements of steel production enterprises, the detection algorithm must be ensured to take accuracy and real-time into consideration. Therefore, when the detection speed of the steel oxidation zone is increased, reducing the size of the model and ensuring high detection precision become technical problems to be solved urgently.
Disclosure of Invention
In order to solve the technical problems, the following technical scheme is provided:
in a first aspect, an embodiment of the present application provides a method for detecting a steel oxidation zone target in industry, where the method includes: acquiring oxidation zone sample images of different steel types under different illumination intensities, and performing image preprocessing to obtain an initial oxidation zone sample data set; dividing the sample data set of the oxidation zone into a training set, a verification set and a test set according to a preset proportion; improving the network structure of the steel oxidation zone based on the original YOLOv4-tiny algorithm, and resetting the prior frame belonging to the steel oxidation zone to obtain a steel oxidation zone target detection algorithm; training the optimized YOLOv4-tiny network by using a training set, and selecting a weight file with the optimal mAP; and integrating the trained network in a detection system, and detecting the steel oxidation zone image to obtain the position information of the oxidation zone.
By adopting the implementation mode, the lightweight network based on YOLOv4-tiny is optimized, the capability of extracting effective characteristics is improved, and misjudgment and missing detection in detection can be reduced. The optimized network not only reduces the calculation parameters, but also improves the capability of the network for extracting effective characteristics, and can reduce the misjudgment and missing detection in the detection. The compressed model can be deployed at a small embedded device end to realize online detection, the robustness of the detection method to a complex industrial environment is improved, and a technical guarantee is provided for the development of industrial intelligent manufacturing.
With reference to the first aspect, in a first possible implementation manner of the first aspect, acquiring oxidation zone sample images of different steel types under different illumination intensities, and performing image preprocessing to obtain an initial oxidation zone sample data set, includes: arranging an industrial grade high-definition camera to shoot different types of steel oxidation zones under different illumination in real time on an operation table; generating a picture for the collected video reading frame, and renaming the picture according to the sequence number to form a unique ID of the picture; marking the position of the oxidation zone of the steel by using marking software to form a corresponding xml file and form an initial oxidation zone sample data set; and carrying out data enhancement processing on the data set, and splicing the four pictures by means of random cutting, random turning, blurring and the like to obtain training data.
With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, dividing the set of oxidized band sample data into a training set, a verification set, and a test set according to a preset proportion includes: dividing a training set, a verification set and a test set according to a ratio of 6:2:2, wherein the training set is used for model training, the verification set is used for evaluating a model, and the test set is used for evaluating the generalization ability of the model; preparing folders according to a VOC format: JPEGImages store all the oxidation zone sample images; the Annotations store the annotation images to generate one-to-one corresponding xml files; and establishing a Mian folder under ImageSets to store train.
With reference to the first aspect, in a third possible implementation manner of the first aspect, the network structure based on the original YOLOv4-tiny algorithm includes a backhaul feature extraction network, a Neck feature fusion module, and a Yolo Head prediction module, where the backhaul feature extraction network is a CSPDarknet53-tiny network structure, and outputs two feature layers with sizes of 26 × 26 and 13 × 13; the Neck feature fusion module performs feature fusion on the two feature layers output in the last step for a feature pyramid FPN; and predicting the obtained features by a Yolo Head prediction module, and outputting prediction results of two feature layers.
With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, the method for obtaining a target detection algorithm of a steel oxidation zone based on an original YOLOv4-tiny algorithm by improving a network structure thereof includes: adding an M-SPP structure after the last characteristic layer of the CSPDarknet-tiny network, and processing by utilizing two maximum pooling with different scales with extremely small calculated amount; model compression is realized through a sparse training and channel pruning multi-stage iterative training mode. The network reasoning time is shortened, and the redundant consumption of the model on the computing power of the small embedded equipment is reduced; stacking and outputting the feature maps subjected to the maximum pooling to FPN convolution, then performing double up-sampling, and performing feature fusion with a second Resblock _ body (26, 256) feature layer of the backbone network; performing regression prediction on the obtained features by a Yolo Head prediction module, wherein the Yolo Head prediction module comprises a first classifier and a second classifier, and the first classifier receives the fusion features with the output size of 13 × 13 output by the feature fusion module; the second classifier receives the fused features with the output size of 26 x 26 from the feature fusion module; the output prediction results shape are (13,13, N), (19,19, N), where N ═ S + Conf + Class, and S ═ 4, represent the position information of the prediction frame, Conf ═ 1 represents the confidence level, i.e., the Class probability, and Class ═ the Class number, respectively; and decoding according to the output prediction result to obtain the position information of the prediction frame, wherein the position information comprises the coordinates (bx, by) at the upper left corner of the prediction frame and the width (bw, bh) of the prediction frame.
With reference to the first aspect, in a fifth possible implementation manner of the first aspect, the training of the optimized YOLOv4-tiny network by using a training set is used to select a weight file with an optimal mep, where the method includes: configuring an operating environment, namely an Ubuntu system, a CUDA10.2 and an OpenCV4.1.1, and selecting YOLOv4-tiny based on a Darknet framework for training; setting model parameters, adjusting values of batch and subdivisions according to the calculation force condition of the small embedded equipment, and setting the width and height values, the learning rate, the iteration times and the category number of input pictures; the initial training uses the official pre-training weight file of YOLOv4-tiny, and the weight file generated during the setup training is stored every 1000 rounds.
With reference to the fourth possible implementation manner of the first aspect, in a sixth possible implementation manner of the first aspect, the method for detecting an image of an oxidation band of steel to obtain location information of the oxidation band includes: according to the position information (bx, by, bw, bh) of the target frame, carrying out standardization processing to obtain a central point position coordinate (cx, cy) of the target frame; the detection system of the small embedded device sends the detection target position information to the production line control device PLC through a serial communication protocol to be transmitted.
With reference to the first aspect or any one of the first to the sixth possible implementation manners of the first aspect, in a seventh possible implementation manner of the first aspect, the resetting the prior frame size according to the real frame labeled in the steel oxidation strip data set includes: and aiming at the real frame marked in the steel oxidation zone data set, obtaining a plurality of anchor frame values by using a clustering algorithm, and modifying the value of an anchor box in the parameter file as an initial parameter before training.
In a second aspect, the present application provides a system for detecting a target of an oxidation zone of steel in industry, where the system includes: the acquisition module is used for acquiring oxidation zone sample images of different steel types under different illumination intensities and carrying out image preprocessing to obtain an initial oxidation zone sample data set; the data set processing module is used for dividing the sample data set of the oxidation zone into a training set, a verification set and a test set according to a preset proportion; the network improvement module is used for improving the network structure of the steel oxidation zone based on the original YOLOv4-tiny algorithm to obtain a target detection algorithm of the steel oxidation zone; the training module is used for training the optimized YOLOv4-tiny network by utilizing a training set and selecting a weight file with the optimal mAP; and the detection module is used for integrating the trained network into a detection system and detecting the steel oxidation zone image to obtain the position information of the oxidation zone.
In a third aspect, an embodiment of the present application provides an apparatus, including: a processor; a memory for storing computer executable instructions; when the processor executes the computer-executable instructions, the processor executes the method for detecting the target of the steel oxidation zone in the industry according to the first aspect or any one of the possible implementation manners of the first aspect, and identifies the steel oxidation zone.
Drawings
FIG. 1 is a schematic flow chart of a method for detecting an oxidation zone target of steel in industry according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of a sample image with data enhancement provided by an embodiment of the present application;
FIG. 3 is a schematic diagram of 6 anchor size results after clustering according to a labeled steel oxidation zone data set provided in the embodiment of the present application;
FIG. 4 is a schematic diagram of an improved YOLOv4-tiny network structure provided in the embodiments of the present application;
FIG. 5 is a schematic structural diagram of an M-SPP module provided in an embodiment of the present application;
FIG. 6 is a schematic diagram illustrating the identification effect of the improved YOLOv4-tiny model on the steel oxidation zone according to the embodiment of the present application;
FIG. 7 is a schematic diagram of a system for detecting a steel oxidation zone target in industry according to an embodiment of the present disclosure;
fig. 8 is a schematic diagram of an apparatus provided in an embodiment of the present application.
Detailed Description
The present invention will be described with reference to the accompanying drawings and embodiments.
Fig. 1 is a schematic flow chart of a method for detecting an industrial steel oxidation zone target according to an embodiment of the present disclosure, and referring to fig. 1, the method for detecting an industrial steel oxidation zone target according to the embodiment includes:
s101, acquiring oxidation zone sample images of different steel types under different illumination intensities, and performing image preprocessing to obtain an initial oxidation zone sample data set.
The detection platform position is fixed in industrial production, for avoiding the light to cause the influence, selects the light of two kinds of tones of warm colour and cold colour to shoot under the intensity of high, well, low three kinds of differences, gathers multiple type steel sample image, must avoid leading to the fact the condition of lou examining because of steel reflection of light and kind are not enough, guarantees the variety and the sufficiency of data set.
Specifically, an industrial grade high-definition camera is placed at a position 60cm above an operation table, the resolution of the camera is adjusted to 640 x 480, and steel oxidation zones of different types under different illumination are shot in real time. And generating pictures for the collected video at the rate of 30 frames per second, and renaming the pictures uniquely according to the sequence numbers to form unique IDs of the pictures. And marking the position of the oxidation zone of the steel by using the labeling software labelImg, and storing xml files corresponding to the image IDs one by one to complete the acquisition of the sample data set of the oxidation zone. The xml file contains the coordinates of the marked real box: the top left and bottom right corners (minx, miny, maxx, maxy) and the labeled class name, in this example the labeled oxidation zone name up.
Data enhancement processing is performed on the data set, and four pictures are spliced through modes of random cutting, random turning, blurring and the like to be used as a training data set, as shown in fig. 2. Specifically, the Mosaic data enhancement processing can enrich the background of the steel oxidation zone, and can calculate four image data at one time during BN (batch normalization) calculation, so that the training efficiency is improved. The processed image is shown in fig. 2 and is realized by the following steps: step one, randomly reading four pictures in a prepared steel oxidation zone data set; secondly, respectively carrying out left-right turning, random scaling and color gamut change (including saturation, hue and brightness) operations on the picture; and thirdly, arranging the processed images according to the directions, wherein the image 1 is arranged at the upper left, the image 2 is arranged at the lower left, the image 3 is arranged at the lower right, and the image 4 is arranged at the upper right. And fourthly, cutting the fixed areas of the four images in a matrix mode, and splicing the fixed areas into one image again.
And S102, dividing the sample data set of the oxidation zone into a training set, a verification set and a test set according to a preset proportion.
In the embodiment, the data set in the S1 is divided into a training set, a verification set and a test set according to the ratio of 6: 2. In this example, 4240 sample images are summed, and the training set is used for the sample set for model fitting. The verification set is used for carrying out preliminary evaluation on the capability of the model; the generalization ability of the model was evaluated using the test set. Preparing folders according to a VOC format: JPEGImages store all the oxidation zone sample images; the Annotations store the annotation images to generate one-to-one corresponding xml files; and establishing a Mian folder under ImageSets to store train. the txt file stores the scaled image names. The content in the val.txt and the text.txt files is generated in the same way by randomly extracting 60% of 4240 images and storing the 60% in the train.txt in a disorder way.
According to the method, original anchor frame data of a Yolov4-tiny algorithm based on a Darknet frame are improved, the size of a prior frame is reset according to a real frame marked in a steel oxidation zone data set, in the embodiment, an optimized clustering algorithm is used for the real frame marked in the steel oxidation zone data set to obtain 6 anchor frame values, the prior frame is beneficial to predicting that the frame is closer to the real frame, and the value of a parameter file YOLOv4-tiny.cfg related to an anchor box in the cfg is modified to serve as an initial parameter before training. The 6 anchor box values are shown in figure 3. The embodiment of the present application provides an optional calculation method, in which the step of calculating the prior frame parameter includes:
firstly, traversing a data set, labeling real frame coordinates (minx, miny, maxx, maxy), and randomly initializing to select a cluster center, wherein the cluster centers are as far as possible. The cluster center is updated each iteration. The distance formula used in this embodiment is: d (box, anchor) ═ 1-IOU (box, anchor):
the intersection (box, anchor) is the intersection of the real frame and the prediction anchor frame, namely the coincidence area;
an intersection (box) is the union of the real box and the predicted anchor box, i.e., the sum of the areas.
And screening the selected clustering centers according to an optimized clustering algorithm until K clustering centers are selected. Calculating the distance D (box, anchor) between the sample point box marked by the data set and the clustering center anchor, storing the distance D (box, anchor) in an array, calculating the sum SUM (box) of the distances, removing a random value w between 0 and 1, and using SUM (box) w-r and r as the threshold value of the sum of the distances. Initializing n as 0, adding the values in the D (box) array to n one by one in sequence until n is larger than r, and selecting the current point as the cluster center. This embodiment requires clustering K6 centers.
S103, improving the network structure of the steel oxidation zone based on the original YOLOv4-tiny algorithm to obtain a steel oxidation zone target detection algorithm.
And selecting a Yolov4-tiny algorithm based on a Darknet frame for improvement, and adding an M-SPP structure in a CSPDarknet53-tiny structure of the feature extraction network part to obtain a steel oxidation zone target detection algorithm.
Referring to fig. 4, the improved algorithm network structure includes a backhaul feature extraction network, an M-SPP structure, a hack feature fusion module, and a Yolo Head prediction module. Wherein the Backbone is a CSPDarknet53-tiny network structure and outputs two feature layers with the sizes of 26 × 26 and 13 × 13; the M-SPP structure utilizes two different scales to carry out maximum pooling treatment; neck is a feature Pyramid FPN (feature Pyramid networks) to perform feature fusion on the two feature layers output in the last step; and predicting the obtained features by the Yolo Head, and outputting the prediction results of the two feature layers.
An M-SPP structure is added after the last DarknetConv2D _ BN _ Leaky (13,13, 512) convolution of the CSPDarknet-tiny network, as shown in figure 5, the maximum pooling of two different scales is utilized for processing, thus the receptive field can be greatly increased, and the most significant contextual features can be separated. The sizes of the pooling cores of the M-SPP are respectively 13 × 13 and 5 × 5, the pooling result is concat, the concat is transmitted to the FPN through one-time 1 × 1 convolution to perform double up-sampling, and feature fusion is performed with a second Resblock _ body (26,26,256) feature layer of the backbone network, wherein the transfer equation of the feedforward network is that
Wherein the content of the first and second substances,
represents a convolution operator;
n represents the output of the nth connection;
tn denotes the gradient of the nth connection.
Different from the direct up-sampling operation after convolution in the original algorithm, compared with the original algorithm, the parameter quantity is only increased by 1.5%, the edge information and the positioning information are strengthened, the data utilization rate is improved, and the problem that the detection precision of the steel oxidation band is not high is effectively solved.
Model compression is realized through a sparse training and channel pruning multi-stage iterative training mode. The network reasoning time is shortened, and the redundant consumption of the model on the computing power of the small embedded equipment is reduced. The invention realizes the sparse training of the network by using the BN (batch normalization) layer of the network, and restrains the channel with lower utilization rate to lead the weight to approach 0. Specifically, normalization processing is performed in the BN layer training process: inputting a batch: b ═ x1...batch_size}，xiA training set picture in batch is taken; obtaining weight data (gamma, beta) in a plurality of rounds of iterative training and learning; and (3) outputting:
the gamma coefficient is a quantitative basis for measuring the importance of the channel, and is influenced by a loss function according to whether the channel contains a target or not. The invention combines the weight sparsification with the target detection process, and proposes a new loss function: loss ═ Σ(x，y)l(x，y)+λ∑γL γ l, where (x, y) represent training data and labels, respectively; l (x, y) original loss function; | γ | is a penalty factor for the γ coefficient. The loss function not only maintains the optimization of the loss function, but also biases the weights toward the important channels. The sparse training punishs the unimportant channel coefficient gamma to be close to 0, the channels with smaller gamma coefficients are pruned according to the pruning rate, the parameters and the calculated amount are reduced through pruning, so that the network is more compact, and compared with the original network, the pruned network is reduced by 2.5%, so that the application performance of the network on the small embedded equipment is better.
And predicting the obtained features by using a Yolo Head, wherein the Yolo Head comprises a first classifier and a second classifier: the first classifier receives the fused features of which the output size is 13 × 13 from the feature fusion module. The second classifier receives the feature fusion module and outputs the fused feature with the size of 26 x 26. The output prediction result shape is (13,13, N), (19,19, N), where:
N＝S+Conf+Class，
s-4, representing position information of the prediction frame;
conf is 1, and represents confidence, i.e., class probability, i.e., confidence;
class 1 represents the number of classes, and the Class in this embodiment is 1.
Decoding the output prediction result, sorting the scores of each prediction frame, and screening non-maximum value inhibition to obtain the position information of the final prediction frame: and (5) predicting coordinates (bx, by) at the upper left corner of the frame and width and height (bw, bh) of the frame, and drawing on the original image.
Specifically, the prediction box confidence Conf is calculated by the following formula:
Pr(Classii Object) represents the probability that an Object belongs to a certain class;
Pr(Object) represents whether the current prediction box contains an Object;
Specifically, a non-maximum suppression step is performed: (1) firstly, acquiring all prediction frame information under the current target category; (2) sorting Conf of the prediction frames from top to bottom, and recording the prediction frame with the largest current Conf value; (3) calculating IOUs of the prediction boxes corresponding to the maximum Conf and the rest prediction boxes, and removing all the prediction boxes larger than the IOU threshold value; (4) and (3) performing the steps (2) and (3) on the residual prediction blocks in a loop until all the prediction blocks meet the requirements.
S104, training the optimized YOLOv4-tiny network by using a training set, and selecting a weight file with the optimal mAP.
Configuration selection the environment required for YOLOv4-tiny training based on the Darknet framework, using the ubuntu18.04 system, CUDA10.2, 0 pencv4.1.1. And setting model parameters. The values of batch 16 and subdivisions 8 are adjusted according to the calculation force condition of the small-sized embedded device. An input picture width and height value (416 ), a learning rate 0.00261, an iteration number 2000, and a category number 1 are set. The initial training uses the official pre-training weight file of YOLOv4-tiny, the weight file generated in the set training process is stored in the backup folder every 1000 rounds, in this embodiment, 5 weight files are stored, and the map (mean Average precision) is calculated.
In the embodiment, the optimal value of mAP reaches 98%. The mAP is an evaluation index for measuring the detection accuracy in the target detection network, and averages the value of AP (average precision) for a plurality of categories. This example category is only one class of oxidation bands, so the maps ═ AP:
and S105, integrating the trained network into a detection system, and detecting the steel oxidation zone image to obtain the position information of the oxidation zone.
According to the position information (bx, by, bw, bh) of the target frame, the center point position coordinates (cx, cy) of the target frame are obtained through standardization processing. Specifically, the center point position of the target frame is normalized:
cx＝bx+bw/2，cy＝by+bh/2
the detection system of small-size embedded equipment sends detection target position information to production line control equipment PLC through serial communication protocol and waits that the upload of order is given down, and the upload information of order includes (bx, by, bw, bh, cx, cy) and angle, and wherein the angle calculates:
angle＝P1P2*P1C/(|P1P2|*|P1C|))
the serial communication adopts a Modbus communication protocol, the baud rate is 9600, 8 bit data bits, 1 stop bit and no check bit. FIG. 6 is a schematic diagram showing the identification effect of the improved YOLOv4-tiny model on the steel oxidation zone.
Corresponding to the method for detecting the target of the steel oxidation zone in the industry provided by the above embodiment, the present application also provides an embodiment of a system for detecting the target of the steel oxidation zone in the industry, and referring to fig. 7, the system 20 for detecting the target of the steel oxidation zone in the industry includes: an acquisition module 201, a dataset processing module 202, a network improvement module 203, a training module 204, and a detection module 205.
The obtaining module 201 is configured to collect oxidation zone sample images of different steel types under different illumination intensities, and perform image preprocessing to obtain an initial oxidation zone sample data set. The data set processing module 202 is configured to divide the oxidized band sample data set into a training set, a verification set, and a test set according to a preset ratio. The network improvement module 203 is used for improving the network structure of the steel product based on the original YOLOv4-tiny algorithm to obtain a target detection algorithm of the steel product oxidation zone; the training module 204 is configured to train the optimized YOLOv4-tiny network by using a training set, and select a weight file with an optimal mAP. The detection module 205 is configured to integrate the trained network into a detection system, and detect an image of an oxidation zone of the steel material to obtain position information of the oxidation zone.
Embodiments of the present application also provide an apparatus, and referring to fig. 8, the apparatus 30 includes: a processor 301, a memory 302, and a communication interface 303.
In fig. 8, the processor 301, the memory 302, and the communication interface 303 may be connected to each other by a bus; the bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 8, but this is not intended to represent only one bus or type of bus.
The processor 301 generally controls the overall functions of the apparatus 30, such as starting the apparatus, controlling an industrial camera to perform image acquisition, processing images, training a network, and the like on the steel oxidation strip after the apparatus is started. Further, the processor 301 may be a general-purpose processor, such as a Central Processing Unit (CPU), a Network Processor (NP), or a combination of a CPU and an NP. The processor may also be a Microprocessor (MCU). The processor may also include a hardware chip. The hardware chips may be Application Specific Integrated Circuits (ASICs), Programmable Logic Devices (PLDs), or a combination thereof. The PLD may be a Complex Programmable Logic Device (CPLD), a Field Programmable Gate Array (FPGA), or the like.
After the device 30 is started, the processor 301 and the memory 302 are powered on, and the processor 301 reads and executes the computer executable instructions stored in the memory 302 to complete all or part of the steps in the above-mentioned embodiment of the steel material oxidation zone target detection method in the industry.
The communication interface 303 is used for the device 30 to transfer data, for example, to enable communication with an industrial camera and a line control device. The communication interface 303 includes a wired communication interface, and may also include a wireless communication interface. The wired communication interface comprises a USB interface, a Micro USB interface and an Ethernet interface. The wireless communication interface may be a WLAN interface, a cellular network communication interface, a combination thereof, or the like.
In an exemplary embodiment, the device 30 provided by embodiments of the present application further includes a power supply component that provides power to the various components of the device 30. The power components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for device 30.
A communications component configured to facilitate communications between device 30 and other devices in a wired or wireless manner. The device 30 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. The communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. The communication component also includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, device 30 may be implemented as one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), devices, micro-devices, processors, or other electronic components.
The same and similar parts among the various embodiments in the specification of the present application may be referred to each other. In particular, for the system and apparatus embodiments, since the method therein is substantially similar to the method embodiments, the description is relatively simple, and reference may be made to the description in the method embodiments for relevant points.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Of course, the above description is not limited to the above examples, and technical features that are not described in this application may be implemented by or using the prior art, and are not described herein again; the above embodiments and drawings are only for illustrating the technical solutions of the present application and not for limiting the present application, and the present application is only described in detail with reference to the preferred embodiments instead, it should be understood by those skilled in the art that changes, modifications, additions or substitutions within the spirit and scope of the present application may be made by those skilled in the art without departing from the spirit of the present application, and the scope of the claims of the present application should also be covered.
Claims (10)
1. A method for detecting a steel oxidation zone target in industry is characterized by comprising the following steps:
acquiring oxidation zone sample images of different steel types under different illumination intensities, and performing image preprocessing to obtain an initial oxidation zone sample data set;
dividing the sample data set of the oxidation zone into a training set, a verification set and a test set according to a preset proportion;
improving the network structure of the steel based on the original YOLOv4-tiny algorithm to obtain a steel oxidation zone target detection algorithm;
training the optimized YOLOv4-tiny network by using a training set, and selecting a weight file with the optimal mAP;
and integrating the trained network in a detection system, and detecting the steel oxidation zone image to obtain the position information of the oxidation zone.
2. The method of claim 1, wherein acquiring images of oxidation zone samples of different steel types at different illumination intensities, and performing image preprocessing to obtain an initial oxidation zone sample set comprises:
arranging an industrial grade high-definition camera to shoot different types of steel oxidation zones under different illumination in real time on an operation table;
generating a picture for the collected video reading frame, and renaming the picture according to the sequence number to form a unique ID of the picture;
marking the position of the oxidation zone of the steel by using marking software to form a corresponding xml file and form an initial oxidation zone sample data set;
and carrying out data enhancement processing on the data set, and splicing the four pictures by means of random cutting, random turning, blurring and the like to obtain training data.
3. The method of claim 2, wherein the step of dividing the set of oxidation band sample data into a training set, a verification set and a test set according to a preset proportion comprises:
dividing a training set, a verification set and a test set according to a ratio of 6:2:2, wherein the training set is used for model training, the verification set is used for evaluating a model, and the test set is used for evaluating the generalization ability of the model;
preparing folders according to a VOC format: JPEGImages store all the oxidation zone sample images; the Annotations store the annotation images to generate one-to-one corresponding xml files; and establishing a Mian folder under ImageSets to store train.
4. The method of claim 1, wherein the original YOLOv4-tiny algorithm-based network structure comprises a backhaul feature extraction network, a Neck feature fusion module and a Yolo Head prediction module, wherein the backhaul feature extraction network is a CSPDarknet53-tiny network structure, and outputs two feature layers with the size of 26 x 26 and 13 x 13; the Neck feature fusion module performs feature fusion on the two feature layers output in the last step for a feature pyramid FPN; and the Yolo Head prediction module performs regression prediction on the obtained features and outputs prediction results of two feature layers.
5. The method as claimed in claim 4, wherein the network structure is improved based on the original YOLOv4-tiny algorithm to obtain the steel oxidation zone target detection algorithm, which comprises:
adding an M-SPP structure after the last characteristic layer of the CSPDarknet-tiny network, and processing by utilizing two maximum pooling with different scales with extremely small calculated amount;
model compression is realized in a mode of sparse training and multi-stage iterative training of channel pruning;
stacking and outputting the stacked objects after being pooled to an FPN convolution, then performing double up-sampling, and performing feature fusion with a second Resblock _ body (26, 256) feature layer of the backbone network;
predicting the obtained features by a Yolo Head prediction module, wherein the Yolo Head prediction module comprises a first classifier and a second classifier, and the first classifier receives the fusion features with the output size of 13 × 13 output by the feature fusion module; the second classifier receives the fused features with the output size of 26 x 26 from the feature fusion module; the output prediction results shape are (13,13, N), (19,19, N), where N ═ S + Conf + Class, and S ═ 4, represent the position information of the prediction frame, Conf ═ 1 represents the confidence level, i.e., the Class probability, and Class ═ the number of classes, respectively;
and decoding according to the output prediction result to obtain the position information of the prediction frame, wherein the position information comprises the coordinates (bx, by) at the upper left corner of the prediction frame and the width (bw, bh) of the prediction frame.
6. The method of claim 1, wherein training the optimized YOLOv4-tiny network with a training set to select a weight file with an optimal mAP comprises:
configuring an operating environment, namely an Ubuntu system, a CUDA10.2 and an OpenCV4.1.1, and selecting YOLOv4-tiny based on a Darknet framework for training;
setting model parameters, adjusting values of batch and subdivisions according to the calculation force condition of the small embedded equipment, and setting the width and height values, the learning rate, the iteration times and the category number of input pictures;
the initial training uses the official pre-training weight file of YOLOv4-tiny, and the weight file generated during the setup training is stored every 1000 rounds.
7. The method of claim 5, wherein the trained network is integrated into a detection system, and the detecting the steel oxidation band image to obtain the position information of the oxidation band comprises:
according to the position information (bx, by, bw, bh) of the target frame, carrying out standardization processing to obtain a central point position coordinate (cx, cy) of the target frame;
and the detection system of the small embedded equipment is utilized to send the detection target position information to the production line control equipment through a serial port communication protocol for subsequent operation.
8. The method of any one of claims 1 to 7, further comprising resetting the prior box size based on a true box labeled in the steel oxidation band dataset, comprising: and aiming at the real frame marked in the steel oxidation zone data set, obtaining a plurality of anchor frame values by using a clustering algorithm, and modifying the value of an anchor box in the parameter file as an initial parameter before training.
9. A steel oxidation zone target detection system in industry, characterized in that the system comprises:
the acquisition module is used for acquiring oxidation zone sample images of different steel types under different illumination intensities and carrying out image preprocessing to obtain an initial oxidation zone sample data set;
the data set processing module is used for dividing the sample data set of the oxidation zone into a training set, a verification set and a test set according to a preset proportion;
the network improvement module is used for improving the network structure of the steel oxidation zone based on the original YOLOv4-tiny algorithm to obtain a target detection algorithm of the steel oxidation zone;
the training module is used for training the optimized YOLOv4-tiny network by utilizing a training set and selecting a weight file with the optimal mAP;
and the detection module is used for integrating the trained network into a detection system and detecting the steel oxidation zone image to obtain the position information of the oxidation zone.
10. An apparatus, comprising:
a processor;
a memory for storing computer executable instructions
When the processor executes the computer-executable instructions, the processor executes the method for detecting a steel oxidation zone target in the industry according to any one of claims 1 to 8 to identify the steel oxidation zone.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110337542.8A CN112926685A (en) | 2021-03-30 | 2021-03-30 | Industrial steel oxidation zone target detection method, system and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110337542.8A CN112926685A (en) | 2021-03-30 | 2021-03-30 | Industrial steel oxidation zone target detection method, system and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112926685A true CN112926685A (en) | 2021-06-08 |
Family
ID=76176463
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110337542.8A Pending CN112926685A (en) | 2021-03-30 | 2021-03-30 | Industrial steel oxidation zone target detection method, system and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112926685A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113378753A (en) * | 2021-06-23 | 2021-09-10 | 华南农业大学 | Improved YOLOv 4-based boundary target identification method for rice field in seedling stage |
| CN113379727A (en) * | 2021-07-01 | 2021-09-10 | 南昌工程学院 | Kiwi fruit foliar disease detection method based on improved YOLOv4-Tiny characteristic fusion |
| CN113592825A (en) * | 2021-08-02 | 2021-11-02 | 安徽理工大学 | YOLO algorithm-based real-time coal gangue detection method |
| CN113781412A (en) * | 2021-08-25 | 2021-12-10 | 南京航空航天大学 | Chip redundancy detection system and method under X-ray high-resolution scanning image based on deep learning |
| CN114037667A (en) * | 2021-10-28 | 2022-02-11 | 北京科技大学 | Machine vision-based copper surface oxidation detection and oxidation degree judgment method |
| CN114066848A (en) * | 2021-11-16 | 2022-02-18 | 苏州俪濠智能科技有限公司 | FPCA appearance defect visual inspection system |
| CN114299366A (en) * | 2022-03-10 | 2022-04-08 | 青岛海尔工业智能研究院有限公司 | Image detection method and device, electronic equipment and storage medium |
| CN115100492A (en) * | 2022-08-26 | 2022-09-23 | 摩尔线程智能科技(北京)有限责任公司 | Yolov3 network training and PCB surface defect detection method and device |
| CN115641518A (en) * | 2022-10-09 | 2023-01-24 | 山东巍然智能科技有限公司 | View sensing network model for unmanned aerial vehicle and target detection method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111062915A (en) * | 2019-12-03 | 2020-04-24 | 浙江工业大学 | Real-time steel pipe defect detection method based on improved YOLOv3 model |
| CN112347943A (en) * | 2020-11-09 | 2021-02-09 | 哈尔滨理工大学 | Anchor optimization safety helmet detection method based on YOLOV4 |
| CN112464910A (en) * | 2020-12-18 | 2021-03-09 | 杭州电子科技大学 | Traffic sign identification method based on YOLO v4-tiny |
| CN112561885A (en) * | 2020-12-17 | 2021-03-26 | 中国矿业大学 | YOLOv 4-tiny-based gate valve opening detection method |
-
2021
- 2021-03-30 CN CN202110337542.8A patent/CN112926685A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111062915A (en) * | 2019-12-03 | 2020-04-24 | 浙江工业大学 | Real-time steel pipe defect detection method based on improved YOLOv3 model |
| CN112347943A (en) * | 2020-11-09 | 2021-02-09 | 哈尔滨理工大学 | Anchor optimization safety helmet detection method based on YOLOV4 |
| CN112561885A (en) * | 2020-12-17 | 2021-03-26 | 中国矿业大学 | YOLOv 4-tiny-based gate valve opening detection method |
| CN112464910A (en) * | 2020-12-18 | 2021-03-09 | 杭州电子科技大学 | Traffic sign identification method based on YOLO v4-tiny |
Non-Patent Citations (1)
| Title |
|---|
| 明悦: "《多源视觉信息感知与识别》", 北京邮电大学出版社, pages: 124 - 126 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113378753A (en) * | 2021-06-23 | 2021-09-10 | 华南农业大学 | Improved YOLOv 4-based boundary target identification method for rice field in seedling stage |
| CN113379727A (en) * | 2021-07-01 | 2021-09-10 | 南昌工程学院 | Kiwi fruit foliar disease detection method based on improved YOLOv4-Tiny characteristic fusion |
| CN113592825A (en) * | 2021-08-02 | 2021-11-02 | 安徽理工大学 | YOLO algorithm-based real-time coal gangue detection method |
| CN113781412A (en) * | 2021-08-25 | 2021-12-10 | 南京航空航天大学 | Chip redundancy detection system and method under X-ray high-resolution scanning image based on deep learning |
| CN114037667A (en) * | 2021-10-28 | 2022-02-11 | 北京科技大学 | Machine vision-based copper surface oxidation detection and oxidation degree judgment method |
| CN114066848A (en) * | 2021-11-16 | 2022-02-18 | 苏州俪濠智能科技有限公司 | FPCA appearance defect visual inspection system |
| CN114066848B (en) * | 2021-11-16 | 2024-03-22 | 苏州极速光学科技有限公司 | FPCA appearance defect visual detection system |
| CN114299366A (en) * | 2022-03-10 | 2022-04-08 | 青岛海尔工业智能研究院有限公司 | Image detection method and device, electronic equipment and storage medium |
| CN115100492A (en) * | 2022-08-26 | 2022-09-23 | 摩尔线程智能科技(北京)有限责任公司 | Yolov3 network training and PCB surface defect detection method and device |
| CN115100492B (en) * | 2022-08-26 | 2023-04-07 | 摩尔线程智能科技(北京)有限责任公司 | Yolov3 network training and PCB surface defect detection method and device |
| CN115641518A (en) * | 2022-10-09 | 2023-01-24 | 山东巍然智能科技有限公司 | View sensing network model for unmanned aerial vehicle and target detection method |
| CN115641518B (en) * | 2022-10-09 | 2023-09-26 | 山东巍然智能科技有限公司 | View perception network model for unmanned aerial vehicle and target detection method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112926685A (en) | Industrial steel oxidation zone target detection method, system and equipment | |
| CN108764372B (en) | Construction method and device, mobile terminal, the readable storage medium storing program for executing of data set | |
| CN108446697B (en) | Image processing method, electronic device and storage medium | |
| US11087447B2 (en) | Systems and methods for quality assurance of image recognition model | |
| CN108921057B (en) | Convolutional neural network-based prawn form measuring method, medium, terminal equipment and device | |
| CN108875821A (en) | The training method and device of disaggregated model, mobile terminal, readable storage medium storing program for executing | |
| CN108921166A (en) | Medical bill class text detection recognition method and system based on deep neural network | |
| CN109886928B (en) | Target cell marking method, device, storage medium and terminal equipment | |
| CN109101934A (en) | Model recognizing method, device and computer readable storage medium | |
| US11776257B2 (en) | Systems and methods for enhancing real-time image recognition | |
| CN111401418A (en) | Employee dressing specification detection method based on improved Faster r-cnn | |
| CN108182421A (en) | Methods of video segmentation and device | |
| CN110737785B (en) | Picture labeling method and device | |
| WO2019218459A1 (en) | Photo storage method, storage medium, server, and apparatus | |
| CN113033516A (en) | Object identification statistical method and device, electronic equipment and storage medium | |
| CN111985477A (en) | Monocular camera-based animal body online claims checking method and device and storage medium | |
| CN111597932A (en) | Road crack image identification method, device and system based on convolutional neural network | |
| CN111259710B (en) | Parking space structure detection model training method adopting parking space frame lines and end points | |
| CN109840559A (en) | Method for screening images, device and electronic equipment | |
| CN115578616A (en) | Training method, segmentation method and device of multi-scale object instance segmentation model | |
| CN112560845A (en) | Character recognition method and device, intelligent meal taking cabinet, electronic equipment and storage medium | |
| CN113850799A (en) | YOLOv 5-based trace DNA extraction workstation workpiece detection method | |
| CN115147862A (en) | Benthonic animal automatic identification method, system, electronic device and readable storage medium | |
| CN115082659A (en) | Image annotation method and device, electronic equipment and storage medium | |
| CN117351472A (en) | Tobacco leaf information detection method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |