CN111784670A

CN111784670A - Hot rolled steel plate surface defect identification method and device based on computer vision

Info

Publication number: CN111784670A
Application number: CN202010620520.8A
Authority: CN
Inventors: 张玉琪
Original assignee: Ping An International Smart City Technology Co Ltd
Current assignee: Shenzhen Saiante Technology Service Co Ltd
Priority date: 2020-06-30
Filing date: 2020-06-30
Publication date: 2020-10-16
Anticipated expiration: 2040-06-30
Also published as: CN111784670B

Abstract

The invention discloses a hot rolled steel plate surface defect identification method based on computer vision, a device, computer equipment and a storage medium, and relates to artificial intelligence image detection and industrial internet. The method realizes the automatic identification of the surface defects of the hot rolled steel plate surface picture, and simultaneously combines a convolutional neural network and a multi-scale target detection model, thereby not only ensuring the rapidity of the identification process, but also ensuring the accuracy of the identification result.

Description

Hot rolled steel plate surface defect identification method and device based on computer vision

Technical Field

The invention relates to the technical field of artificial intelligence image detection, in particular to a hot rolled steel plate surface defect identification method and device based on computer vision, computer equipment and a storage medium.

Background

The hot rolled steel plate is a product which is prepared by taking a continuous casting plate blank or a primary rolling plate blank as a raw material, heating by a stepping heating furnace, descaling, passing through a roughing mill, a finishing mill, laminar cooling, coiling by a coiling machine, and finally cutting the end and the tail, straightening, flattening, cutting the plate or rewinding. The hot rolled steel plate has higher forming temperature, so that the forming requirement is lower than that of cold rolling forming, but the formed plate has more defects. The quality of the product is directly influenced by the number of the defects of the plate, so that the real-time monitoring of the quality of the steel plate on an actual production line is very important.

At present, most steel mills still adopt a method for identifying the surface defects of the hot rolled steel plate by naked eyes, so that the requirements on the professional technology of quality inspection personnel are high, the labor cost for identifying the surface defects of the hot rolled steel plate is high, and the identification efficiency is low.

Disclosure of Invention

The embodiment of the invention provides a hot rolled steel plate surface defect identification method and device based on computer vision, computer equipment and a storage medium, and aims to solve the problems that in the prior art, the labor cost for identifying the surface defects of a hot rolled steel plate is high and the identification efficiency is low due to the fact that the surface defects of the hot rolled steel plate are identified by naked eyes manually on a production line of a steel mill.

In a first aspect, an embodiment of the present invention provides a hot-rolled steel sheet surface defect identification method based on computer vision, which includes:

receiving a current frame picture of the surface of the hot rolled steel plate uploaded by an image acquisition terminal;

cutting and combining the current frame picture according to a preset picture cutting pixel size to obtain a corresponding combined batch image set;

calling a pre-trained convolutional neural network, and inputting a pixel matrix corresponding to the synthesized batch image set into the convolutional neural network to obtain a corresponding image classification result set; wherein the total number of image classification results included in the image classification result set is equal to the total number of cropped pictures included in the composite batch image set;

judging whether the image classification result exists in the image classification result set or not as a defect classification result;

if the image classification result exists in the image classification result set and is a defect classification result, calling a pre-trained multi-scale target detection model, inputting a pixel matrix corresponding to each cut picture in the synthesis batch image set to the multi-scale target detection model, and obtaining a fine screening classification result corresponding to each cut picture in the synthesis batch image set to form a fine screening classification result set; wherein, the fine screening classification result comprises a defective classification result and a non-defective classification result;

if the fine screening classification result is a defective classification result, acquiring a fine screening target image corresponding to the fine screening classification result as the defective classification result to form a fine screening target image set;

acquiring the size of a defect area in each fine screening target image in the fine screening target image set to obtain a defect type corresponding to each fine screening target image; and

and sending the fine screening target image set corresponding to the current frame picture and the defect types corresponding to the fine screening target images to a monitoring terminal corresponding to the image acquisition terminal.

In a second aspect, an embodiment of the present invention provides a device for identifying surface defects of a hot rolled steel sheet based on computer vision, including:

the current frame picture receiving unit is used for receiving a current frame picture on the surface of the hot rolled steel plate uploaded by the image acquisition terminal;

the picture cutting and synthesizing unit is used for cutting and synthesizing the current frame picture according to the preset picture cutting pixel size to obtain a corresponding synthesized batch image set;

the fast identification unit is used for calling a pre-trained convolutional neural network, inputting a pixel matrix corresponding to the synthesis batch image set into the convolutional neural network, and obtaining a corresponding image classification result set; wherein the total number of image classification results included in the image classification result set is equal to the total number of cropped pictures included in the composite batch image set;

a classification result judging unit, configured to judge whether the image classification result exists in the image classification result set as a defect classification result;

the fine screening unit is used for calling a pre-trained multi-scale target detection model if the image classification result exists in the image classification result set and is a defect classification result, inputting the pixel matrix corresponding to each cut picture in the synthesis batch image set to the multi-scale target detection model, and obtaining the fine screening classification result corresponding to each cut picture in the synthesis batch image set to form a fine screening classification result set; wherein, the fine screening classification result comprises a defective classification result and a non-defective classification result;

the fine screening target image set acquiring unit is used for acquiring a fine screening target image corresponding to the fine screening classification result as a defective classification result if the fine screening classification result is a defective classification result in the fine screening classification result so as to form a fine screening target image set;

the defect type obtaining unit is used for obtaining the size of a defect area in each fine screening target image in the fine screening target image set so as to obtain the defect type corresponding to each fine screening target image; and

and the fine screening result sending unit is used for sending the fine screening target image set corresponding to the current frame picture and the defect types corresponding to the fine screening target images to the monitoring terminal corresponding to the image acquisition terminal.

In a third aspect, an embodiment of the present invention further provides a computer apparatus, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the method for identifying defects on a surface of a hot-rolled steel sheet based on computer vision according to the first aspect.

In a fourth aspect, the embodiments of the present invention further provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, causes the processor to execute the method for identifying defects on a surface of a hot-rolled steel sheet based on computer vision according to the first aspect.

The embodiment of the invention provides a hot rolled steel plate surface defect identification method, a hot rolled steel plate surface defect identification device, computer equipment and a storage medium based on computer vision, which take the characteristics of multiple types of defects, large defect size span and high-speed movement of a production line in the actual production process into consideration, provide a hot rolled steel plate real-time surface defect detection and identification scheme combined with computer vision, can finish automatic detection, identification and alarm of steel plate defects on the production line moving quickly, and simultaneously combine a convolutional neural network and a multi-scale target detection model, thereby not only ensuring the rapidity of the identification process, but also ensuring the accuracy of the identification result.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic view of an application scenario of a hot rolled steel sheet surface defect identification method based on computer vision provided in an embodiment of the present invention;

FIG. 2 is a schematic flow chart of a hot-rolled steel sheet surface defect identification method based on computer vision according to an embodiment of the present invention;

FIG. 3 is a schematic block diagram of a hot rolled steel sheet surface defect identification device based on computer vision provided by an embodiment of the invention;

FIG. 4 is a schematic block diagram of a computer device provided by an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

Referring to fig. 1 and fig. 2, fig. 1 is a schematic view of an application scenario of a hot rolled steel sheet surface defect identification method based on computer vision according to an embodiment of the present invention; fig. 2 is a schematic flow chart of a hot rolled steel sheet surface defect identification method based on computer vision according to an embodiment of the present invention, where the hot rolled steel sheet surface defect identification method based on computer vision is applied to a server, and the method is executed by application software installed in the server.

As shown in fig. 2, the method includes steps S110 to S180.

S110, receiving a current frame picture of the surface of the hot rolled steel plate uploaded by the image acquisition terminal.

In the embodiment, the specific application scenario relates to the field of industrial internet, and is to detect the surface defects of hot rolled steel plates on an industrial production line of a steel mill. The terminal that involves in this application has server and image acquisition terminal (when specifically implementing, image acquisition terminal adopts a plurality of resolution ratio to be 4K's 4K camera, or a plurality of resolution ratio to be 4K's 4K camera), wherein image acquisition terminal is used for gathering the surface picture of hot-rolled steel plate on the production line, and the server calls the convolutional neural network of training in advance and the multi-scale target detection model to whether each subregion has the defect in the discernment surface picture, thereby realizes to hot-rolled steel plate real-time surface defect detection and discernment.

The method comprises the following steps of taking the width of a hot-rolled steel plate as a reference, if the width of the hot-rolled steel plate is 0.6m, horizontally placing a 4K camera right above an industrial production line, wherein the target surface of the 4K camera is parallel to the steel plate plane of the hot-rolled steel plate, and adjusting the height of the camera according to the focal length of the camera to enable the imaging length on the picture of the 4K camera to be equal to the width of the hot-rolled steel plate; if the width of the hot-rolled steel sheet is 3.6m, 6 4K cameras need to be arranged in parallel at equal intervals in the width direction right above the production line. Considering that the moving speed of the hot rolled steel sheet is 5m/s, if the frame rate of the 4K camera is fixedly set to 20 frames, one snapshot can be taken for each 0.25m moving average of the steel sheet. For a 4K camera, the picture size is 4096 × 2160, and each pixel size corresponds to the real world steel plate size of 0.6m/4096 ≈ 0.15mm, i.e. the number of pixels corresponding to a defect of 10mmx10mm is 4660 pixels, even if the defect of 1mmx1mm can have 46 pixels, which can satisfy the minimum requirement for recognition.

In an embodiment, step S110 further includes:

receiving a unique code of current equipment uploaded by an image acquisition terminal;

calling a pre-stored device white list;

and if the current equipment unique code exists in the equipment white list, establishing communication connection with the image acquisition terminal.

In this embodiment, since a large number of image capturing terminals can be deployed on an industrial production line, a current frame picture on each production line can be acquired, at this time, in order to clearly know which image capturing terminal uploads the current frame picture, the server needs to verify the device information, and the specific process is as follows: firstly, receiving a unique code of current equipment uploaded by an image acquisition terminal; then, calling a pre-stored device white list; and finally, if the current equipment unique code exists in the equipment white list, establishing communication connection with the image acquisition terminal to receive the current frame picture uploaded by the image acquisition terminal.

Through the equipment identity verification process, the data uploaded by the legal image acquisition terminal registered in the server is ensured, and the uploaded data can be released at the moment.

And S120, cutting and combining the current frame picture according to a preset picture cutting pixel size to obtain a corresponding combined batch image set.

In this embodiment, in order to improve the image recognition speed and accuracy, the current frame picture may be cut and combined in batches according to a preset picture cutting pixel size, so as to obtain a corresponding combined batch image set. By the processing mode, one frame of current frame picture is divided into a plurality of pictures and then combined in batches, so that whether a defect area exists in the pictures or not can be rapidly identified in a regional mode.

In one embodiment, step S120 includes:

calling a preset picture cutting pixel size, and cutting the current frame picture into a plurality of cutting pictures according to the picture cutting pixel size;

respectively adding timestamps to the plurality of cut pictures to correspondingly obtain a plurality of time-stamped pictures;

and synthesizing the plurality of time-stamped pictures in batch to obtain a corresponding synthesized batch image set.

In this embodiment, after a current frame picture of a hot-rolled steel plate to be detected on a production line is acquired and acquired by an image acquisition device and uploaded to a server, the server cuts the current frame picture into a plurality of cut pictures according to the picture cutting pixel size.

In an embodiment, the step of calling a preset picture clipping pixel size and clipping the current frame picture into a plurality of clipping pictures according to the picture clipping pixel size includes:

acquiring the actual pixel size of the current frame picture;

dividing the actual pixel size by the picture cutting pixel size to obtain the number of target cutting pictures;

and according to the number of the target cutting pictures and the size of the picture cutting pixels, cutting the current frame picture into cutting pictures with corresponding numbers.

In this embodiment, for example, after 1 frame picture is captured by the 4K camera, a 4096 × 2160 picture is cropped into 128 300 × 300 pictures and time stamped. Since the picture size is 300 × 300 by cutting one picture every 256 pixels in the picture length direction and one picture every 270 pixels in the picture height direction, 4096/256 × 2160/270-16 × 8-128 pictures of 300 × 300 are obtained.

Since a picture is captured every 256 pixels in the picture length direction and every 270 pixels in the picture height direction of the current frame picture, a sub-region composed of 256x270 pixel points in the captured picture is recorded as an initial sub-region, the final picture is 300x300 in size, a sub-region composed of 300x300 pixel points is recorded as a final sub-region, and if the final sub-region is regarded as a full set region and the initial sub-region is regarded as a subset region, a plurality of 0 s are filled in the complementary set region of the subset region (wherein the subset region and the complementary set region are combined to obtain the full set region), so that the size of 300x300 is achieved.

For a deep learning network with a GPU as a computing device, a lot of images are combined and processed (batch) (the common understanding of image combination batch processing is that a lot of pictures are processed at a time, and a batch processing file has a suffix name of bat, and the lots of pictures are not combined into one picture but processed at the same time) together, the processing speed is much faster than that of a single image in sequence, so that 128 cut pictures are selected to be combined and processed (batch) and each cut picture is added with a timestamp.

In an embodiment, the step of adding timestamps to the plurality of cropped pictures respectively to obtain a plurality of time-stamped pictures correspondingly includes:

and acquiring a timestamp corresponding to the current frame picture, adding the timestamp to the plurality of cutting pictures, and correspondingly acquiring a plurality of time-stamped pictures.

When a timestamp is added to each of the cropped pictures, since the cropped pictures are obtained by cropping from the same current frame picture, the same timestamp as the current frame picture can be automatically added to the lower left corner of the cropped pictures (the timestamp can be understood as the collection date and the collection time added to the lower right corner of the picture).

S130, calling a pre-trained convolutional neural network, and inputting a pixel matrix corresponding to the synthesized batch image set into the convolutional neural network to obtain a corresponding image classification result set; wherein the total number of image classification results included in the image classification result set is equal to the total number of cropped pictures included in the composite batch image set.

In this embodiment, the server may obtain relevant model parameters of the convolutional neural network from the block chain network, so as to obtain a local pre-trained convolutional neural network of the server, where the convolutional neural network is used to identify whether the surface of the hot-rolled steel plate has a defective convolutional neural network, and the convolutional neural network realizes a rapid primary screening of whether the surface of the hot-rolled steel plate has a defect.

Because each cropped picture in the composite batch image set corresponds to a pixel matrix (for example, one composite batch image set includes 128 cropped pictures, each cropped picture corresponds to a size of [300 × 300], that is, each cropped picture corresponds to a 300 × 300 pixel matrix), a set formed by connecting the pixel matrices in series (a 38400 × 300 pixel matrix is formed by connecting the pixel matrices in series in the longitudinal direction), that is, the set can be used as an input of the convolutional neural network to perform calculation, so as to obtain an image classification result set corresponding to the composite batch image set. The results in this set of image classification results are either "defective" or "non-defective".

In an embodiment, step S130 further includes:

receiving a first model parameter set corresponding to the convolutional neural network acquired from the blockchain network so as to locally generate the convolutional neural network; wherein the convolutional neural network is an EfficientNet-b0 network.

In this embodiment, the server may serve as a block chain node device to upload the first model parameter set corresponding to the convolutional neural network to the block chain network, and the data solidification storage is implemented by fully utilizing the characteristic that the block chain data cannot be tampered. Moreover, the server can download the first model parameter set corresponding to the convolutional neural network from the blockchain so as to locally generate the convolutional neural network. In specific implementation, the convolutional neural network is an EfficientNet-b0 network.

The corresponding digest information is obtained based on the first model parameter set, and specifically, the digest information is obtained by hashing the first model parameter set, for example, by using a sha256 algorithm. Uploading summary information to the blockchain can ensure the safety and the fair transparency of the user. The server may download the summary information from the blockchain to verify whether the first set of model parameters is tampered with. The blockchain referred to in this example is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm, and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.

The convolutional neural network in the application specifically adopts an EfficientNet network with a relatively good effect at present (more specifically, the EfficientNet-b0 network is a convolutional neural network model provided by Google). Tests show that the inference time of the EfficientNet-b0 network on a GPU of GeForceRTX2080Ti is less than 10 ms. The defect-free determination of the pooled image set ensures real-time performance. When any one of the 128 captured small pictures is classified as defective, the current 4096x2160 picture is saved together with the time stamp, and a preliminary warning of defective is given.

S140, judging whether the image classification result exists in the image classification result set or not as a defect classification result.

In this embodiment, when the pixel matrix corresponding to the synthesized batch image set is input to the EfficientNet-b0 network, an image classification result set is obtained quickly, and at this time, it is necessary to determine whether an image classification result exists in the image classification result set as a defect classification result, so as to determine whether an image classification result exists in the image classification result set as a defect classification result through a further fine recognition model in the subsequent step.

S150, if the image classification result set has defect classification results, calling a pre-trained multi-scale target detection model, inputting the pixel matrix corresponding to each cut picture in the synthesis batch image set into the multi-scale target detection model, and obtaining fine screening classification results corresponding to each cut picture in the synthesis batch image set to form a fine screening classification result set; wherein, the fine screening classification result comprises a defective classification result and a non-defective classification result.

In this embodiment, for the "defective" pictures that are early-warned in step S130, the synthesized batch of image sets is inferred by an image detection network with a multi-scale detection function, specifically, by using an auto focus (also called auto focus) method and a training method of a SNIPER network. The detection network with multi-scale detection function used here has a long inference time, but since only the suspected defective pictures after preliminary screening in step S130 need to be detected, the inference time can still meet the actual requirement. The multi-scale detection model adopted in the method has high accuracy, so that the aim of accurate fine screening can be fulfilled. Among them, the SNIPER network is a multi-scale (multi-scale) training algorithm that combines the advantages of RCNN (fully called Region-CNN, an algorithm that applies deep learning to target detection) in scale processing and the advantages of fastern in speed.

In an embodiment, step S150 further includes:

receiving a second model parameter set corresponding to a multi-scale target detection model acquired from a blockchain network to locally generate the multi-scale target detection model; wherein the multi-scale target detection model is a SNIPER network.

In this embodiment, the server may serve as a block chain link point device to upload the second model parameter set corresponding to the multi-scale target detection model to the block chain network, and fully utilize the property that the block chain data cannot be tampered with, so as to implement data solidification storage. Moreover, the server may download a second set of model parameters corresponding to the multi-scale object detection model from the blockchain to locally generate the multi-scale object detection model. In a specific implementation, the multi-scale target detection module is a SNIPER network.

And S160, if the fine screening classification result is a defect classification result, acquiring a fine screening target image corresponding to the fine screening classification result as the defect classification result to form a fine screening target image set.

In this embodiment, when the fine screening classification result is a defect classification result in the fine screening classification result, which indicates that a certain region of the hot-rolled steel plate has a defect, a fine screening target image corresponding to the defect is obtained at this time, so as to form a fine screening target image set. Because each fine screening target image correspondingly comprises information such as a time stamp, a fine screening target image set can be formed by combining the time stamp and the screening target image.

S170, acquiring the size of a defect area in each fine screening target image in the fine screening target image set to obtain the defect type corresponding to each fine screening target image.

In this example, the defect size of the irregular surface inclusions on the surface of the hot-rolled steel sheet was 100 mm; the scale of the inclusion defect on the belt-shaped surface is 100 mm; the defect size of the bubbles was 50 mm; the defect size of the scab (heavy skin) is 50 mm; the defect size of the delamination was 10 mm; the defect size of the warping skin is 50 mm; the defect size of the flying wing is 10 mm; the defect size of edge crack is 10 mm; the size of the defect of edge overburning is 5 mm; the defect size of the edge crack is 5 mm; defect size of the air holes is <1 mm; the pressed defect size of the iron scale is 5 mm; the pressed defect size of the powdery iron oxide scale is 20 mm; the defect scale of red rust is 100 mm; the defect size of the pits is 100 mm; the defect size of the roll mark is 10 mm; the defect size of the indentation is 20 mm; the flaw size of the longitudinal cracks is 20 mm; the defect size of the transverse crack is 40 mm; the defect size of the cracks was 50 mm; the defect size of the M-shaped defect is 50 mm; the defect size of the penetration crack is 10 mm; the defect size of the fold is 50 mm; the flaw scale of the scratch, scratch damage is >100 mm.

The defect area of each fine screening picture in the fine screening target image set can be accurately positioned, and the size of the defect area can be calculated by acquiring the pixel point area corresponding to the defect area of each fine screening picture. For example, the number of pixels corresponding to a defect of 10mmx10mm in the example picture is 4660 pixels, and there may be 46 pixels even if there is a defect of 1mmx1 mm.

When the sizes corresponding to various defect types are known and the sizes of the defect areas in the fine screening target images in the fine screening target image set are known, the defect types corresponding to the fine screening target images can be obtained. Since different types of defect scales may correspond to the same defect size, for example, the defect scales of air bubbles and scabs (re-skinning) are all 50mm, when the size of a defect area in a fine screening target image is 50mm, the defect type corresponding to the fine screening target image is air bubbles or scabs (re-skinning).

And S180, sending the fine screening target image set corresponding to the current frame image and the defect types corresponding to the fine screening target images to a monitoring terminal corresponding to the image acquisition terminal.

In this embodiment, after the fine screening target image set corresponding to the current frame picture and the defect types corresponding to the fine screening target images are acquired, the detected defect types and defect positions can be displayed on the accurate fine screening page of the monitoring terminal together with the timestamp, and the detected defect types and defect positions can be stored in the local monitoring terminal, so that the source tracing can be performed later.

In an embodiment, step S140 is followed by:

and if the image classification result set does not have the defect classification result, sending the notification information for obtaining the next frame of picture on the surface of the hot rolled steel plate to an image acquisition terminal.

In this embodiment, if the image classification result does not exist in the image classification result set, the current frame picture on the surface of the hot-rolled steel plate uploaded by the image acquisition terminal and received by the server is defect-free, and the next frame picture on the surface of the hot-rolled steel plate can be continuously received for continuous identification. At the moment, the notification information for obtaining the next frame of picture on the surface of the hot rolled steel plate is sent to the image acquisition terminal.

The method realizes the automatic identification of the surface defects of the current frame picture, and simultaneously combines the convolutional neural network and the multi-scale target detection model, thereby not only ensuring the rapidity of the identification process, but also ensuring the accuracy of the identification result.

The embodiment of the invention also provides a hot rolled steel plate surface defect identification device based on computer vision, which is used for executing any one of the embodiments of the hot rolled steel plate surface defect identification method based on computer vision. Specifically, referring to fig. 3, fig. 3 is a schematic block diagram of a hot-rolled steel plate surface defect identification device based on computer vision according to an embodiment of the present invention. The hot rolled steel sheet surface defect recognition apparatus 100 based on computer vision may be configured in a server.

As shown in fig. 3, the apparatus 100 for identifying surface defects of a hot rolled steel sheet based on computer vision includes: a current frame picture receiving unit 110, a picture cutting and combining unit 120, a fast identifying unit 130, a classification result judging unit 140, a fine screening unit 150, a fine screening target image set acquiring unit 160, a defect type acquiring unit 170, and a fine screening result sending unit 180.

And a current frame picture receiving unit 110, configured to receive a current frame picture of the surface of the hot rolled steel plate uploaded by the image acquisition terminal.

In an embodiment, the apparatus 100 for identifying defects on surface of hot rolled steel sheet based on computer vision further comprises:

the device unique code receiving unit is used for receiving the current device unique code uploaded by the image acquisition terminal;

the device white list acquisition unit is used for calling a pre-stored device white list;

and the connection establishing unit is used for establishing communication connection with the image acquisition terminal if the current equipment unique code exists in the equipment white list.

And the picture clipping and synthesizing unit 120 is configured to clip and synthesize the current frame picture according to a preset picture clipping pixel size to obtain a corresponding synthesized batch image set.

In an embodiment, the picture cropping composition unit 120 comprises:

the picture cutting unit is used for calling a preset picture cutting pixel size and cutting the current frame picture into a plurality of cutting pictures according to the picture cutting pixel size;

the time stamp adding unit is used for respectively adding time stamps to the plurality of cutting pictures to correspondingly obtain a plurality of time-stamped pictures;

and the picture synthesis unit is used for carrying out synthesis batch processing on the plurality of time-stamped pictures to obtain a corresponding synthesis batch image set.

In an embodiment, the picture cropping unit includes:

the actual size acquisition unit is used for acquiring the actual pixel size of the current frame picture;

the number calculation unit is used for dividing the actual pixel size by the picture cutting pixel size to obtain the number of the target cutting pictures;

and the cutting unit is used for cutting the current frame picture into the cutting pictures with corresponding numbers according to the number of the target cutting pictures and the picture cutting pixel size.

In an embodiment, the timestamp adding unit is further configured to:

A fast recognition unit 130, configured to invoke a pre-trained convolutional neural network, and input a pixel matrix corresponding to the synthesized batch image set to the convolutional neural network, so as to obtain a corresponding image classification result set; wherein the total number of image classification results included in the image classification result set is equal to the total number of cropped pictures included in the composite batch image set.

the first calling unit is used for receiving a first model parameter set corresponding to the convolutional neural network acquired from the blockchain network so as to locally generate the convolutional neural network; wherein the convolutional neural network is an EfficientNet-b0 network.

A classification result determining unit 140, configured to determine whether the image classification result exists in the image classification result set and is a defect classification result.

In this embodiment, when the pixel matrix corresponding to the synthesized batch image set is input to the EfficientNet-b0 network, an image classification result set is obtained quickly, and at this time, it is necessary to determine whether an image classification result exists in the image classification result set as a defect classification result, so as to determine whether an image classification result exists in the image classification result set as a defect classification result through a fine recognition model in a subsequent process.

A fine screening unit 150, configured to, if the image classification result in the image classification result set is a defective classification result, invoke a pre-trained multi-scale target detection model, input a pixel matrix corresponding to each cut picture in the composite batch image set to the multi-scale target detection model, and obtain a fine screening classification result corresponding to each cut picture in the composite batch image set to form a fine screening classification result set; wherein, the fine screening classification result comprises a defective classification result and a non-defective classification result.

In this embodiment, for the "defective" pictures that are early-warned in the fine screening unit 150, the synthesized batch of image sets is inferred by an image detection network with a multi-scale detection function, specifically, by using an autoiocus (also called auto focus) method and a SNIPER network training method. The detection network with multi-scale detection function used here has a long inference time, but since only the suspected defective pictures that are preliminarily screened by the fast recognition unit 130 need to be detected, the inference time can still meet the actual requirement. The multi-scale detection model adopted in the method has high accuracy, so that the aim of accurate fine screening can be fulfilled. Among them, the SNIPER network is a multi-scale (multi-scale) training algorithm that combines the advantages of RCNN (fully called Region-CNN, an algorithm that applies deep learning to target detection) in scale processing and the advantages of fastern in speed.

In one embodiment, the device 100 for identifying surface defects of hot rolled steel plate based on computer vision further comprises:

the second calling unit is used for receiving a second model parameter set corresponding to the multi-scale target detection model acquired from the block chain network so as to locally generate the multi-scale target detection model; wherein the multi-scale target detection model is a SNIPER network.

And the fine screening target image set obtaining unit 160 is configured to obtain a fine screening target image corresponding to the fine screening classification result as a defective classification result if the fine screening classification result is a defective classification result in the fine screening classification result, so as to form a fine screening target image set.

And the defect type acquiring unit 170 is configured to acquire the size of a defect area in each fine screening target image in the fine screening target image set, so as to obtain a defect type corresponding to each fine screening target image.

And the fine screening result sending unit 180 is configured to send the fine screening target image set corresponding to the current frame picture and the defect types corresponding to the fine screening target images to the monitoring terminal corresponding to the image acquisition terminal.

and the prompting unit is used for sending the notification information for obtaining the next frame of picture on the surface of the hot-rolled steel plate to the image acquisition terminal if the image classification result does not exist in the image classification result set and is a defect classification result.

The device realizes automatic identification of the surface defects of the current frame picture, and simultaneously combines a convolutional neural network and a multi-scale target detection model, so that rapidity of an identification process can be ensured, and accuracy of an identification result is ensured.

The above-mentioned computer vision-based hot rolled steel sheet surface defect identifying apparatus may be implemented in the form of a computer program that can be run on a computer device as shown in fig. 4.

Referring to fig. 4, fig. 4 is a schematic block diagram of a computer device according to an embodiment of the present invention. The computer device 500 is a server, and the server may be an independent server or a server cluster composed of a plurality of servers.

Referring to fig. 4, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.

The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032, when executed, may cause the processor 502 to perform a hot rolled steel sheet surface defect identification method based on computer vision.

The processor 502 is used to provide computing and control capabilities that support the operation of the overall computer device 500.

The internal memory 504 provides an environment for running the computer program 5032 in the non-volatile storage medium 503, and when the computer program 5032 is executed by the processor 502, the processor 502 can be caused to execute a hot rolled steel sheet surface defect identification method based on computer vision.

The network interface 505 is used for network communication, such as providing transmission of data information. Those skilled in the art will appreciate that the configuration shown in fig. 4 is a block diagram of only a portion of the configuration associated with aspects of the present invention and is not intended to limit the computing device 500 to which aspects of the present invention may be applied, and that a particular computing device 500 may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

The processor 502 is configured to run the computer program 5032 stored in the memory to implement the method for identifying the surface defects of the hot rolled steel sheet based on computer vision disclosed in the embodiment of the present invention.

Those skilled in the art will appreciate that the embodiment of a computer device illustrated in fig. 4 does not constitute a limitation on the specific construction of the computer device, and that in other embodiments a computer device may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components. For example, in some embodiments, the computer device may only include a memory and a processor, and in such embodiments, the structures and functions of the memory and the processor are consistent with those of the embodiment shown in fig. 4, and are not described herein again.

It should be understood that, in the embodiment of the present invention, the processor 502 may be a Central Processing Unit (CPU), and the processor 502 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

In another embodiment of the invention, a computer-readable storage medium is provided. The computer readable storage medium may be a non-volatile computer readable storage medium. The computer readable storage medium stores a computer program, wherein the computer program when executed by a processor implements the computer vision-based hot rolled steel sheet surface defect identification method disclosed by the embodiment of the invention.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses, devices and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only a logical division, and there may be other divisions when the actual implementation is performed, or units having the same function may be grouped into one unit, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only memory (ROM), a magnetic disk, or an optical disk.

While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A hot rolled steel plate surface defect identification method based on computer vision is characterized by comprising the following steps:

2. The computer vision-based hot-rolled steel sheet surface defect identification method of claim 1, wherein the cropping and synthesizing the current frame picture according to a preset picture cropping pixel size to obtain a corresponding synthesized batch image set comprises:

3. The computer vision-based hot-rolled steel sheet surface defect identification method according to claim 2, wherein the calling a preset picture cropping pixel size, and cropping the current frame picture into a plurality of cropping pictures according to the picture cropping pixel size comprises:

acquiring the actual pixel size of the current frame picture;

4. The computer vision-based hot rolled steel sheet surface defect identification method according to claim 2, wherein the adding timestamps to the plurality of cropping pictures respectively to obtain a plurality of time-stamped pictures correspondingly comprises:

5. The computer vision-based hot rolled steel sheet surface defect identification method of claim 1, further comprising:

calling a pre-stored device white list;

6. The method for identifying the surface defect of the hot rolled steel sheet based on the computer vision is characterized in that before the method for identifying the surface defect of the hot rolled steel sheet based on the computer vision calls a pre-trained convolutional neural network, and a pixel matrix corresponding to the synthesized batch image set is input to the convolutional neural network to obtain a corresponding image classification result set, the method further comprises the following steps:

receiving a first model parameter set corresponding to the convolutional neural network acquired from the blockchain network so as to locally generate the convolutional neural network; wherein the convolutional neural network is an EfficientNet-b0 network;

7. The method for identifying the surface defects of the hot rolled steel sheet based on the computer vision is characterized in that after the step of judging whether the image classification result exists in the image classification result set as the defect classification result, the method further comprises the following steps:

8. A hot rolled steel plate surface defect recognition device based on computer vision is characterized by comprising:

9. A computer apparatus comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the computer program implements the computer vision based hot rolled steel sheet surface defect identification method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, causes the processor to execute the computer vision-based hot rolled steel sheet surface defect identification method according to any one of claims 1 to 7.