Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, the method for detecting defects of a metal oil pipe according to an embodiment of the present invention includes the following steps:
s1, obtaining a sample data set, wherein the sample data set comprises a plurality of sample product images with metal oil pipe defects and a plurality of sample product images without metal oil pipe defects.
In one embodiment of the invention, a large number of sample products can be photographed by a camera, for example, a metal oil pipe can be photographed by an industrial camera, and a continuous curved surface can be polished in a diffuse reflection environment during photographing, so that a high-quality sample product image can be obtained. Whether the sample product has the metal oil pipe defect or not can be used as a sample label and stored together with the sample product image to form a sample data set.
It should be noted that the surface of the metal oil pipe is a curved surface, and after the defects of the metal oil pipe, such as cracks, 3D distortion and the like, are photographed and imaged, the defect characteristics are mostly not obvious enough and are not uniform enough, so that the feature extraction and the feature recognition are not easy to be realized.
In order to solve the problem that the defect features of the curved surface are not easy to extract and identify, in an embodiment of the present invention, the surface of the metal oil pipe of the curved surface in the image can be converted into a plane.
Specifically, a linear graph of the surface of the metal oil pipe can be sequentially obtained along the circumferential direction of the metal oil pipe, and the linear graph can be spliced into a plane graph. The length direction of lines in the linear graph is consistent with the axial direction of the metal oil pipe, and the spliced planar graph means that the surface of the metal oil pipe in the image is equivalent to a plane extended from an original curved surface.
In an embodiment of the invention, the linear graph of the surface of the metal oil pipe can be obtained by rotating the metal oil pipe and sequentially shooting according to the preset rotating speed and shooting frequency. Or, sequentially shooting by a rotary camera according to the preset rotating speed and shooting frequency to obtain a linear graph of the surface of the metal oil pipe. In the above two shooting modes, the cameras used for shooting can be line scan cameras.
It should be understood that, under the condition that the rotating speed of the metal oil pipe or the camera is fixed, the higher the shooting frequency of the camera is, the smaller the width of the line used for splicing in the obtained linear graph can be, and the closer the planar graph obtained by subsequent splicing is to the actual image of the metal oil pipe after being unfolded; under the condition that the shooting frequency of the camera is fixed, the higher the rotating speed of the metal oil pipe or the camera is, the smaller the width of a line for splicing in the obtained linear graph can be, and the closer the planar graph obtained by subsequent splicing is to the actual image of the metal oil pipe after being unfolded. Therefore, the rotating speed of the metal oil pipe or the camera and the shooting frequency of the camera can be comprehensively set according to actual requirements.
And S2, training the neural network through the sample data set to obtain a metal oil pipe defect detection model.
If the sample product image in the sample data set is a plan view, model training can be performed through a common neural network, for example, a convolutional neural network such as a VGG network or an inclusion network. The convolutional neural network includes an input layer, a hidden layer, and an output layer, wherein the hidden layer includes convolutional layers. At the beginning of training, the filter of the convolutional layer is completely random and will not activate, i.e., detect, any features. A blank filter is modified in weight to detect a specific mode, and the whole process is like feedback in engineering. Through such feedback, the convolutional neural network can learn the core features to be judged by itself.
For each sample product image data, the training process may include image input, feature extraction, result prediction, result comparison, and feature memorization. Specifically, the convolutional neural network can match each feature with a corresponding sample label, the correctly matched features are retained by the memory module, the incorrectly matched features are ignored through the loss parameter, and a large number of pictures are continuously iterated through multilayer convolutional deep learning, so that the core features which the convolutional neural network wants to memorize are finally learned, and different core features are classified. The finally trained neural network, namely the metal oil pipe defect detection model can detect a new image according to the characteristics.
And if the sample product image in the sample data set is an original image containing a curved surface, performing model training by using a set network. The structure of the group integrated network is shown in fig. 2, which is an improved technique using federal learning, and the group integrated network, i.e., federal Net, is divided into several groups by a shared and multi-headed based structure to enable explicit ensemble learning in a single Net. Due to grouping volumes and shared libraries, federated nets can leverage the advantages of explicit ensemble learning while retaining the same computations as single nets. Furthermore, "group averaging," "group walking," and "group Boosting" can be taken as three different strategies to aggregate these members. Federal Net is superior to large single network and standard integration of small networks. The integrated network carries out the repetition and the effective utilization of effective characteristics in a plurality of network models through jump link and dense connection strategies, thereby realizing the characteristic extraction and the characteristic identification of defects on the curved surface.
And S3, acquiring an image of the product to be detected.
The acquisition mode of the product image to be detected is the same as that of the sample data set sample product image, and is not described herein again. It should be noted that the product image to be detected should be consistent with the sample product image, i.e. should be the original image containing the curved surface, or should be the converted plan view. And if the image of the product to be detected is an original image containing a curved surface, the same metal steel pipe can correspond to images at least three angles so as to realize comprehensive detection of the surface of the metal steel pipe.
Further, in the embodiment of the present invention, if the sample product image acquired in step S1 and the product image to be detected acquired in this step include an object other than the metal tubing, for example, an environmental background, it is also possible to acquire the metal tubing region in the image by a template matching method and intercept the metal tubing region as an image for training and detection. Specifically, referring to fig. 3, the sample product image may first be decomposed into a matrix form, and the features in the matrix form image may be arranged according to coordinates. After the matrixing process, the features in the image are obvious, for example, as shown in fig. 3, the pixel with the pixel value of 30 can be selected conveniently and quickly. The method can be used for matching the metal oil pipe template image with the area with the corresponding size of the whole image one by running a function matchTemplate through OpenCV, so that the pixel area and the pixel coordinate of the metal oil pipe are obtained.
And S4, inputting the image of the product to be detected into the metal oil pipe defect detection model to judge whether the metal oil pipe defect exists.
And inputting the image of the metal oil pipe to be detected into the metal oil pipe defect detection model to obtain an output result of whether the metal oil pipe has defects or not.
In addition, when the detection result is obtained, corresponding detection result information can be sent, for example, alarm information can be sent when the defect of the metal oil pipe is detected, a high-low level signal can be output, or an operation indication signal can be sent.
According to the metal oil pipe defect detection method provided by the embodiment of the invention, the neural network is trained through a large number of sample product images to obtain the metal oil pipe defect detection model, and whether the metal oil pipe to be detected has defects is detected through the metal oil pipe defect detection model, so that the detection efficiency is high, the labor cost is low, and the detection accuracy is high.
Corresponding to the metal oil pipe defect detection method of the embodiment, the invention further provides a metal oil pipe defect detection device.
As shown in fig. 4, the metal oil pipe defect detecting apparatus according to the embodiment of the present invention includes a first obtaining module 10, a training module 20, a second obtaining module 30, and a detecting module 40. The first obtaining module 10 is configured to obtain a sample data set, where the sample data set includes a plurality of sample product images with metal oil pipe defects and a plurality of sample product images without metal oil pipe defects; the training module 20 is used for training the neural network through the sample data set to obtain a metal oil pipe defect detection model; the second obtaining module 30 is used for obtaining an image of a product to be detected; the detection module 40 is configured to input the image of the product to be detected into the metal oil pipe defect detection model to determine whether a metal oil pipe defect exists.
In one embodiment of the invention, a large number of sample products can be photographed by a camera, for example, a metal oil pipe can be photographed by an industrial camera, and a continuous curved surface can be polished in a diffuse reflection environment during photographing, so that a high-quality sample product image can be obtained. Whether the sample product has the metal oil pipe defect or not can be used as a sample label and stored together with the sample product image to form a sample data set.
It should be noted that the surface of the metal oil pipe is a curved surface, and after the defects of the metal oil pipe, such as cracks, 3D distortion and the like, are photographed and imaged, the defect characteristics are mostly not obvious enough and are not uniform enough, so that the feature extraction and the feature recognition are not easy to be realized.
In order to solve the problem that the defect features of the curved surface are not easy to extract and identify, in an embodiment of the present invention, the first obtaining module 10 may further convert the surface of the metal oil pipe of the curved surface in the image into a plane.
Specifically, the first obtaining module 10 may sequentially obtain line graphs of the surface of the metal oil pipe along the circumferential direction of the metal oil pipe, and stitch the line graphs into a plan view. The length direction of lines in the linear graph is consistent with the axial direction of the metal oil pipe, and the spliced planar graph means that the surface of the metal oil pipe in the image is equivalent to a plane extended from an original curved surface.
In an embodiment of the invention, the linear graph of the surface of the metal oil pipe can be obtained by rotating the metal oil pipe and sequentially shooting according to the preset rotating speed and shooting frequency. Or, sequentially shooting by a rotary camera according to the preset rotating speed and shooting frequency to obtain a linear graph of the surface of the metal oil pipe. In the above two shooting modes, the cameras used for shooting can be line scan cameras.
It should be understood that, under the condition that the rotating speed of the metal oil pipe or the camera is fixed, the higher the shooting frequency of the camera is, the smaller the width of the line used for splicing in the obtained linear graph can be, and the closer the planar graph obtained by subsequent splicing is to the actual image of the metal oil pipe after being unfolded; under the condition that the shooting frequency of the camera is fixed, the higher the rotating speed of the metal oil pipe or the camera is, the smaller the width of a line for splicing in the obtained linear graph can be, and the closer the planar graph obtained by subsequent splicing is to the actual image of the metal oil pipe after being unfolded. Therefore, the rotating speed of the metal oil pipe or the camera and the shooting frequency of the camera can be comprehensively set according to actual requirements.
If the sample product image in the sample data set is a plan view, model training can be performed through a common neural network, for example, a convolutional neural network such as a VGG network or an inclusion network. The convolutional neural network includes an input layer, a hidden layer, and an output layer, wherein the hidden layer includes convolutional layers. At the beginning of training, the filter of the convolutional layer is completely random and will not activate, i.e., detect, any features. A blank filter is modified in weight to detect a specific mode, and the whole process is like feedback in engineering. Through such feedback, the convolutional neural network can learn the core features to be judged by itself.
For each sample product image data, the training process may include image input, feature extraction, result prediction, result comparison, and feature memorization. Specifically, the convolutional neural network can match each feature with a corresponding sample label, the correctly matched features are retained by the memory module, the incorrectly matched features are ignored through the loss parameter, and a large number of pictures are continuously iterated through multilayer convolutional deep learning, so that the core features which the convolutional neural network wants to memorize are finally learned, and different core features are classified. The finally trained neural network, namely the metal oil pipe defect detection model can detect a new image according to the characteristics.
And if the sample product image in the sample data set is an original image containing a curved surface, performing model training by using a set network. The structure of the group integrated network is shown in fig. 2, which is an improved technique using federal learning, and the group integrated network, i.e., federal Net, is divided into several groups by a shared and multi-headed based structure to enable explicit ensemble learning in a single Net. Due to grouping volumes and shared libraries, federated nets can leverage the advantages of explicit ensemble learning while retaining the same computations as single nets. Furthermore, "group averaging," "group walking," and "group Boosting" can be taken as three different strategies to aggregate these members. Federal Net is superior to large single network and standard integration of small networks. The integrated network carries out the repetition and the effective utilization of effective characteristics in a plurality of network models through jump link and dense connection strategies, thereby realizing the characteristic extraction and the characteristic identification of defects on the curved surface.
The manner in which the second obtaining module 30 obtains the image of the product to be detected is the same as the manner in which the first obtaining module 10 obtains the image of the sample product in the sample data set, and is not described herein again. It should be noted that the product image to be detected should be consistent with the sample product image, i.e. should be the original image containing the curved surface, or should be the converted plan view.
The detection module 40 can obtain an output result of whether the metal oil pipe has the metal oil pipe defect by inputting the image of the metal oil pipe to be detected into the metal oil pipe defect detection model.
In addition, when a detection result is obtained, corresponding detection result information can be sent out through the result indicating module, for example, alarm information can be sent out when the defect of the metal oil pipe is detected, a high-low level signal is output, or an operation indicating signal and the like can be sent out.
According to the metal oil pipe defect detection device provided by the embodiment of the invention, the neural network is trained through a large number of sample product images to obtain the metal oil pipe defect detection model, and whether the metal oil pipe to be detected has defects is detected through the metal oil pipe defect detection model, so that the detection efficiency is high, the labor cost is low, and the detection accuracy is high.
The invention further provides a computer device corresponding to the embodiment.
The computer device of the embodiment of the invention comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, and when the processor executes the computer program, the metal oil pipe defect detection method according to the embodiment of the invention can be realized.
According to the computer equipment provided by the embodiment of the invention, when the processor executes the computer program stored on the memory, the neural network is trained through a large number of sample product images to obtain the metal oil pipe defect detection model, and whether the metal oil pipe to be detected has defects is detected through the metal oil pipe defect detection model, so that the detection efficiency is higher, the labor cost is lower, and the detection accuracy is higher.
The invention also provides a non-transitory computer readable storage medium corresponding to the above embodiment.
A non-transitory computer-readable storage medium of an embodiment of the present invention stores thereon a computer program, and when the computer program is executed by a processor, the method for detecting defects of a metal oil pipe according to the above-described embodiment of the present invention can be implemented.
According to the non-transitory computer-readable storage medium of the embodiment of the invention, when the processor executes the computer program stored on the processor, the neural network is trained through a large number of sample product images to obtain the metal oil pipe defect detection model, and whether the metal oil pipe to be detected has defects is detected through the metal oil pipe defect detection model, so that the detection efficiency is high, the labor cost is low, and the detection accuracy is high.
The present invention also provides a computer program product corresponding to the above embodiments.
When the instructions in the computer program product of the embodiment of the present invention are executed by the processor, the method for detecting defects of a metal oil pipe according to the above-mentioned embodiment of the present invention can be executed.
According to the computer program product provided by the embodiment of the invention, when the processor executes the instruction, the neural network is trained through a large number of sample product images to obtain the metal oil pipe defect detection model, and whether the metal oil pipe to be detected has defects is detected through the metal oil pipe defect detection model, so that the detection efficiency is high, the labor cost is low, and the detection accuracy is high.
In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The meaning of "plurality" is two or more unless specifically limited otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, the first feature "on" or "under" the second feature may be directly contacting the first and second features or indirectly contacting the first and second features through an intermediate. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature may be directly under or obliquely under the first feature, or may simply mean that the first feature is at a lesser elevation than the second feature.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.