CN117197101A - Defect detection method, computing device and storage medium - Google Patents

Defect detection method, computing device and storage medium Download PDF

Info

Publication number
CN117197101A
CN117197101A CN202311203492.XA CN202311203492A CN117197101A CN 117197101 A CN117197101 A CN 117197101A CN 202311203492 A CN202311203492 A CN 202311203492A CN 117197101 A CN117197101 A CN 117197101A
Authority
CN
China
Prior art keywords
image
defect
pixel
column
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311203492.XA
Other languages
Chinese (zh)
Inventor
陈宁
李孟员
刘坚
李蓉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hunan University
Original Assignee
Hunan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hunan University filed Critical Hunan University
Priority to CN202311203492.XA priority Critical patent/CN117197101A/en
Publication of CN117197101A publication Critical patent/CN117197101A/en
Pending legal-status Critical Current

Links

Classifications

    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Landscapes

  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

The invention discloses a defect detection method, a computing device and a storage medium, wherein the defect detection method is executed in the computing device, and the method comprises the following steps: converting an annular area in the annular workpiece image into a rectangle to obtain a rectangular image; generating at least one sample image based on the rectangular image; respectively inputting each sample image into a defect determination model for processing, so as to correspondingly output a detection result and generate a corresponding class activation thermodynamic diagram set, wherein the detection result comprises whether each sample image contains a defect or not; determining an initial target area containing defects aiming at various types of activated thermodynamic diagram groups and generating a self-reference template, wherein the self-reference template indicates an area which does not contain defects in the sample image; and matching each initial target area with a corresponding self-reference template to determine the pixel position of the defect area. The method combines the defect determination model with the defect-free template, so that the defects can be detected more efficiently and accurately, and the method is beneficial to reducing the cost of constructing the groove defect data set.

Description

Defect detection method, computing device and storage medium
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a defect detection method, a computing device, and a storage medium.
Background
Due to the continuous development of technology, energy demand is continuously increased, traditional non-renewable energy sources are continuously exhausted and cause serious pollution, so that nuclear energy is used as a clean energy source, more attention is gradually paid, however, leakage accidents can occur in a nuclear power station, great loss is caused, and therefore, the safety of a nuclear reaction process is very important to be accurately controlled.
The zirconium tube filled with nuclear fuel is one of the main components of the heavy water reactor nuclear fuel element, and both ends of the zirconium tube are welded with the end caps to close the nuclear fuel in the tube, so that grooves are formed on the inner and outer walls of both ends to ensure the reliability and stability of the welding process, however, the grooves are easily damaged and are inevitably polluted in the disassembly process, which may cause unstable or failure of welding, seriously affecting the safety of nuclear reaction.
With the ever-expanding nuclear industry, the need for zirconium tubes has increased significantly, however, these tubes have a challenging feature-small diameter and defects are very difficult to identify with the naked eye, currently, technicians typically use magnifying glass and light sources to spot check the groove, however, this manual inspection method is inefficient, lacks standardized defect assessment procedures, results in many unacceptable parts being ignored, and, in addition, ensures that the nuclear safety requirements are fully inspected for all nuclear fuel rod grooves, and thus, relying on manual visual inspection alone is no longer sufficient to meet this stringent requirement.
In recent years, the method based on machine vision is widely applied to defect detection because of high detection speed and high precision and the capability of establishing unified standards, but the method based on self-template provides a feasible alternative method by realizing high precision without depending on a large amount of marking data because of the requirement of endowing specific labels, in particular semantic segmentation, with all training images and is time-consuming and high in cost. These methods generate a specific template for each sample, significantly improving the generalization ability of the detection algorithm, but often create templates for non-defective areas indifferently, resulting in inefficiency, and furthermore, robustness of the algorithm is challenged when there are aggregation defects in the groove.
Therefore, there is an urgent need to develop a simple, efficient and accurate method for detecting groove defects of nuclear fuel rods.
Disclosure of Invention
The present invention has been made in view of the above problems, and provides a defect detection method, a computing device, and a storage medium that overcome or at least partially solve the above problems.
According to one aspect of the present invention, there is provided a defect detection method, executed in a computing device, adapted to detect a ring-shaped workpiece, the method comprising: converting an annular area in the annular workpiece image into a rectangle to obtain a rectangular image; generating at least one sample image based on the rectangular image; respectively inputting each sample image into a defect determination model for processing, so as to correspondingly output a detection result and generate a corresponding class activation thermodynamic diagram set, wherein the detection result comprises whether each sample image contains a defect or not; determining an initial target area containing defects aiming at various types of activated thermodynamic diagram groups and generating a self-reference template, wherein the self-reference template indicates an area which does not contain defects in the sample image; and matching each initial target area with a corresponding self-reference template to determine the pixel position of the defect area.
Optionally, in the defect detection method according to the present invention, wherein generating the self-reference template includes: determining at least one image column containing defects as a defect column according to the image columns corresponding to the columns in the rectangular image in the initial target area; selecting a candidate image column set from the rectangular image for each defect column; based on each candidate image column set, a corresponding respective reference template is generated.
Optionally, in the defect detecting method according to the present invention, wherein each column in the initial target area corresponds to each image column in the rectangular image, the determination is made by:
wherein L is the pixel column index of the rectangular image, p is the sorting index of the sample image, q is the sub-image index contained in the sample image, W is the width value of the rectangle, alpha is the scaling factor of the sample image to which the initial target area belongs, and L is the pixel column index of the initial target area in the sample area.
Optionally, in the defect detection method according to the present invention, wherein the candidate image column set is acquired by:
where D is the set of pixel columns of the initial target region in the rectangular image, d= { [ R ] L1 ,R L2 ]∪[R L3 ,R L4 ]∪…},R L And t is the number of columns of preset columns at two sides of the target defect column.
Optionally, in the defect detection method according to the present invention, wherein the self-reference template is generated by:
where N is the number of column vectors in M.
Optionally, in the defect detection method according to the present invention, the matching of each initial target area with a corresponding self-reference template, determining the pixel position of the defect area includes: for each defect column, calculating pixel difference values between the defect column and corresponding pixel points in the corresponding self-reference templates; if the absolute value of the pixel difference value is larger than the threshold value, determining the pixel point corresponding to the defect column as a defect pixel point; and counting the positions of all the defective pixel points, and determining the pixel positions of the defective area.
Optionally, in the defect detecting method according to the present invention, wherein converting the annular region in the annular workpiece image into a rectangle to obtain a rectangular image includes: performing binarization processing on the annular workpiece image to obtain a binarization chart; acquiring graphic parameters of a circular ring area in the binarization graph based on Hough circle detection, wherein the graphic parameters at least comprise an outer circumference, an inner circumference, gray values of all pixel points of the circular ring area, and horizontal coordinates and vertical coordinates of a circle center of the circular ring in a pixel coordinate system; and obtaining a rectangular image by using the graphic parameters.
Optionally, in the defect detection method according to the present invention, wherein obtaining a rectangular image using the graphic parameter includes: constructing an initial rectangle by taking the outer circumference in the circular ring area as the width of the rectangular image and the difference value between the outer radius and the inner radius as the height of the rectangle; and correspondingly filling the gray value of each pixel point of the circular ring area into the initial rectangle to obtain a rectangular image.
Optionally, in the defect detection method according to the present invention, the filling the gray value of each pixel point of the ring area into the initial rectangle correspondingly to obtain the rectangular image includes: based on the following relation, the corresponding relation between each pixel point of the circular area and the pixel point in the initial rectangle is obtained:
R i.j =O m,n ,i=1,2,……H,j=1,2,……,W,
wherein O is m,n Is the gray value of the pixel point in the ring, R i.j The gray value of the pixel point in the rectangle, i and j are respectively the height and width indexes of the rectangular pixel point, W is the width value of the rectangular image, W is equal to the circumference of the outer circle of the circular ring, H is the height value of the rectangular image, and H is equal to the circular ringThe radius difference between the inner and outer circles, r is the average value of the radius of the inner circle and the outer circle of the circular ring, c x ,c y Respectively the horizontal coordinate and the vertical coordinate of the circle center in the pixel coordinate system.
Optionally, in the defect detecting method according to the present invention, further comprising: if the pixel points with unfilled gray values in the rectangular image are detected, the gray values of the blank pixel points are obtained by using a bilinear interpolation method.
Optionally, in the defect detection method according to the present invention, wherein generating at least one sample image based on the rectangular image includes: dividing the rectangular image into at least one group of sub-images along the width direction of the rectangle; and aiming at each group of sub-images, sequentially splicing all the sub-images contained in the sub-images in the rectangular height direction to obtain a sample image.
Optionally, in the defect detecting method according to the present invention, wherein dividing the rectangular image into at least one group of sub-images in the rectangular width direction includes: equally dividing the rectangular image in the width direction of the rectangle to obtain a plurality of sub-images; adjacent 3 sub-images are selected as a group of sub-images.
Optionally, in the defect detection method according to the present invention, wherein the defect determination model includes at least a plurality of feature extraction layers and classification layers; and generating a corresponding class activation thermodynamic diagram set comprising: aiming at each sample image containing defects, obtaining a feature image output by each feature extraction layer and a prediction probability output by a classification layer corresponding to the sample image; based on each feature map and the prediction probability, generating a class activation thermodynamic diagram corresponding to each feature map as a class activation thermodynamic diagram group.
Optionally, in the defect detection method according to the present invention, wherein determining the initial target area including the defect includes: marking the areas with the thermodynamic values larger than the thermodynamic threshold value of each class of the activation thermodynamic diagram in the class activation thermodynamic diagram group as a sub-target area; and merging all the sub-target areas to generate a target area.
Optionally, in the defect detection method according to the present invention, wherein the processing of inputting each sample image into the defect determination model includes:
processing each sample image to obtain each sample image with consistent size; and respectively inputting the sample images with the consistent sizes into a defect determination model to determine whether the sample images contain defects, wherein the defect determination model comprises a plurality of feature extraction layers, an attention layer, a global average pooling layer and a classification layer.
According to yet another aspect of the present invention, there is provided a computing device comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the above-described method.
According to yet another aspect of the present invention, there is provided a readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform the above-described method.
According to the scheme of the invention, the defect determination model is combined with the defect-free template, so that the defects can be detected more efficiently and accurately, and the cost for constructing the groove defect data set is reduced.
According to the scheme of the invention, aiming at the input image containing the defects, the designed defect-free template avoids the interference of adjacent defect areas to a great extent, so that the robustness of the self-reference template is improved.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 shows a schematic view of a zirconium tube end face ramp structure;
FIG. 2 shows a schematic diagram of a computing device 200 according to one embodiment of the invention;
FIG. 3 illustrates a flow chart of a defect detection method 300 according to one embodiment of the invention;
FIG. 4 shows a rectangular image schematic according to one embodiment of the invention;
FIG. 5 illustrates a schematic diagram of defect determination model results, according to one embodiment of the invention;
FIG. 6 shows a pictorial representation of a class activation thermodynamic diagram in accordance with an embodiment of the present invention;
FIG. 7 illustrates a flow diagram of nuclear fuel rod groove defect detection in accordance with one embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The nuclear fuel rod is a hollow zirconium tube, and each end face is provided with a groove which is manufactured by turning. Industrial cameras are used to capture images of the bevel surface for defect detection. As shown in fig. 1, fig. 1 shows a schematic view of a zirconium tube end face slope structure. When the end surface of the zirconium tube has defects, the welding is unstable or fails, and the safety of nuclear reaction is seriously affected.
Because zirconium tubes are small in diameter, it becomes very difficult to identify defects with the naked eye, and currently, technicians typically use magnifying glass and light sources to spot check the groove, however, this manual inspection method is inefficient, lacks a standardized defect evaluation process, and results in many unacceptable parts being ignored.
Accordingly, to solve the above-mentioned problems with the prior art, an embodiment of the present invention provides a defect detection method, which can detect defects more efficiently and accurately by combining a defect determination model with a defect-free template, and is also beneficial to reducing the cost of constructing a groove defect dataset. For an input image containing defects, the designed defect-free template largely avoids interference of adjacent defect areas, so that the robustness of the self-reference template is improved. The method may be performed in a computing device. FIG. 1 illustrates a block diagram of a computing device 200 according to one embodiment of the invention. It should be noted that the computing device 200 shown in fig. 2 is only an example, and in practice, the computing device for implementing the defect detection method of the present invention may be any type of device, and the hardware configuration of the computing device may be the same as that of the computing device 200 shown in fig. 2 or may be different from that of the computing device 200 shown in fig. 2. In practice, the computing device for implementing the groove defect detection method of the present invention may add or delete hardware components of the computing device 200 shown in fig. 2, and the present invention is not limited to the specific hardware configuration of the computing device.
As shown in FIG. 2, in a basic configuration 202, computing device 200 typically includes a system memory 206 and one or more processors 204. A memory bus 208 may be used for communication between the processor 204 and the system memory 206.
Depending on the desired configuration, the processor 204 may be any type of processing including, but not limited to: a microprocessor (μp), a microcontroller (μc), a digital information processor (DSP), or any combination thereof. Processor 204 may include one or more levels of cache, such as a first level cache 210 and a second level cache 212, a processor core 214, and registers 216. The example processor core 214 may include an Arithmetic Logic Unit (ALU), a Floating Point Unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The example memory controller 218 may be used with the processor 204, or in some implementations, the memory controller 218 may be an internal part of the processor 204.
Depending on the desired configuration, system memory 206 may be any type of memory including, but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. Physical memory in a computing device is often referred to as volatile memory, RAM, and data in disk needs to be loaded into physical memory in order to be read by processor 204. The system memory 206 may include an operating system 220, one or more applications 222, and program data 224. In some implementations, the application 222 may be arranged to execute instructions on an operating system by the one or more processors 204 using the program data 224. The operating system 220 may be, for example, linux, windows or the like, which includes program instructions for handling basic system services and performing hardware-dependent tasks. The application 222 includes program instructions for implementing various user desired functions, and the application 222 may be, for example, a browser, instant messaging software, a software development tool (e.g., integrated development environment IDE, compiler, etc.), or the like, but is not limited thereto. When an application 222 is installed into computing device 200, a driver module may be added to operating system 220.
When the computing device 200 starts up running, the processor 204 reads the program instructions of the operating system 220 from the memory 206 and executes them. Applications 222 run on top of operating system 220, utilizing interfaces provided by operating system 220 and underlying hardware, to implement various user-desired functions. When the user launches the application 222, the application 222 is loaded into the memory 206, and the processor 204 reads and executes the program instructions of the application 222 from the memory 206.
Computing device 200 also includes a storage device 232, where storage device 232 includes removable storage 236 and non-removable storage 238, where removable storage 236 and non-removable storage 238 are each connected to storage interface bus 234.
Computing device 200 may also include an interface bus 240 that facilitates communication from various interface devices (e.g., output devices 242, peripheral interfaces 244, and communication devices 246) to basic configuration 202 via bus/interface controller 230. The example output device 242 includes a graphics processing unit 248 and an audio processing unit 250. They may be configured to facilitate communication with various external devices, such as a display or speakers, via one or more a/V ports 252. The example peripheral interface 244 may include a serial interface controller 254 and a parallel interface controller 256, which may be configured to facilitate communication via one or more I/O ports 258 and external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., printer, scanner, etc.). The example communication device 246 may include a network controller 260 that may be arranged to facilitate communication with one or more other computing devices 262 over a network communication link via one or more communication ports 264.
The network communication link may be one example of a communication medium. Communication media may typically be embodied by computer readable instructions, data structures, program modules, and may include any information delivery media in a modulated data signal, such as a carrier wave or other transport mechanism. A "modulated data signal" may be a signal that has one or more of its data set or changed in such a manner as to encode information in the signal. By way of non-limiting example, communication media may include wired media such as a wired network or special purpose network, and wireless media such as acoustic, radio Frequency (RF), microwave, infrared (IR) or other wireless media. The term computer readable media as used herein may include both storage media and communication media.
In computing device 200 according to the present invention, application 222 includes instructions for performing groove defect detection method 300 of the present invention, which may instruct processor 204 to perform the groove defect detection method of the present invention. Those skilled in the art will appreciate that the application 222 may include other applications 226 for implementing other functions in addition to the instructions for performing the defect detection method 300.
FIG. 3 illustrates a flow chart of a defect detection method 300 according to one embodiment of the invention. The method 300 is suitable for execution in a computing device (e.g., the computing device 300 described previously).
As shown in fig. 3, the purpose of the method 300 is to implement a method for defect detection of an annular workpiece (e.g., a groove on an end surface of a zirconium tube filled with nuclear fuel), which utilizes a defect determination model in combination with a defect-free template to detect defects more efficiently and accurately, which is advantageous in reducing the cost of constructing a groove defect dataset.
The method 300 begins at step 302 by converting an annular region in an annular workpiece image to a rectangle, resulting in a rectangular image, at step 302.
In this embodiment, the annular workpiece image may be understood as an image of the end face of the zirconium tube, and an annular light source adapted to the end face groove of the zirconium tube may be used in image acquisition of the end face of the zirconium tube. Thus, the groove area in the acquired end image is clearer, so that the accuracy of groove defect detection can be ensured. Of course, the image may be acquired based on other light sources, which is not particularly limited by the present invention.
Referring to fig. 1, the groove region to be detected is a circular ring (annular region) whose area ratio is less than 10% in the acquired image, and in order to eliminate interference of the non-interested region, it is necessary to expand the circular ring into a rectangle in order to improve detection accuracy.
Specifically, first, a binarization process is performed on an annular workpiece image to obtain a binarization map. The end image may be binarized using an adaptive binarization method. However, it should be noted that the present invention is not limited to the specific algorithm used for binarizing the end image, and any known or future known image binarization method is within the scope of the present invention.
In some embodiments, holes and edge burrs in the binary image may be removed (i.e., the binary image is noise reduced), leaving only the annular region. Preferably, morphological processing methods can be employed to remove holes and edge burrs in the end binary image. Of course, other methods may be used to remove the holes and edge burrs in the end binary image, which is not particularly limited in the present invention.
Then, based on Hough circle detection, obtaining graphic parameters of a circular ring area in the binarization graph, wherein the graphic parameters at least comprise an outer circumference 2 pi R Outer part Inner circumference 2 pi R Inner part Gray value O of each pixel point in circular ring area m,n Horizontal coordinate c of circle center of circular ring in pixel coordinate system x And vertical coordinate c y
Wherein, the outer circle radius R Outer part The external square of the excircle of the annular region can be determined first, and then half of the side length of the external square is taken as the radius of the excircle. Same inner radius R Inner part Is half the side length of the circumscribed square of the inner circle of the annular area.
Finally, a rectangular image is obtained by using the graphic parameters. Specifically, an initial rectangle is constructed with the outer circumference in the annular region as the width of the rectangular image, and the difference between the outer and inner radii as the height of the rectangular image. And correspondingly filling the gray value of each pixel point of the circular ring area into the initial rectangle to obtain a rectangular image.
The corresponding relation between each pixel point of the circular area and the pixel point in the initial rectangle can be obtained based on the following relation;
R i.j =O m,n ,i=1,2,……H,j=1,2,……,W
wherein O is m,n Is the gray value of the pixel point in the ring, R i.j The gray value of the pixel point in the rectangle, i and j are respectively the height and width indexes of the rectangular pixel point, W is the width value of the rectangular image, W is equal to the circumference of the outer circle of the circular ring, H is the height value of the rectangular image, H is equal to the radius difference between the inner circle and the outer circle of the circular ring, r is the average value of the radius of the inner circle and the outer circle of the circular ring, and c x ,c y Respectively the horizontal coordinate and the vertical coordinate of the circle center in the pixel coordinate system.
After the corresponding relation between each pixel point of the annular region and the pixel points in the initial rectangle is determined, each pixel in the annular region is converted into a rectangular image.
In a specific example, according to the coordinates of any point a in the rectangular image, the coordinates of a point a 'corresponding to the point a in the annular image are calculated by using the correspondence, and then the pixel value of the point a' in the annular groove image is assigned to the point a in the rectangular image. Thus, the pixel value of the point a in the rectangular image is equal to the pixel value of the point a' in the corresponding annular image, so as to obtain the rectangular image (as shown in fig. 4).
In many cases, the calculated coordinates of the point a' are not an integer. That is, the calculated point a' is not one pixel point in the ring groove image, but one point between the pixel points, and it is apparent that the pixel value of the point a in the rectangular image has not been determined in this case. Thus, according to one embodiment, a bilinear difference method may be used to determine the pixel value of point A in a rectangular image from the pixel values of pixels near point A'. If the pixel points with unfilled gray values in the rectangular image are detected, the gray values of the blank pixel points are obtained by a bilinear interpolation method.
Of course, interpolation methods such as nearest neighbor interpolation, bicubic interpolation of 4x4 pixel neighborhood, etc. may also be used to determine the pixel values of each point in the rectangular image, which is not limited in this invention.
After obtaining the rectangular image, step 304 is entered in which at least one sample image is generated based on the rectangular image.
Specifically, first, a rectangular image is divided into at least one group of sub-images in the rectangular width direction. In some embodiments, the rectangular image may be equally divided in the width direction of the rectangle, resulting in a plurality of sub-images. The adjacent 3 sub-images are taken as a group of sub-images. For example, a rectangular image is equally divided into 9 sub-images in the width direction (W direction in fig. 4), and is sequentially labeled 1, 2 … …, with 1, 2, and 3 sub-images as a group of sub-images, 4, 5, and 6 sub-images as a group of sub-images, and 7, 8, and 9 sub-images as a group of sub-images.
Then, for each group of sub-images, the sub-images contained in the sub-images are spliced in sequence in the rectangular height direction, so that a sample image is obtained. Continuing with the above example, for sub-image groups 1, 2, and 3, 2 sub-images are translated below 1 sub-image (direction H in fig. 4), then 3 sub-images are translated below the moved 2 sub-images, resulting in a sample image, and so on, to obtain sample images of 4, 5, and 6 sub-image groups and sample images of 7, 8, and 9 sub-image groups, respectively. The rectangular image is segmented to obtain a plurality of sample images, so that the calculation efficiency can be effectively improved.
After obtaining each sample image, the process proceeds to step 306, where each sample image is respectively input into a defect determination model for processing, so as to correspondingly output a detection result and generate a corresponding class activation thermodynamic diagram set, where the detection result includes whether each sample image includes a defect.
Specifically, each sample image is first processed (cut and scaled) to obtain each sample image having a uniform size. Preferably, the size of each sample image may be modified to 224 x 224.
Then, each sample image with the same size is respectively input into a defect determination model to determine whether each sample image contains a defect part.
As shown in fig. 5, the defect determination model of the present embodiment is composed of three parts: a feature extraction layer, an attention layer, a global averaging pooling layer, and a classification layer.
The sample image is firstly input into a feature extraction layer for feature extraction, and a feature map of the sample image is obtained.
The feature extraction layer is connected with the attention layer, the feature map is used as input of the attention layer, and the size of the feature map is not changed in the process. The attention layer makes the network more focused on defective areas, thereby enhancing feature extraction capabilities. Preferably, in order to improve the applicability of the classification model, the present embodiment uses a pyramid extrusion attention module as an attention layer, and the module uses convolution kernels and group convolutions with different sizes, so that a multi-scale feature map can be obtained on a channel.
In this embodiment, in order to reduce the number of parameters, the feature map is not directly flattened, but global average pooling is adopted, and finally, an output result is obtained through full connection. In order to improve classification accuracy and speed up convergence, migration learning is utilized by selecting a backbone network with pre-training weights as a feature extraction layer.
In some embodiments, the optimizer uses a random gradient to drop the SGD during network training, and the loss function uses cross entropy loss.
If the defect determination model finds that the sample data has defects, a corresponding class activation thermodynamic diagram group is generated by using each sample image containing the defects.
Specifically, for each sample image including a defect, first, a feature map output by the feature extraction layer (i.e., a feature map output by the feature extraction layer in the defect determination model) and a prediction probability output by the classification layer corresponding to the sample image are obtained.
Then, based on each feature map and the prediction probability, a class activation thermodynamic diagram corresponding to each feature map is generated as the class activation thermodynamic diagram group, as shown in fig. 6, and fig. 6 shows a physical display diagram of the class activation thermodynamic diagram according to an embodiment of the present invention. It should be noted that, in order to better illustrate the different thermal values between the pixels in the class activation thermodynamic diagram of the present embodiment, fig. 6 and fig. 7 of the present embodiment each select a color chart for display.
Preferably, the LayerCAM method may be used to construct each initial class activation thermodynamic diagram.
After obtaining the class activation heat map set, step 308 is performed to determine an initial target area containing the defect for each class of activation heat map set and generate a self-reference template, wherein the self-reference template indicates an area in the sample image that does not contain the defect. Specifically, the area, in the class activation thermodynamic diagram group, in which the thermodynamic value of each class activation thermodynamic diagram is greater than the thermodynamic threshold is marked as a sub-target area. All sub-target areas are merged to generate an initial target area (part a rectangular box area in fig. 6). In other words, to ensure that defects are not missed, the union region of class activation thermodynamic diagrams of feature maps of different network depths is selected as the final initial target region.
In some embodiments, in order to make the extracted target area fit to the actual defect position as much as possible, different thresholds are set for feature maps of different depths when constructing class activation thermodynamic diagrams of each feature map, and the deeper the network, the greater the threshold.
It should be noted that, in the present embodiment, the initial target area is a rough area including defects, and the initial target area includes all defect areas, and includes non-defect areas, so as to obtain more accurate defect areas, in step 310, each initial target area is matched with a corresponding self-reference template, so as to determine the pixel positions of the defect areas.
Wherein the step of constructing a defect-free template comprises:
first, at least one image column containing a defect is determined as a defective column (R L )。
Wherein, the mapping relation between each column (L) in the initial target area and each image column (L) in the rectangular image is as follows:
wherein L is the pixel column index of the rectangular image, p is the sorting index of the sample image, q is the sub-image index contained in the sample image, W is the rectangular width value, α is the scaling factor of the sample image to which the initial target area belongs (i.e. the scaling factor when the sample image size is modified to 224×224 as described above), and L is the pixel column index of the initial target area in the sample area.
Then, for each defective column (R L ) Candidate image column sets (M) are respectively selected from the rectangular images.
Let R be j Is a pixel column vector with dimension of H multiplied by 1 in the original rectangular image, the set of pixel columns with the initial target area in the rectangular image is D= { [ R L1 ,R L2 ]∪[R L3 ,R L4 ]∪…}。
Let the target defect column be R L By the sum of t columns from the left side thereofThe right t columns form a table of an image column matrix (i.e., a set of candidate image columns) that is shown as:
And finally, generating corresponding self-reference templates based on the candidate image column sets.
The target defect is listed as R L Can be expressed as a defect-free template of:
where N is the number of column vectors in M.
Notably, since the rectangle is formed by circular ring expansion, when R L When the number of columns on the left is less than t, the number should be replenished from the end of the rectangle. Similarly, when R L When the number of columns on the right side is smaller than t, the number of columns should be complemented from the beginning of the rectangle.
After obtaining the non-defective templates of the amount of each target area, calculating pixel difference values between the defective columns and corresponding pixel points in the corresponding self-reference templates according to each defective column. If the absolute value of the pixel difference value is greater than the threshold value, the pixel point corresponding to the defect column is determined as a defective pixel point.
Specifically, for each R L The difference from the defect-free template is calculated according to the following formula:
Δ L =|R L -T L |=[Δ 0,L ,Δ 1,L …Δ H,L ] T
if the absolute value of the difference is greater than the threshold K, the location is considered a defect. In addition, since the difference may be greater than a threshold due to noise interference of a single pixel, a pixel-level defective region is obtained after removing an abnormal point through the etching and expanding operations.
In one specific example, in connection with fig. 7, fig. 7 shows a schematic flow diagram of nuclear fuel rod groove defect detection according to one embodiment of the present invention.
In fig. 7, stage 1 corresponds to steps 302 through 304 of method 300 described above, stage 2 corresponds to step 306, stage 3 and stage 4 correspond to step 308, and stage 5 corresponds to step 310.
For a specific process of detecting a groove defect of a nuclear fuel rod, please refer to the description of the method 300, which is not repeated in the present disclosure.
The method provided by the application can detect the defects more efficiently and accurately by combining the defect determination model with the defect-free template, and is beneficial to reducing the cost of constructing the groove defect data set.
The method aims at the input image containing the defects, and the designed non-defective template avoids the interference of adjacent defect areas to a great extent, so that the robustness of the self-reference template is improved.
A9, the method of A8, wherein the step of correspondingly filling the gray value of each pixel point of the circular ring area into the initial rectangle to obtain the rectangular image comprises the following steps:
based on the following relation, the corresponding relation between each pixel point of the circular area and the pixel point in the initial rectangle is obtained:
R i.j =O m,n ,i=1,2,……H,j=1,2,……,W,
wherein O is m,n Is the gray value of the pixel point in the ring, R i.j The gray value of the pixel point in the rectangle, i and j are respectively the height and width indexes of the rectangular pixel point, W is the width value of the rectangular image, W is equal to the circumference of the outer circle of the circular ring, H is the height value of the rectangular image, H is equal to the radius difference between the inner circle and the outer circle of the circular ring, r is the average value of the radius of the inner circle and the outer circle of the circular ring, and c x ,c y Respectively the horizontal coordinate and the vertical coordinate of the circle center in the pixel coordinate system.
A10, the method of A9, further comprising:
and if the pixel points with unfilled gray values in the rectangular image are detected, acquiring the gray values of the blank pixel points by using a bilinear interpolation method.
A11, the method of A1, wherein generating at least one sample image based on the rectangular image comprises:
dividing the rectangular image into at least one group of sub-images along the width direction of the rectangle;
and aiming at each group of sub-images, sequentially splicing all the sub-images contained in the sub-images in the rectangular height direction to obtain a sample image.
A12, the method of a11, wherein dividing the rectangular image into at least one group of sub-images along the rectangular width direction, comprises:
equally dividing the rectangular image in the width direction of the rectangle to obtain a plurality of sub-images;
adjacent 3 sub-images are selected as a group of sub-images.
A13, the method of A1, wherein the defect determination model at least comprises a plurality of feature extraction layers and classification layers;
and
generating a corresponding class activation thermodynamic diagram set comprising:
for each sample image that contains a defect,
Acquiring a feature map output by each feature extraction layer and a prediction probability output by a classification layer corresponding to the sample image;
based on each feature map and the prediction probability, generating a class activation thermodynamic diagram corresponding to each feature map as the class activation thermodynamic diagram group.
A14, the method of a13, wherein determining an initial target area containing defects comprises:
marking the areas with the thermodynamic values larger than the thermodynamic threshold value of each class of the activation thermodynamic diagram in the class activation thermodynamic diagram group as a sub-target area;
and merging all the sub-target areas to generate the target area.
A15, the method of A1, wherein inputting each sample image into the defect determination model for processing comprises the following steps:
processing each sample image to obtain each sample image with consistent size;
and respectively inputting the sample images with the consistent sizes into a defect determination model to determine whether the sample images contain defects, wherein the defect determination model comprises a plurality of feature extraction layers, an attention layer, a global average pooling layer and a classification layer.
The various techniques described herein may be implemented in connection with hardware or software or, alternatively, with a combination of both. Thus, the methods and apparatus of the present invention, or certain aspects or portions of the methods and apparatus of the present invention, may take the form of program code (i.e., instructions) embodied in tangible media, such as removable hard drives, U-drives, floppy diskettes, CD-ROMs, or any other machine-readable storage medium, wherein, when the program is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.
In the case of program code execution on programmable computers, the computing device will generally include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Wherein the memory is configured to store program code; the processor is configured to perform the method of the invention in accordance with instructions in said program code stored in the memory.
By way of example, and not limitation, readable media comprise readable storage media and communication media. The readable storage medium stores information such as computer readable instructions, data structures, program modules, or other data. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Combinations of any of the above are also included within the scope of readable media.
In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with examples of the invention. The required structure for a construction of such a system is apparent from the description above. In addition, the present invention is not directed to any particular programming language. It should be appreciated that the teachings of the present invention as described herein may be implemented in a variety of programming languages and that the foregoing description of specific languages is provided for disclosure of preferred embodiments of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment, or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or may be further divided into a plurality of sub-modules.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.
Furthermore, some of the embodiments are described herein as methods or combinations of method elements that may be implemented by a processor of a computer system or by other means of performing the functions. Thus, a processor with the necessary instructions for implementing the described method or method element forms a means for implementing the method or method element. Furthermore, the elements of the apparatus embodiments described herein are examples of the following apparatus: the apparatus is for carrying out the functions performed by the elements for carrying out the objects of the invention.
As used herein, unless otherwise specified the use of the ordinal terms "first," "second," "third," etc., to describe a general object merely denote different instances of like objects, and are not intended to imply that the objects so described must have a given order, either temporally, spatially, in ranking, or in any other manner.
While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of the above description, will appreciate that other embodiments are contemplated within the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the invention, which is defined by the appended claims.

Claims (10)

1. A defect detection method, executed in a computing device, adapted to detect a ring-shaped workpiece, the method comprising:
converting an annular area in the annular workpiece image into a rectangle to obtain a rectangular image;
generating at least one sample image based on the rectangular image;
inputting each sample image into a defect determination model respectively for processing so as to correspondingly output detection results and generate corresponding class activation thermodynamic diagram sets, wherein the detection results comprise whether each sample image contains defects or not;
determining an initial target area containing defects aiming at various types of activated thermodynamic diagram groups and generating a self-reference template, wherein the self-reference template indicates areas which do not contain defects in the sample image;
and matching each initial target area with a corresponding self-reference template to determine the pixel position of the defect area.
2. The method of claim 1, wherein generating the self-referencing template comprises:
determining at least one image column containing defects as a defect column according to each column in the initial target area corresponding to each image column in the rectangular image;
selecting a candidate image column set from the rectangular image for each defect column;
Based on each set of candidate image columns, a corresponding each self-referencing template is generated.
3. The method of claim 2, wherein each column in the initial target area corresponds to each column of images in the rectangular image, determined by:
wherein L is the pixel column index of the rectangular image, p is the sorting index of the sample image, q is the sub-image index contained in the sample image, W is the width value of the rectangle, alpha is the scaling factor of the sample image to which the initial target area belongs, and L is the pixel column index of the initial target area in the sample area.
4. A method as claimed in claim 3, wherein the set of candidate image columns is obtained by:
M=[R L-t R L-t+1 …R i …R L+t ],
wherein D is a set of pixel columns of the initial target region in the rectangular image, d= { [ R ] L1 ,R L2 ]∪[R L3 ,R L4 ]∪…},R L And t is the number of columns of preset columns at two sides of the target defect column.
5. The method of claim 4, wherein the self-referencing template is generated by:
where N is the number of column vectors in M.
6. The method of claim 5, wherein matching each initial target region with a corresponding self-referencing template, determining pixel locations of defective regions, comprises:
For each defect column, calculating pixel difference values between the defect column and corresponding pixel points in the corresponding self-reference templates;
if the absolute value of the pixel difference value is larger than the threshold value, determining the pixel point corresponding to the defect column as a defect pixel point;
and counting the positions of all the defective pixel points, and determining the pixel positions of the defective area.
7. The method of claim 1, wherein converting the annular region in the annular workpiece image to a rectangle to obtain a rectangular image comprises:
performing binarization processing on the annular workpiece image to obtain a binarization chart;
acquiring graphic parameters of a circular ring area in the binarization graph based on Hough circle detection, wherein the graphic parameters at least comprise an outer circumference, an inner circumference, gray values of all pixel points of the circular ring area, and horizontal coordinates and vertical coordinates of a circle center of the circular ring in a pixel coordinate system;
and obtaining the rectangular image by using the graphic parameters.
8. The method of claim 7, wherein obtaining the rectangular image using the graphics parameters comprises:
constructing an initial rectangle by taking the outer circumference in the circular ring area as the width of the rectangular image and the difference value between the outer radius and the inner radius as the height of the rectangle;
And correspondingly filling the gray value of each pixel point of the circular ring area into the initial rectangle to obtain the rectangular image.
9. A computing device, comprising:
at least one processor; and
a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method of any of claims 1-8.
10. A readable storage medium storing program instructions which, when read and executed by a computing device, cause the computing device to perform the method of any of claims 1-8.
CN202311203492.XA 2023-09-18 2023-09-18 Defect detection method, computing device and storage medium Pending CN117197101A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311203492.XA CN117197101A (en) 2023-09-18 2023-09-18 Defect detection method, computing device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311203492.XA CN117197101A (en) 2023-09-18 2023-09-18 Defect detection method, computing device and storage medium

Publications (1)

Publication Number Publication Date
CN117197101A true CN117197101A (en) 2023-12-08

Family

ID=88997596

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311203492.XA Pending CN117197101A (en) 2023-09-18 2023-09-18 Defect detection method, computing device and storage medium

Country Status (1)

Country Link
CN (1) CN117197101A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112102316A (en) * 2020-11-05 2020-12-18 常州微亿智造科技有限公司 Method and device for acquiring distribution of product defect positions
CN113687227A (en) * 2021-08-24 2021-11-23 桂林电子科技大学 Motor magnetic shoe defect classification method based on region-of-interest enhancement
CN113780385A (en) * 2021-08-30 2021-12-10 武汉理工大学 Driving risk monitoring method based on attention mechanism
US20230033187A1 (en) * 2021-07-28 2023-02-02 Coopervision International Limited Systems and methods for acquiring and inspecting lens images of ophthalmic lenses
KR20230023263A (en) * 2021-08-10 2023-02-17 세종대학교산학협력단 Deep learning-based sewerage defect detection method and apparatus

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112102316A (en) * 2020-11-05 2020-12-18 常州微亿智造科技有限公司 Method and device for acquiring distribution of product defect positions
US20230033187A1 (en) * 2021-07-28 2023-02-02 Coopervision International Limited Systems and methods for acquiring and inspecting lens images of ophthalmic lenses
KR20230023263A (en) * 2021-08-10 2023-02-17 세종대학교산학협력단 Deep learning-based sewerage defect detection method and apparatus
CN113687227A (en) * 2021-08-24 2021-11-23 桂林电子科技大学 Motor magnetic shoe defect classification method based on region-of-interest enhancement
CN113780385A (en) * 2021-08-30 2021-12-10 武汉理工大学 Driving risk monitoring method based on attention mechanism

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
刘勃: "基于迁移学习的光刻缺陷的识别与可视化", 中国优秀硕士学位论文全文数据库(电子期刊)信息科技辑, no. 01, 15 January 2022 (2022-01-15), pages 135 - 554 *
刘坚 等: "基于平均模板法的锆管坡口异物视觉检测研究", 湖南大学学报(自然科学版), vol. 47, no. 12, 31 December 2020 (2020-12-31), pages 53 - 60 *

Similar Documents

Publication Publication Date Title
Park et al. Patch-based crack detection in black box images using convolutional neural networks
CN113378686B (en) Two-stage remote sensing target detection method based on target center point estimation
CN110427946B (en) Document image binarization method and device and computing equipment
CN111259878A (en) Method and equipment for detecting text
CN110751154B (en) Complex environment multi-shape text detection method based on pixel-level segmentation
CN110334709B (en) License plate detection method based on end-to-end multi-task deep learning
CN115496746A (en) Method and system for detecting surface defects of plate based on fusion of image and point cloud data
CN111260675B (en) High-precision extraction method and system for image real boundary
CN112819748B (en) Training method and device for strip steel surface defect recognition model
CN113221867A (en) Deep learning-based PCB image character detection method
CN111027538A (en) Container detection method based on instance segmentation model
CN112507876A (en) Wired table picture analysis method and device based on semantic segmentation
CN115331245A (en) Table structure identification method based on image instance segmentation
CN115752969A (en) Method, system and equipment for detecting sealing performance of aluminum foil seal
CN115147403A (en) Method and device for detecting liquid pollutants, electronic equipment and medium
Pan et al. Recovering building information model from 2D drawings for mechanical, electrical and plumbing systems of ageing buildings
CN113255555A (en) Method, system, processing equipment and storage medium for identifying Chinese traffic sign board
CN116824122A (en) LED chip positioning method and LED chip positioning device based on deep learning
CN113066051A (en) Groove defect detection method, computing equipment and readable storage medium
CN116883313A (en) Method for rapidly detecting vehicle body paint surface defects, image processing equipment and readable medium
CN117197101A (en) Defect detection method, computing device and storage medium
CN117314880A (en) Image defect detection method and device
CN112784675A (en) Target detection method and device, storage medium and terminal
CN111179278A (en) Image detection method, device, equipment and storage medium
CN116228637A (en) Electronic component defect identification method and device based on multi-task multi-size network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination