CN115564773A - Small sample image defect detection method, device and equipment based on meta-learning - Google Patents

Small sample image defect detection method, device and equipment based on meta-learning Download PDF

Info

Publication number
CN115564773A
CN115564773A CN202211446408.2A CN202211446408A CN115564773A CN 115564773 A CN115564773 A CN 115564773A CN 202211446408 A CN202211446408 A CN 202211446408A CN 115564773 A CN115564773 A CN 115564773A
Authority
CN
China
Prior art keywords
image
power grid
grid image
defect detection
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211446408.2A
Other languages
Chinese (zh)
Other versions
CN115564773B (en
Inventor
黄文琦
曾群生
吴洋
周锐烨
姚森敬
李端姣
李雄刚
刘高
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southern Power Grid Digital Grid Research Institute Co Ltd
Original Assignee
Southern Power Grid Digital Grid Research Institute Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southern Power Grid Digital Grid Research Institute Co Ltd filed Critical Southern Power Grid Digital Grid Research Institute Co Ltd
Priority to CN202211446408.2A priority Critical patent/CN115564773B/en
Publication of CN115564773A publication Critical patent/CN115564773A/en
Application granted granted Critical
Publication of CN115564773B publication Critical patent/CN115564773B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/17—Terrestrial scenes taken from planes or by drones
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Abstract

The application relates to a method, a device and equipment for detecting small sample image defects based on meta-learning. The method comprises the following steps: the method comprises the steps of carrying out data amplification processing according to a plurality of power grid images to be detected to obtain a power grid image set, determining a first power grid image from the power grid image set, determining second power grid images from each power grid image subset, inputting the first power grid image and the second power grid images into a preset identification model for feature extraction and defect detection to obtain a defect detection result. By adopting the method, the defect position and the defect type of the small sample image to be detected can be detected according to the preset identification model and the common sample augmentation mode, so that the defect detection accuracy of the small sample image is improved.

Description

Small sample image defect detection method, device and equipment based on meta-learning
Technical Field
The present application relates to the field of image data processing technologies, and in particular, to a method, an apparatus, and a device for detecting small sample image defects based on meta learning.
Background
Along with the development of a power grid system, unmanned aerial vehicle inspection of a power grid overhead line is more and more common, and the problem of large workload exists in a mode that an image obtained after unmanned aerial vehicle inspection is manually inspected to find defects, so that an image intelligent defect detection technology is realized. The technique requires comparing the images after inspection with samples in an existing sample library in the grid system to determine the location and type of the defect.
However, the existing image intelligent defect detection technology has the problem of low accuracy in detecting defects.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a method, an apparatus and a device for detecting a defect in a small sample image based on meta learning, which can improve a defect detection rate.
In a first aspect, the application provides a method for detecting a small sample image defect based on meta-learning. The method comprises the following steps:
carrying out data amplification processing on a plurality of power grid images to be detected to obtain a power grid image set; the power grid image set comprises at least two different types of power grid image subsets, and the number of images of a plurality of power grid images is smaller than a first threshold value;
determining a first power grid image from the power grid image set, and determining a second power grid image from each power grid image subset;
inputting the first power grid image and each second power grid image into a preset identification model for feature extraction and defect detection to obtain a defect detection result; the defect detection result includes the location and type of the defect.
In one embodiment, the identifying model includes a feature extraction network and a fusion network, and the step of inputting the first power grid image and each second power grid image into a preset identifying model for feature extraction and defect detection to obtain a defect detection result includes:
inputting the first power grid image and each second power grid image into a feature extraction network for feature extraction to obtain a first image feature of the first power grid image and a second image feature of each second power grid image;
and inputting the first image characteristics and the second image characteristics into a fusion network for characteristic fusion and defect detection to obtain a defect detection result.
In one embodiment, the converged network is configured to perform the steps of:
fusing the first image characteristic and the position information of the first power grid image to obtain a first fused characteristic;
fusing the second image characteristic, the position information of the second power grid image and the task identifier to obtain a second fused characteristic;
and detecting the defects according to the first fusion characteristic and the second fusion characteristic to obtain a defect detection result.
In one embodiment, the method further comprises:
performing grid division on the first power grid image and the second power grid image to obtain a first grid image corresponding to the first power grid image and a second grid image corresponding to the second power grid image;
and carrying out grid position coding on the second grid image to obtain the position information of the second grid image.
In one embodiment, the feature extraction network is a resnet101 network, and/or the convergence network is a transform network.
In one embodiment, the training method for the recognition model comprises the following steps:
training a preset initial neural network model according to the first sample image set to obtain an initial recognition model; the first sample image set comprises at least two types of sample image subsets, and the number of the first sample images of each sample image subset is greater than a second threshold value;
adding the second sample image into the first sample image set to obtain a second sample image set; the number of second sample images is less than a first threshold;
and training the initial recognition model according to the second sample image set to obtain the recognition model.
In one embodiment, training the initial recognition model according to the second sample image set to obtain a recognition model includes:
in each iterative training process, determining a first sample image to be trained from a second sample image set, and determining a second sample image to be trained from each sample image subset;
and training the initial recognition model according to the first sample image to be trained and each second sample image to be trained until a preset convergence condition is met to obtain the recognition model.
In a second aspect, the application further provides a small sample image defect detection device based on meta-learning. The above-mentioned device includes:
the first acquisition module is used for performing data amplification processing on a plurality of power grid images to be detected to obtain a power grid image set; the power grid image set comprises at least two different types of power grid image subsets, and the number of the images of the power grid images is smaller than a first threshold value;
the determining module is used for determining a first power grid image from the power grid image set and determining a second power grid image from each power grid image subset;
the second acquisition module is used for inputting the first power grid image and each second power grid image into a preset identification model for feature extraction and defect detection to obtain a defect detection result; the defect detection result includes the location and type of the defect.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the following steps when executing the computer program:
carrying out data amplification processing on a plurality of power grid images to be detected to obtain a power grid image set; the power grid image set comprises at least two different types of power grid image subsets, and the number of the images of the power grid images is smaller than a first threshold value;
determining a first power grid image from the power grid image set, and determining a second power grid image from each power grid image subset;
inputting the first power grid image and each second power grid image into a preset identification model for feature extraction and defect detection to obtain a defect detection result; the defect detection result includes the location and type of the defect.
In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
carrying out data amplification processing on a plurality of power grid images to be detected to obtain a power grid image set; the power grid image set comprises at least two different types of power grid image subsets, and the number of images of a plurality of power grid images is smaller than a first threshold value;
determining a first power grid image from the power grid image set, and determining a second power grid image from each power grid image subset;
inputting the first power grid image and each second power grid image into a preset identification model for feature extraction and defect detection to obtain a defect detection result; the defect detection result includes the location and type of the defect.
In a fifth aspect, the present application further provides a computer program product. The computer program product comprising a computer program which when executed by a processor performs the steps of:
carrying out data augmentation processing on a plurality of power grid images to be detected to obtain a power grid image set; the power grid image set comprises at least two different types of power grid image subsets, and the number of images of a plurality of power grid images is smaller than a first threshold value;
determining a first power grid image from the power grid image set, and determining a second power grid image from each power grid image subset;
inputting the first power grid image and each second power grid image into a preset identification model for feature extraction and defect detection to obtain a defect detection result; the defect detection result includes the location and type of the defect.
According to the small sample image defect detection method, device and equipment based on meta learning, data augmentation processing is carried out according to a plurality of power grid images to be detected to obtain a power grid image set, a first power grid image is determined from the power grid image set, second power grid images are determined from all power grid image subsets, the first power grid image and the second power grid images are input into a preset identification model to carry out feature extraction and defect detection, and a defect detection result is obtained. Compared with the prior art, the technical scheme of the application has the advantages that data augmentation processing is carried out on a plurality of power grid images to be detected, the richness of a power grid image set of a tiny sample is increased, the first power grid image and each second power grid image are determined based on the power grid image set after the data augmentation processing, the first power grid image and each second power grid image are input into the preset identification model to carry out feature extraction and defect detection, so that a defect detection result is obtained, the defect position and the type of the small sample image to be detected are detected by adopting the preset identification model and a common sample augmentation mode, and the accuracy of detecting the defect of the small sample image is improved.
Drawings
FIG. 1 is a diagram illustrating an exemplary application of a method for defect detection of small sample images based on meta-learning;
FIG. 2 is a flowchart illustrating a method for defect detection of small sample images based on meta-learning according to an embodiment;
FIG. 3 is a flowchart illustrating the steps of obtaining defect detection results according to one embodiment;
FIG. 4 is a flowchart illustrating a method for obtaining defect detection results in another embodiment;
FIG. 5 is a schematic diagram of a process for determining location information of a first grid image and a second grid image according to an embodiment;
FIG. 6 is a flow diagram illustrating the determination of a recognition model in one embodiment;
FIG. 7 is a schematic flow chart illustrating the determination of a recognition model in one embodiment;
FIG. 8 is a flowchart illustrating a method for defect detection of small sample images based on meta-learning according to an embodiment;
FIG. 9 is a schematic flow chart of recognition model training in one embodiment;
FIG. 10 is a block diagram illustrating an exemplary embodiment of a defect detection apparatus for small sample images based on meta-learning;
FIG. 11 is a block diagram of an embodiment of a small sample image defect detection apparatus based on meta-learning;
fig. 12 is a block diagram of a defect detection apparatus for small sample images based on meta learning according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In the defect detection field of the power grid, the unmanned aerial vehicle inspection technology of the power grid overhead line basically and comprehensively replaces the traditional manual inspection mode, and the problems of large workload and long time consumption exist in the process of manually inspecting images after inspection to find defects, so that the intelligent image defect detection technology is realized.
The insulator is an electronic component for supporting and fixing the bus bar and the live conductor and keeping a sufficient distance between the live conductors or between the live conductor and the ground. The insulator has sufficient electrical insulation strength and moisture resistance. Insulators are generally installed between conductors of different potentials or between a conductor and ground potential, and are able to withstand the dual effects of voltage and mechanical stress. The insulator is a special insulating control part and can play an important role in an overhead transmission line, once a defect occurs in an insulator part, the problem of power supply can be caused due to the fact that the fault of the transmission line occurs, however, the M pin of the insulator is lost, the problem that a target is small and samples are few exists, the practical application of the existing image intelligent defect detection technology is difficult to achieve when the M pin of the insulator is lost, and therefore the accuracy of the image intelligent defect detection technology is low.
The following briefly describes an implementation environment related to a small sample image defect detection method based on meta learning provided in an embodiment of the present application. The small sample image defect detection method based on meta learning provided by the embodiment of the application can be applied to the application environment shown in fig. 1. The computer device comprises a processor and a memory connected by a system bus, wherein a computer program is stored in the memory, and the steps of the method embodiments described below can be executed when the processor executes the computer program. Optionally, the computer device may further comprise a network interface, a display screen and an input device. The computer equipment can perform data augmentation processing on the image to be detected, and can also perform feature extraction and defect detection on the image to be detected through a preset identification model, so that a defect detection result of the image to be detected is obtained. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a nonvolatile storage medium storing an operating system and a computer program, and an internal memory. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. Optionally, the computer device may be a server, a personal computer, a personal digital assistant, other terminal devices such as a tablet computer, a mobile phone, and the like, or a cloud or a remote server, and the specific form of the computer device is not limited in the embodiment of the present application.
In one embodiment, as shown in fig. 2, a method for detecting defects of small sample images based on meta-learning is provided, which is described by taking the method as an example applied to the computer device in fig. 1, and includes the following steps:
s201, performing data augmentation processing on a plurality of power grid images to be detected to obtain a power grid image set; the power grid image set comprises at least two different types of power grid image subsets, and the number of images of the power grid images is smaller than a first threshold value.
The power grid images to be detected can be images acquired after the unmanned aerial vehicle patrols the power grid overhead line, the number of the images of the power grid images to be detected is smaller than a first threshold value, for example, the power grid images to be detected are images with a small sample size, that is, the power grid images are small sample images, for example, the power grid images can be images including M pin missing of insulators, and can also be images including types such as self-explosion defects of the insulators. The first threshold may be a threshold statistically obtained according to the number of images with a smaller sample size. The power grid image set includes the small-sample power grid image, and the power grid image set may include some power grid images with a larger sample size, for example, the power grid image set may further include large-sample power grid images with different defect types, and the large-sample power grid images with different defect types may include a power grid line disconnection defect image, a power grid line corrosion defect image, and the like, which is not limited in this embodiment of the application. The power grid images with the same defect type are divided into one power grid image subset, that is, a plurality of power grid image subsets can be divided according to the defect type of the power grid images.
In the embodiment of the present application, the data augmentation processing method may include, but is not limited to, processing methods such as contrast adjustment, histogram equalization, random noise, sharpening, and flipping. Optionally, data augmentation processing may be performed on a plurality of power grid images to be detected, and the size of each power grid image after the data augmentation processing is uniformly scaled to a specific size, so that the power grid images with uniform size are obtained and classified into a power grid image set. It should be noted that, the size of each grid image after the data augmentation processing is uniformly scaled to a specific size, for example, the size of the width and the height of the image may be uniformly 672 × 480.
S202, determining a first power grid image from the power grid image set, and determining a second power grid image from each power grid image subset.
The first grid image may be a grid image randomly extracted from the grid image set obtained in S202. Each grid image subset may be determined according to a specific rule according to the grid image set, and it should be noted that the specific rule may be according to the type of the defect in the grid image, for example, an image of the grid image as a grid line disconnection defect is classified into a first grid image subset, and an image of the grid image as a grid line corrosion defect is classified into a second grid image subset. The second grid image may be determined by randomly drawing a grid image from each subset of grid images.
In this embodiment of the present application, one first power grid image may be determined from the power grid image set obtained in step S202, and a plurality of second power grid images may be determined from each power grid image subset. For example, assuming that N grid images are shared in the grid image set, one of the N grid images may be randomly selected as a first grid image, the N grid images are divided into M types of grid images according to defect types, one grid image is respectively extracted from the M types of grid images, and M grid images are collectively extracted as a second grid image.
S203, inputting the first power grid image and each second power grid image into a preset identification model for feature extraction and defect detection to obtain a defect detection result; the defect detection result includes the location and type of the defect.
The preset identification model may be a neural network model, and the feature extraction may be to extract features of the grid image, such as texture, edge, color, and the like, through the preset identification model, which is not limited in the embodiment of the present application.
In this embodiment of the application, the first power grid image and each second power grid image obtained in S202 may be input into a preset recognition model for feature extraction, and defect detection may be performed according to the extracted features of the first power grid image and the features of each second power grid image, so as to obtain a defect detection result. Wherein the defect detection result comprises the position and the type of the defect.
According to the small sample image defect detection method based on meta-learning, data amplification processing is carried out on a plurality of power grid images to be detected to obtain a power grid image set, a first power grid image is determined from the power grid image set, second power grid images are determined from each power grid image subset, the first power grid image and the second power grid images are input into a preset identification model to carry out feature extraction and defect detection, and a defect detection result is obtained. Compared with the prior art, the technical scheme of the application carries out data augmentation processing on a plurality of power grid images to be detected, the richness of a power grid image set of a tiny sample is increased, the first power grid image and each second power grid image are determined based on the power grid image set after the data augmentation processing, the first power grid image and each second power grid image are input into a preset identification model to carry out feature extraction and defect detection, a defect detection result is obtained, the defect position and the type of the tiny sample image to be detected are detected by adopting the preset identification model and a common sample augmentation mode, and therefore the accuracy of detecting defects of the tiny sample image is improved.
In an embodiment, on the basis of the embodiment shown in fig. 2, the present embodiment describes in detail the process of obtaining the defect detection result, and as shown in fig. 3, the step S203 "inputs the first power grid image and each second power grid image into a preset identification model for feature extraction and defect detection to obtain the defect detection result", may include:
s301, inputting the first power grid image and each second power grid image into a feature extraction network for feature extraction to obtain a first image feature of the first power grid image and a second image feature of each second power grid image.
In the embodiment of the present application, the recognition model mentioned in the above embodiment may include a feature extraction network and a fusion network. The feature extraction network can be a resnet network, an overfeat network, an alexnet network, and the like. In the embodiment of the application, optionally, the feature extraction network selects a ResNet101 network, and the ResNet101 network adopts a 101-layer depth residual error network ResNet-101 on the basis of a pre-training model, so that features of an original image can be rapidly extracted by utilizing the extremely strong feature extraction capability of the ResNet101 network, and the original image can be classified. The fusion network can be a Transformer network, compared with other types of neural networks, the Transformer network can perform parallel computation so as to shorten the computation time, and the Transformer network can also reduce the computation operation number by not increasing the required operation number along with the extension of the operation distance when computing the association between two positions. The first image feature and the second image feature may be features such as texture, edge, and color features of the grid image.
In this embodiment of the application, the first power grid image and each second power grid image acquired in S202 may be input to a feature extraction network for feature extraction, a first image feature of the first power grid image is obtained according to the feature extracted from the first power grid image, and a second image feature of the second power grid image is obtained according to the feature extracted from the second power grid image. It should be noted that, in the embodiment of the present application, the first power grid image and each second power grid image may be sequentially input to the feature extraction network for feature extraction, or the first power grid image and each second power grid image may be simultaneously input to the feature extraction network for feature extraction.
S302, inputting the first image characteristics and the second image characteristics into a fusion network for characteristic fusion and defect detection to obtain a defect detection result.
The feature fusion may be to fuse the first image feature and the position feature of the first image, or may be to fuse the second image feature and the position feature of the second image. The defect detection can be that after the first image characteristics and the second image characteristics are subjected to characteristic fusion, the fused characteristics are detected through a fusion network to obtain information such as defect positions and defect types of the to-be-detected power grid images.
In this embodiment, the first image feature and each second image feature obtained in S301 may be input into a fusion network for feature fusion, and defect detection may be performed on the new first image feature after feature fusion and the new second image feature after feature fusion, so as to obtain a defect detection result. Alternatively, the new first image feature after feature fusion and the new second image feature after feature fusion may be fused again, and defect detection may be performed on the finally fused image feature, thereby obtaining a defect detection result.
According to the method for obtaining the defect detection result, the first power grid image and each second power grid image are input into the feature extraction network for feature extraction, so that the first image feature of the first power grid image and the second image feature of each second power grid image are obtained, and the first image feature and each second image feature are input into the fusion network for feature fusion and defect detection, so that the defect detection result is obtained. Because the first image features are extracted by the first power grid image through the resnet101 network with relatively strong feature extraction capability, and the second image features are extracted by the second power grid image through the resnet101 network with relatively strong feature extraction capability, the obtained first image features and the second image features are more accurate, further, the first image features and the second image features are input into the fusion network for feature fusion and defect detection, and the obtained defect detection result is more accurate by adopting the fusion features.
In an embodiment, on the basis of the embodiment shown in fig. 3, as shown in fig. 4, the embodiment describes in detail the execution steps of the converged network, where the converged network is configured to execute the following steps:
s401, fusing the first image characteristic and the position information of the first power grid image to obtain a first fusion characteristic.
The position information of the first grid image may be position information of a point defined by an infinitesimal area on the first grid image. For example, the first grid image may be subjected to gridding processing to encode grid position information to obtain position information of the first grid image, or each pixel position of the first grid image may be encoded to obtain position information of the first grid image.
In this embodiment, the first image feature obtained in S301 and the position information of the first power grid image may be fused, and the fused feature may be used as the first fusion feature.
S402, fusing the second image characteristic, the position information of the second power grid image and the task identifier to obtain a second fusion characteristic.
The position information of the second grid image may be position information of a point defined by an infinitesimal area on any one second grid image. For example, the second grid image may be subjected to gridding processing to encode grid position information to obtain position information of the second grid image, or each pixel position of the second grid image may be encoded to obtain position information of the second grid image. The task identifier of the second grid image may be the identifier of each second grid image after the second grid images are sequentially identified, for example, assuming that there are 5 second grid images, the identifiers of the 5 second grid images may be sequentially identified according to the numerical sequence, and may be 1, 2, 3, 4, and 5, then the task identifier of the first second grid image may be 1, and the task identifier of the third second grid image may also be 3.
In this embodiment, the second image feature obtained in S301 and the position information and the task identifier of the second power grid image may be fused, and the fused feature may be used as a second fusion feature.
And S403, detecting the defect according to the first fusion characteristic and the second fusion characteristic to obtain a defect detection result.
In this embodiment of the application, the defect detection may be performed on the first fusion feature obtained in S401 and the second fusion feature obtained in S402, and the detection result may be used as a final defect detection result.
The execution step of the fusion network provided by the embodiment of the application is to fuse the first image feature and the position information of the first power grid image to obtain a first fusion feature, fuse the second image feature, the position information of the second power grid image and the task identifier to obtain a second fusion feature, and perform defect detection according to the first fusion feature and the second fusion feature to obtain a defect detection result. The first fusion characteristic is obtained according to the first image characteristic, the position information of the first power grid image, the second fusion characteristic is obtained according to the second image characteristic, the position information of the second power grid image and the task identifier, so that the fusion characteristic not only comprises the characteristics such as the color of the image and the like, but also comprises the position information of each point in the image, the determination of the fusion characteristic is more accurate, further, defect detection is carried out based on the first fusion characteristic and the second fusion characteristic, a defect detection result is obtained, and the position of the defect is more accurate.
In an embodiment, based on the embodiment shown in fig. 4, as shown in fig. 5, the method further includes:
s501, performing grid division on the first power grid image and the second power grid image to obtain a first grid image corresponding to the first power grid image and a second grid image corresponding to the second power grid image.
The grid division may be orthogonal division of the image according to a rectangular coordinate axis. The grid image may be a grid image obtained by orthogonally dividing an image according to a rectangular coordinate axis.
In the embodiment of the application, the first grid image may be subjected to meshing to obtain a plurality of first grid images subjected to meshing, and each second grid image may also be subjected to meshing to obtain a plurality of second grid images subjected to meshing.
S502, grid position coding is carried out on the first grid image to obtain position information of the first grid image, and grid position coding is carried out on the second grid image to obtain position information of the second grid image.
The grid position coding may be to perform position coding on each of the plurality of first grid images acquired in S501.
In this embodiment of the application, the grid position coding may be performed on any one of the first grid images obtained in S501, the position code obtained after the position coding of the first grid image is used as the position information of the first grid image, all the first grid images are traversed, and the position information of all the first grid images is obtained, where the position information of all the first grid images is the position information of the first grid image.
In this embodiment of the application, the grid position coding may be performed on any one of the second grid images obtained in S501, the position code obtained after the position coding of the second grid image is used as the position information of the second grid image, all the second grid images are traversed, and the position information of all the second grid images is obtained, where the position information of all the second grid images is the position information of the second grid image.
The method for determining the position information of the power grid image, provided by the embodiment of the application, includes the steps of performing grid division on a first power grid image and a second power grid image to obtain a first grid image corresponding to the first power grid image and a second grid image corresponding to the second power grid image, performing grid position coding on the first grid image to obtain the position information of the first power grid image, and performing grid position coding on the second grid image to obtain the position information of the second power grid image. The image is subjected to grid division, the position information of each grid image is determined according to the grid, and the position information of the image is further determined according to the position information of each grid image, so that the determined position information of the image is more accurate, the determination of the fusion characteristics is more accurate, and the determined position of the defect is more accurate.
In an embodiment, on the basis of the embodiment shown in fig. 5, as shown in fig. 6, the embodiment describes in detail the training method of the recognition model, and the training method of the recognition model includes:
s601, training a preset initial neural network model according to the first sample image set to obtain an initial recognition model; the first sample image set comprises at least two types of sample image subsets, and the number of the first sample images of each sample image subset is larger than a second threshold value.
The first sample image set includes at least two types of sample image subsets, and the number of the first sample images of each sample image subset may be greater than the second threshold, that is, the number of the first sample images of each sample image subset may be greater.
In the embodiment of the application, a preset initial neural network model can be trained according to the first sample image set, and an initial image recognition model is obtained after training. For example, an image is arbitrarily extracted from a first sample image set to serve as a first power grid image, a second power grid image is randomly extracted from each subset of the first sample image to serve as a second power grid image, and the first power grid image and the second power grid images are input into an initial neural network for iteration each time until the neural network converges to obtain a model of the initial recognition model.
S602, adding a second sample image into the first sample image set to obtain a second sample image set; the number of second sample images is less than the first threshold.
The number of the second sample images may be the number of new category images of small samples that are extremely difficult to obtain, and the number of the second sample images may be smaller than the first threshold, and it should be noted that the second sample images are extremely small samples.
In the embodiment of the present application, a second sample image is added to the first sample set, so as to obtain a second sample image set. For example, assuming that the number of the second sample images is k, the first sample image may be divided into 2 sample image subsets according to image features, and k sample images are extracted from each sample image subset of the first sample image set, and form 3k second sample image sets together with k sample images of the second sample image. For another example, assuming that the number of the second sample images is k, the first sample image may be divided into 4 sample image subsets according to image features, and k sample images are extracted from each sample image subset of the first sample image set, and together with k sample images of the second sample image, the k sample images form a 5k second sample image set.
And S603, training the initial recognition model according to the second sample image set to obtain the recognition model.
The process of training the initial recognition model according to the second sample image set may be that, in each iterative training process, a first sample image to be trained is determined from the second sample image set, a second sample image to be trained is determined from each second sample image subset, the initial recognition model is trained according to the first sample image to be trained and each second sample image to be trained, and the trained model is used as the recognition model.
In this embodiment of the application, on the basis of the above embodiment, as shown in fig. 7, the process of training the initial recognition model according to the second sample image set to obtain the recognition model may include:
s701, in each iterative training process, determining a first sample image to be trained from a second sample image set, and determining a second sample image to be trained from each sample image subset.
The first sample image to be trained may be an image randomly extracted from the second sample image set as the first sample image to be trained. The second sample image to be trained may be a sample image set classified according to a certain characteristic, the images of the same category are classified into a sample image subset, N sample image subsets are provided, one image is randomly extracted from each of the N sample image subsets, and the N extracted images are used as the second sample image to be trained.
S702, training the initial recognition model according to the first sample image to be trained and each second sample image to be trained until a preset convergence condition is met, and obtaining the recognition model.
The preset convergence condition may be that the number of iterations reaches a preset number, or that the value of the loss function of the initial recognition model after multiple training reaches a preset value.
In this embodiment of the application, the initial recognition model may be trained according to the first to-be-trained sample image and each second to-be-trained sample image obtained in the above step S701 until a preset convergence condition is satisfied, so as to obtain the recognition model. It should be noted that, when the initial recognition model is trained for the first time, the first sample image to be trained and each second sample image to be trained may be input into the initial recognition model for training, and after the first training is finished, the above steps S701 and S702 are repeated until a preset convergence condition is satisfied, so as to obtain a final recognition model.
Illustratively, experiments prove that after 500 times of training of an initial recognition model and 500 times of training of the recognition model and convergence of a neural network, the recognition model is adopted to recognize 20 images with porcelain insulator M pin missing, the average accuracy of the recognition result is 71%, and the average recall rate is 74%.
According to the determination method of the identification model, the preset initial neural network model is trained according to the first sample image set to obtain the initial identification model, so that the identification model can identify the defects of the large sample power grid image, the second sample image is added into the first sample image set to obtain the second sample image set, the initial identification model is trained according to the second sample image set to obtain the identification model, so that the identification model can accurately identify the defects of the small sample power grid image, and the accuracy of defect detection is improved.
In an embodiment, fig. 8 is a flowchart of a method for detecting a defect in a small sample image based on meta learning according to an embodiment of the present application, where the method may include the following steps:
s801, performing data amplification processing on a plurality of power grid images to be detected to obtain a power grid image set.
S802, determining a first power grid image from the power grid image set, and determining a second power grid image from each power grid image subset.
And S803, inputting the first power grid image and each second power grid image into a feature extraction network for feature extraction to obtain a first image feature of the first power grid image and a second image feature of each second power grid image.
S804, grid division is carried out on the first power grid image and the second power grid image, and a first grid image corresponding to the first power grid image and a second grid image corresponding to the second power grid image are obtained.
And S805, carrying out grid position coding on the first grid image to obtain position information of the first grid image, and carrying out grid position coding on the second grid image to obtain position information of the second grid image.
S806, inputting the first image feature and the position information of the first power grid image into a fusion network for fusion to obtain a first fusion feature.
And S807, inputting the second image characteristic, the position information of the second power grid image and the task identifier into a fusion network for fusion to obtain a second fusion characteristic.
And S808, detecting the defect according to the first fusion characteristic and the second fusion characteristic to obtain a defect detection result.
As shown in fig. 9, the training method for identifying a model may include:
s901, training a preset initial neural network model according to the first sample image set to obtain an initial recognition model.
And S902, adding the second sample image to the first sample image set to obtain a second sample image set.
And S903, in each iterative training process, determining a first sample image to be trained from the second sample image set, and determining a second sample image to be trained from each sample image subset.
And S904, training the initial recognition model according to the first sample image to be trained and each second sample image to be trained until a preset convergence condition is met, and obtaining the recognition model.
According to the small sample image defect detection method based on meta-learning, data augmentation processing is carried out on a plurality of power grid images to be detected to obtain a power grid image set, a first power grid image is determined from the power grid image set, second power grid images are determined from all power grid image subsets, and the first power grid image and the second power grid images are input into a preset identification model to carry out feature extraction and defect detection to obtain a defect detection result. Compared with the prior art, the technical scheme of the application has the advantages that data augmentation processing is carried out on a plurality of power grid images to be detected, the richness of a power grid image set of a tiny sample is increased, the first power grid image and each second power grid image are determined based on the power grid image set after the data augmentation processing, the first power grid image and each second power grid image are input into the preset identification model to carry out feature extraction and defect detection, so that a defect detection result is obtained, the defect position and the type of the small sample image to be detected are detected by adopting the preset identification model and a common sample augmentation mode, and the accuracy of detecting the defect of the small sample image is improved.
It should be understood that, although the steps in the flowcharts related to the embodiments as described above are sequentially displayed as indicated by arrows, the steps are not necessarily performed sequentially as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the embodiments described above may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the execution order of the steps or stages is not necessarily sequential, but may be rotated or alternated with other steps or at least a part of the steps or stages in other steps.
Based on the same inventive concept, the embodiment of the present application further provides a small sample image defect detection apparatus based on meta learning, for implementing the above-mentioned small sample image defect detection method based on meta learning. The implementation scheme for solving the problem provided by the apparatus is similar to the implementation scheme described in the above method, so that the specific limitations in one or more embodiments of the apparatus for detecting defects in small sample images based on meta-learning provided below may refer to the limitations on the method for detecting defects in small sample images based on meta-learning in the above description, and are not described herein again.
In one embodiment, as shown in fig. 10, there is provided a small sample image defect detecting apparatus based on meta learning, including: a first obtaining module 101, a determining module 102, and a second obtaining module 103, wherein:
the first acquisition module 101 is configured to perform data augmentation processing on multiple power grid images to be detected to obtain a power grid image set; the power grid image set comprises at least two different types of power grid image subsets, and the number of images of a plurality of power grid images is smaller than a first threshold value;
the determining module 102 is configured to determine a first grid image from the grid image set, and determine a second grid image from each grid image subset;
the second obtaining module 103 is configured to input the first power grid image and each second power grid image into a preset identification model for feature extraction and defect detection, so as to obtain a defect detection result; the defect detection result includes the location and type of the defect.
In an embodiment, the recognition model includes a feature extraction network and a fusion network, and the second obtaining module 103 includes: a first acquisition unit and a second acquisition unit. Wherein, the first and the second end of the pipe are connected with each other,
the first acquisition unit is specifically used for inputting the first power grid image and each second power grid image into a feature extraction network for feature extraction to obtain a first image feature of the first power grid image and a second image feature of each second power grid image;
and the second acquisition unit is specifically used for inputting the first image characteristics and the second image characteristics into the fusion network for characteristic fusion and defect detection to obtain a defect detection result.
In one embodiment, the converged network is configured to perform the following steps:
fusing the first image characteristic and the position information of the first power grid image to obtain a first fused characteristic;
fusing the second image characteristic, the position information of the second power grid image and the task identifier to obtain a second fusion characteristic;
and detecting the defects according to the first fusion characteristic and the second fusion characteristic to obtain a defect detection result.
In one embodiment, as shown in fig. 11, the apparatus further comprises:
a dividing module 104, configured to perform mesh division on the first power grid image and the second power grid image to obtain a first mesh image corresponding to the first power grid image and a second mesh image corresponding to the second power grid image;
the encoding module 105 is configured to perform grid position encoding on the first grid image to obtain position information of the first grid image, and perform grid position encoding on the second grid image to obtain position information of the second grid image.
In one embodiment, the feature extraction network is a resnet101 network, and/or the convergence network is a transform network.
In one embodiment, as shown in fig. 12, the above apparatus further comprises:
the first training module 106 is configured to train a preset initial neural network model according to the first sample image set to obtain an initial recognition model; the first sample image set comprises at least two types of sample image subsets, and the number of the first sample images of each sample image subset is greater than a second threshold value;
an adding module 107, configured to add the second sample image to the first sample image set, so as to obtain a second sample image set; the number of second sample images is less than a first threshold;
and the second training module 108 is configured to train the initial recognition model according to the second sample image set, so as to obtain a recognition model.
In one embodiment, the second training module 108 includes:
the determining unit is specifically configured to determine a first sample image to be trained from the second sample image set and determine a second sample image to be trained from each sample image subset in each iterative training process;
and the third obtaining unit is specifically used for training the initial recognition model according to the first sample image to be trained and each second sample image to be trained until a preset convergence condition is met, so as to obtain the recognition model.
The modules in the small sample image defect detection device based on meta learning can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent of a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
carrying out data amplification processing on a plurality of power grid images to be detected to obtain a power grid image set; the power grid image set comprises at least two different types of power grid image subsets, and the number of images of a plurality of power grid images is smaller than a first threshold value;
determining a first power grid image from the power grid image set, and determining a second power grid image from each power grid image subset;
inputting the first power grid image and each second power grid image into a preset identification model for feature extraction and defect detection to obtain a defect detection result; the defect detection result includes the location and type of the defect.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
inputting the first power grid image and each second power grid image into a feature extraction network for feature extraction to obtain a first image feature of the first power grid image and a second image feature of each second power grid image;
and inputting the first image characteristics and each second image characteristic into a fusion network for characteristic fusion and defect detection to obtain a defect detection result.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
fusing the first image characteristic and the position information of the first power grid image to obtain a first fused characteristic;
fusing the second image characteristic, the position information of the second power grid image and the task identifier to obtain a second fusion characteristic;
and performing defect detection according to the first fusion characteristic and the second fusion characteristic to obtain a defect detection result.
In one embodiment, the processor when executing the computer program further performs the steps of:
performing grid division on the first power grid image and the second power grid image to obtain a first grid image corresponding to the first power grid image and a second grid image corresponding to the second power grid image;
and carrying out grid position coding on the second grid image to obtain the position information of the second grid image.
In one embodiment, the processor when executing the computer program further implements the following:
the feature extraction network is a resnet101 network, and/or the fusion network is a Transformer network.
In one embodiment, the processor when executing the computer program further performs the steps of:
training a preset initial neural network model according to the first sample image set to obtain an initial recognition model; the first sample image set comprises at least two types of sample image subsets, and the number of the first sample images of each sample image subset is greater than a second threshold value;
adding the second sample image into the first sample image set to obtain a second sample image set; the number of second sample images is less than a first threshold;
and training the initial recognition model according to the second sample image set to obtain the recognition model.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
in each iterative training process, determining a first sample image to be trained from a second sample image set, and determining a second sample image to be trained from each sample image subset;
and training the initial recognition model according to the first sample image to be trained and each second sample image to be trained until a preset convergence condition is met, and obtaining the recognition model.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
carrying out data amplification processing on a plurality of power grid images to be detected to obtain a power grid image set; the power grid image set comprises at least two different types of power grid image subsets, and the number of images of a plurality of power grid images is smaller than a first threshold value;
determining a first power grid image from the power grid image set, and determining a second power grid image from each power grid image subset;
inputting the first power grid image and each second power grid image into a preset identification model for feature extraction and defect detection to obtain a defect detection result; the defect detection result includes the location and type of the defect.
In one embodiment, the computer program when executed by the processor further performs the steps of:
inputting the first power grid image and each second power grid image into a feature extraction network for feature extraction to obtain a first image feature of the first power grid image and a second image feature of each second power grid image;
and inputting the first image characteristics and the second image characteristics into a fusion network for characteristic fusion and defect detection to obtain a defect detection result.
In one embodiment, the computer program when executed by the processor further performs the steps of:
fusing the first image characteristic and the position information of the first power grid image to obtain a first fused characteristic;
fusing the second image characteristic, the position information of the second power grid image and the task identifier to obtain a second fused characteristic;
and detecting the defects according to the first fusion characteristic and the second fusion characteristic to obtain a defect detection result.
In one embodiment, the computer program when executed by the processor further performs the steps of:
performing grid division on the first power grid image and the second power grid image to obtain a first grid image corresponding to the first power grid image and a second grid image corresponding to the second power grid image;
and carrying out grid position coding on the second grid image to obtain the position information of the second grid image.
In one embodiment, the computer program when executed by the processor further implements the following:
the feature extraction network is a resnet101 network, and/or the fusion network is a Transformer network.
In one embodiment, the computer program when executed by the processor further performs the steps of:
training a preset initial neural network model according to the first sample image set to obtain an initial recognition model; the first sample image set comprises at least two types of sample image subsets, and the number of the first sample images of each sample image subset is greater than a second threshold value;
adding the second sample image into the first sample image set to obtain a second sample image set; the number of second sample images is less than a first threshold;
and training the initial recognition model according to the second sample image set to obtain the recognition model.
In one embodiment, the computer program when executed by the processor further performs the steps of:
in each iterative training process, determining a first sample image to be trained from a second sample image set, and determining a second sample image to be trained from each sample image subset;
and training the initial recognition model according to the first sample image to be trained and each second sample image to be trained until a preset convergence condition is met to obtain the recognition model.
In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, performs the steps of:
carrying out data amplification processing on a plurality of power grid images to be detected to obtain a power grid image set; the power grid image set comprises at least two different types of power grid image subsets, and the number of images of a plurality of power grid images is smaller than a first threshold value;
determining a first power grid image from the power grid image set, and determining a second power grid image from each power grid image subset;
inputting the first power grid image and each second power grid image into a preset identification model for feature extraction and defect detection to obtain a defect detection result; the defect detection result includes the location and type of the defect.
In one embodiment, the computer program when executed by the processor further performs the steps of:
inputting the first power grid image and each second power grid image into a feature extraction network for feature extraction to obtain a first image feature of the first power grid image and a second image feature of each second power grid image;
and inputting the first image characteristics and the second image characteristics into a fusion network for characteristic fusion and defect detection to obtain a defect detection result.
In one embodiment, the computer program when executed by the processor further performs the steps of:
fusing the first image characteristic and the position information of the first power grid image to obtain a first fused characteristic;
fusing the second image characteristic, the position information of the second power grid image and the task identifier to obtain a second fusion characteristic;
and detecting the defects according to the first fusion characteristic and the second fusion characteristic to obtain a defect detection result.
In one embodiment, the computer program when executed by the processor further performs the steps of:
performing grid division on the first power grid image and the second power grid image to obtain a first grid image corresponding to the first power grid image and a second grid image corresponding to the second power grid image;
and carrying out grid position coding on the second grid image to obtain the position information of the second grid image.
In one embodiment, the computer program when executed by the processor further implements:
the feature extraction network is a resnet101 network, and/or the fusion network is a Transformer network.
In one embodiment, the computer program when executed by the processor further performs the steps of:
training a preset initial neural network model according to the first sample image set to obtain an initial recognition model; the first sample image set comprises at least two types of sample image subsets, and the number of the first sample images of each sample image subset is greater than a second threshold value;
adding the second sample image into the first sample image set to obtain a second sample image set; the number of second sample images is less than a first threshold;
and training the initial recognition model according to the second sample image set to obtain the recognition model.
In one embodiment, the computer program when executed by the processor further performs the steps of:
in each iterative training process, determining a first sample image to be trained from a second sample image set, and determining a second sample image to be trained from each sample image subset;
and training the initial recognition model according to the first sample image to be trained and each second sample image to be trained until a preset convergence condition is met, and obtaining the recognition model.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), magnetic Random Access Memory (MRAM), ferroelectric Random Access Memory (FRAM), phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others. The databases involved in the embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.
All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims (10)

1. A small sample image defect detection method based on meta-learning is characterized by comprising the following steps:
carrying out data amplification processing on a plurality of power grid images to be detected to obtain a power grid image set; the power grid image set comprises at least two different types of power grid image subsets, and the number of the images of the power grid images is smaller than a first threshold value;
determining a first power grid image from the power grid image set, and determining a second power grid image from each power grid image subset;
inputting the first power grid image and each second power grid image into a preset identification model for feature extraction and defect detection to obtain a defect detection result; the defect detection result includes a location and a type of the defect.
2. The method according to claim 1, wherein the recognition model includes a feature extraction network and a fusion network, and the inputting the first power grid image and each of the second power grid images into a preset recognition model for feature extraction and defect detection to obtain a defect detection result includes:
inputting the first power grid image and each second power grid image into the feature extraction network for feature extraction to obtain a first image feature of the first power grid image and a second image feature of each second power grid image;
and inputting the first image characteristics and the second image characteristics into the fusion network for characteristic fusion and defect detection to obtain the defect detection result.
3. The method of claim 2, wherein the converged network is configured to perform the steps of:
fusing the first image characteristic and the position information of the first power grid image to obtain a first fusion characteristic;
fusing the second image characteristic, the position information of the second power grid image and the task identifier to obtain a second fused characteristic;
and detecting defects according to the first fusion characteristic and the second fusion characteristic to obtain the defect detection result.
4. The method of claim 3, further comprising:
performing grid division on the first power grid image and the second power grid image to obtain a first grid image corresponding to the first power grid image and a second grid image corresponding to the second power grid image;
and carrying out grid position coding on the first grid image to obtain the position information of the first grid image, and carrying out grid position coding on the second grid image to obtain the position information of the second grid image.
5. The method according to any one of claims 2 to 4, wherein the feature extraction network is a resnet101 network, and/or wherein the converged network is a transform network.
6. The method according to any one of claims 1 to 4, wherein the training method of the recognition model comprises:
training a preset initial neural network model according to the first sample image set to obtain an initial recognition model; the first sample image set comprises at least two types of sample image subsets, and the number of the first sample images of each sample image subset is greater than a second threshold value;
adding a second sample image to the first sample image set to obtain a second sample image set; the number of second sample images is less than the first threshold;
and training the initial recognition model according to the second sample image set to obtain the recognition model.
7. The method of claim 6, wherein the training the initial recognition model according to the second sample image set to obtain the recognition model comprises:
in each iterative training process, determining a first sample image to be trained from the second sample image set, and determining a second sample image to be trained from each sample image subset;
and training the initial recognition model according to the first sample image to be trained and each second sample image to be trained until a preset convergence condition is met, so as to obtain the recognition model.
8. A small sample image defect detection apparatus based on meta-learning, the apparatus comprising:
the first acquisition module is used for performing data amplification processing on a plurality of power grid images to be detected to obtain a power grid image set; the power grid image set comprises at least two different types of power grid image subsets, and the number of the images of the power grid images is smaller than a first threshold value;
the determining module is used for determining a first power grid image from the power grid image set and determining a second power grid image from each power grid image subset;
the second acquisition module is used for inputting the first power grid image and each second power grid image into a preset identification model for feature extraction and defect detection to obtain a defect detection result; the defect detection result includes a location and a type of the defect.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
CN202211446408.2A 2022-11-18 2022-11-18 Small sample image defect detection method, device and equipment based on meta-learning Active CN115564773B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211446408.2A CN115564773B (en) 2022-11-18 2022-11-18 Small sample image defect detection method, device and equipment based on meta-learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211446408.2A CN115564773B (en) 2022-11-18 2022-11-18 Small sample image defect detection method, device and equipment based on meta-learning

Publications (2)

Publication Number Publication Date
CN115564773A true CN115564773A (en) 2023-01-03
CN115564773B CN115564773B (en) 2023-04-18

Family

ID=84770444

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211446408.2A Active CN115564773B (en) 2022-11-18 2022-11-18 Small sample image defect detection method, device and equipment based on meta-learning

Country Status (1)

Country Link
CN (1) CN115564773B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115908988A (en) * 2023-03-09 2023-04-04 苏州苏映视图像软件科技有限公司 Defect detection model generation method, device, equipment and storage medium
CN116205916A (en) * 2023-04-28 2023-06-02 南方电网数字电网研究院有限公司 Method and device for detecting defects of small electric power sample, computer equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019233166A1 (en) * 2018-06-04 2019-12-12 杭州海康威视数字技术股份有限公司 Surface defect detection method and apparatus, and electronic device
US20210374940A1 (en) * 2019-12-30 2021-12-02 Goertek Inc. Product defect detection method, device and system
WO2022036953A1 (en) * 2020-08-19 2022-02-24 上海商汤智能科技有限公司 Defect detection method and related apparatus, device, storage medium, and computer program product
CN114663729A (en) * 2022-03-29 2022-06-24 南京工程学院 Cylinder sleeve small sample defect detection method based on meta-learning

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019233166A1 (en) * 2018-06-04 2019-12-12 杭州海康威视数字技术股份有限公司 Surface defect detection method and apparatus, and electronic device
US20210374940A1 (en) * 2019-12-30 2021-12-02 Goertek Inc. Product defect detection method, device and system
WO2022036953A1 (en) * 2020-08-19 2022-02-24 上海商汤智能科技有限公司 Defect detection method and related apparatus, device, storage medium, and computer program product
CN114663729A (en) * 2022-03-29 2022-06-24 南京工程学院 Cylinder sleeve small sample defect detection method based on meta-learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
郭子粳: "基于元学习框架的小样本钢材表面缺陷检测算法研究", 《万方学位论文全文数据库》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115908988A (en) * 2023-03-09 2023-04-04 苏州苏映视图像软件科技有限公司 Defect detection model generation method, device, equipment and storage medium
CN115908988B (en) * 2023-03-09 2023-05-30 苏州苏映视图像软件科技有限公司 Defect detection model generation method, device, equipment and storage medium
CN116205916A (en) * 2023-04-28 2023-06-02 南方电网数字电网研究院有限公司 Method and device for detecting defects of small electric power sample, computer equipment and storage medium
CN116205916B (en) * 2023-04-28 2023-09-15 南方电网数字电网研究院有限公司 Method and device for detecting defects of small electric power sample, computer equipment and storage medium

Also Published As

Publication number Publication date
CN115564773B (en) 2023-04-18

Similar Documents

Publication Publication Date Title
CN115564773B (en) Small sample image defect detection method, device and equipment based on meta-learning
JP2022103040A (en) Image defect detection method, device, electronic apparatus, storage medium, and product
CN113065525A (en) Age recognition model training method, face age recognition method and related device
CN106897683B (en) Ground object detection method and system of remote sensing image
CN111753729B (en) False face detection method and device, electronic equipment and storage medium
Mikosch et al. The integrated periodogram of a dependent extremal event sequence
CN110992306B (en) Method and device for segmenting deteriorated insulator in infrared image based on deep learning
CN117036855A (en) Object detection model training method, device, computer equipment and storage medium
Lin et al. Geometrical flow‐guided fast beamlet transform for crack detection
CN116579998A (en) Defect detection method, device, equipment, medium and program product of transformer equipment
CN115953578A (en) Method and device for evaluating inference result of fracture semantic segmentation model
CN116205916B (en) Method and device for detecting defects of small electric power sample, computer equipment and storage medium
Peng et al. Volatility-Based Measurements Allocation for Distributed Data Storage in Mobile Crowd Sensing
CN113762120A (en) Insulator image segmentation method and device, electronic equipment and storage medium
CN113989632A (en) Bridge detection method and device for remote sensing image, electronic equipment and storage medium
CN114638304A (en) Training method of image recognition model, image recognition method and device
CN112862002A (en) Training method of multi-scale target detection model, target detection method and device
CN113628089A (en) Image processing method, image processing device, storage medium and computer equipment
CN114397306B (en) Power grid grading ring hypercomplex category defect multi-stage model joint detection method
CN113259302B (en) Relationship decomposition method and device for network attack data and computer equipment
CN117372348A (en) Defect analysis method, device, computer equipment and storage medium thereof
CN117372801A (en) Transformer substation defect anomaly detection method, device and computer equipment
US20230419473A1 (en) Method for detecting product for defects, electronic device, and storage medium
CN117237826A (en) Method, device and equipment for removing defects of power distribution network based on space pose information
CN115830374A (en) Foreign matter detection method and device for transformer substation, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant