CN115439674A - Intelligent image labeling method and device based on power image knowledge graph - Google Patents

Abstract

The invention relates to the technical field of artificial intelligence, and particularly provides an intelligent image labeling method and device based on an electric power image knowledge graph, which comprises the following steps: acquiring image data of an electric power target; performing superpixel segmentation on the image data of the power target, and clustering the superpixel segmentation results; and inputting the clustering result into a pre-constructed ViT power defect target detection model to obtain a defect diagnosis result of the power target output by the pre-constructed ViT power defect target detection model. According to the technical scheme provided by the invention, the aim of maximum model algorithm iterative training efficiency can be realized with the minimum labeling cost, and the sample labeling efficiency can be greatly improved only by manually checking the automatically labeled data for a short time.

Description

Intelligent image labeling method and device based on power image knowledge graph

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to an intelligent image labeling method and device based on an electric power image knowledge graph.

Background

At present, the commercialization of artificial intelligence reaches the stage maturity in the aspects of computing power, algorithm and technology. The specific pain point of the industry is really solved through the landing of the algorithm and the application, a large amount of original data related to artificial intelligence needs to be collected, algorithm training support is carried out after marking processing, and the data can be said to determine the landing degree of the AI.

With the application of the artificial intelligence image recognition technology in national grid power business, massive business image data are accumulated, but because the image recognition technology depends on a large amount of data labeling support at present, the existing artificial intelligence automatic labeling technology is mostly completed through manpower according to corresponding labeling standards, so that the labor cost is high, the efficiency is low, repeated operation is easy to generate, and the great waste of resources is caused.

Disclosure of Invention

In order to overcome the defects, the invention provides an intelligent image annotation method and device based on an electric power image knowledge graph.

In a first aspect, an intelligent image annotation method based on an electric power image knowledge graph is provided, and the intelligent image annotation method based on the electric power image knowledge graph comprises the following steps:

acquiring image data of an electric power target;

performing superpixel segmentation on the image data of the power target, and clustering the superpixel segmentation results;

and inputting the clustering result into a pre-constructed ViT power defect target detection model to obtain a defect diagnosis result of the power target output by the pre-constructed ViT power defect target detection model.

Preferably, the super-pixel segmentation of the image data of the power target includes:

and performing super-pixel segmentation on the image data of the power target by adopting an SLIC algorithm.

Preferably, the clustering the super-pixel segmentation results includes:

and clustering the super-pixel segmentation result by adopting a DBSCAN algorithm.

Preferably, the obtaining process of the pre-constructed ViT power defect target detection model includes:

acquiring a sample image from a pre-constructed electric power image knowledge graph;

labeling the sample image;

performing superpixel segmentation on the labeled sample image, and clustering the superpixel segmentation result to obtain a first clustering result;

constructing first training data by using the first clustering result and the corresponding labeling result;

and training the initial deep neural network by using the first training data to obtain a pre-constructed ViT power defect target detection model.

Further, after the training of the initial deep neural network by using the training data is performed to obtain a pre-constructed ViT power deficiency target detection model, the method includes:

acquiring a sample image from a pre-constructed electric power image knowledge graph;

performing superpixel segmentation on the sample image, and clustering superpixel segmentation results to obtain a second clustering result;

inputting the second clustering result into the pre-constructed ViT power defect target detection model to obtain a label corresponding to the sample image output by the pre-constructed ViT power defect target detection model;

constructing second training data by using the second clustering result and the corresponding label;

constructing third training data by the first training data and the second training data according to a preset sampling proportion;

and training the pre-constructed ViT power defect target detection model by using the third training data.

Further, the sampling ratio is 0.2.

Further, the algorithm for labeling the sample image is as follows: a rectangular frame marking algorithm, an example segmentation marking algorithm or human skeleton point line marking.

In a second aspect, an intelligent image annotation device based on an electric power image knowledge-graph is provided, which includes:

the acquisition module is used for acquiring image data of the power target;

the processing module is used for carrying out superpixel segmentation on the image data of the power target and clustering superpixel segmentation results;

and the diagnosis module is used for inputting the clustering result to a pre-constructed ViT power defect target detection model to obtain a defect diagnosis result of the power target output by the pre-constructed ViT power defect target detection model.

In a third aspect, a computer device is provided, comprising: one or more processors;

the processor to store one or more programs;

when the one or more programs are executed by the one or more processors, the intelligent image annotation method based on the power image knowledge graph is realized.

One or more technical schemes of the invention at least have one or more of the following beneficial effects:

the invention provides an intelligent image labeling method and device based on an electric power image knowledge graph, which comprises the following steps: acquiring image data of an electric power target; performing super-pixel segmentation on the image data of the power target, and clustering the super-pixel segmentation results; and inputting the clustering result into a pre-constructed ViT power defect target detection model to obtain a defect diagnosis result of the power target output by the pre-constructed ViT power defect target detection model. According to the technical scheme provided by the invention, the aim of maximum model algorithm iterative training efficiency can be realized with the minimum labeling cost, and the sample labeling efficiency can be greatly improved only by manually checking the automatically labeled data for a short time.

Drawings

FIG. 1 is a schematic flow chart illustrating the main steps of an intelligent image annotation method based on an electric power image knowledge graph according to an embodiment of the present invention;

FIG. 2 is an automated annotation flow diagram of an embodiment of the invention;

FIG. 3 is a schematic diagram of the operation mechanism of the self-learning intelligent labeling mode according to the embodiment of the invention;

FIG. 4 is a schematic diagram illustrating an image labeling method according to an embodiment of the present invention;

FIG. 5 is a flowchart illustrating the construction of a power image knowledge-graph according to an embodiment of the invention;

FIG. 6 is an exemplary power image knowledge map of an embodiment of the invention;

FIG. 7 is a diagram illustrating an example of an embodiment of a natural language representation of an electric power knowledge entity in an electric power image knowledge graph;

fig. 8 is a main structural block diagram of an intelligent image annotation device based on an electric power image knowledge graph according to an embodiment of the present invention.

Detailed Description

The following provides a more detailed description of embodiments of the present invention, with reference to the accompanying drawings.

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.

Example 1

Referring to fig. 1, fig. 1 is a flow chart illustrating main steps of an intelligent image annotation method based on an electric power image knowledge graph according to an embodiment of the invention. As shown in fig. 1, the intelligent image annotation method based on power image knowledge graph in the embodiment of the present invention mainly includes the following steps:

step S101: acquiring image data of an electric power target;

step S102: performing superpixel segmentation on the image data of the power target, and clustering the superpixel segmentation results;

step S103: and inputting the clustering result into a pre-constructed ViT power defect target detection model to obtain a defect diagnosis result of the power target output by the pre-constructed ViT power defect target detection model.

In this embodiment, the performing the super-pixel segmentation on the image data of the power target includes:

and performing super-pixel segmentation on the image data of the power target by adopting an SLIC algorithm.

In this embodiment, the clustering the super-pixel segmentation result includes:

and clustering the super-pixel segmentation results by adopting a DBSCAN algorithm.

In one embodiment, by means of the technology of fine screening, labeling and model self-learning of image samples, automatic training and automatic labeling of background models are achieved through a short-term progressive labeling process, the maximum model algorithm iterative training efficiency is achieved with the minimum labeling cost, manual work only needs to conduct short-term verification on data after automatic labeling, and sample labeling efficiency is greatly improved.

The automatic labeling relates to a target detection model, a target to be detected is highlighted by adopting SLIC and DBSCAN modes, target detection is realized by adopting ViT, and the automatic labeling process is shown in figure 2. Firstly, performing super-pixel segmentation on a power image by using an SLIC (Linear segmentation in-line) technology to obtain a super-pixel segmentation result; then clustering the segmentation results by adopting a DBSCAN technology to highlight a target area; and then, identifying the image through ViT to obtain a good identification result.

The operation mechanism of the self-learning intelligent labeling mode is shown in FIG. 3:

based on a model self-learning technology, constructing a pre-labeled model according to the labeled sample after fine screening, automatically judging whether the labeled quantity meets the requirement according to the model index, and automatically performing two rounds of screening work on the residual sample if the labeled quantity does not meet the requirement; and if so, automatically labeling the residual samples. And manually verifying the data marked by the pre-marked model, and continuously repeating the process of self-learning marking of the model by the verified data until the automatic marking meets the marking requirement set manually.

In this embodiment, the process of obtaining the pre-constructed ViT power defect target detection model includes:

acquiring a sample image from a pre-constructed electric power image knowledge graph;

labeling the sample image;

performing superpixel segmentation on the marked sample image, and clustering the superpixel segmentation result to obtain a first clustering result;

constructing first training data by using the first clustering result and the corresponding labeling result;

and training the initial deep neural network by using the first training data to obtain a pre-constructed ViT power defect target detection model.

Further, after the training of the initial deep neural network is performed by using the training data to obtain a pre-constructed ViT power deficiency target detection model, the method includes:

acquiring a sample image from a pre-constructed electric power image knowledge graph;

performing superpixel segmentation on the sample image, and clustering superpixel segmentation results to obtain a second clustering result;

inputting the second clustering result into the pre-constructed ViT power defect target detection model to obtain a label corresponding to the sample image output by the pre-constructed ViT power defect target detection model;

constructing second training data by using the second clustering result and the corresponding label;

constructing third training data by the first training data and the second training data according to a preset sampling proportion;

and training the pre-constructed ViT power defect target detection model by using the third training data.

In the actual implementation process of the patent, the method is a pseudo label method, and a parameter needs to be added in the process: sample ratio (sample _ rate), which represents the ratio of the genuine tag data that would have been used as a fake tag sample, setting sample _ rate to 0.0 means that the model would have used only genuine tag data, while sample _ rate =0.5 means that the model would have used all genuine tag data and half of the fake tag data, in either case, the model would have used all genuine tag data, which in the present invention is set to 0.2, i.e. 20% of the genuine tag data.

In one embodiment, the algorithm for labeling the sample image is as follows: a rectangular frame marking algorithm, an example segmentation marking algorithm or human skeleton point line marking.

In the specific embodiment provided by the invention, based on the labeling mode construction technology of the power image knowledge graph, high-fitness labeling modes (example segmentation, polygon labeling, skeleton point line labeling and the like) are automatically and intelligently screened aiming at the image characteristics (granularity and contour light) of different defect targets, and the most efficient labeling mode is provided for sample pre-labeling.

The labeling method suitable for different image features is shown in fig. 4:

(1) And marking a rectangular frame. The rectangular frame labeling is also called as the pull frame labeling, is an image labeling method which is most widely applied at present, and can rapidly frame a specified target object in image or video data in a relatively simple and convenient mode. The marking method is suitable for marking foreign matters of power components, power transmission lines and the like.

(2) And (5) example segmentation marking. According to the attributes of the power defects, the complex irregular power image is subjected to region division, and corresponding attributes are marked to help the training of an image intelligent recognition model, so that the method is suitable for marking oil stains on the ground and the like.

(3) Marking the human bone dotted line. The skeleton point-line marking refers to marking key points at positions of specified skeleton connecting points of a human body in a manual mode, is commonly used for training a face recognition model and a statistical model, and is suitable for marking of operation site personnel recognition, dangerous action recognition and the like.

In a preferred embodiment, the pre-constructed power image knowledge graph of the present invention is a multi-dimensional high quality knowledge base;

the electric power image knowledge graph construction comprises 5 links of positioning a target knowledge base, extracting concept vocabularies, associating the concept vocabularies, organizing metadata and storing the knowledge base, and is shown in figure 5.

(1) And constructing a positioning target knowledge graph. And determining to belong to a knowledge system in the power field, and positioning to construct a power image knowledge graph.

(2) And extracting concept vocabularies of the power field. And extracting concept words from the meta-knowledge in the power related field, wherein the concept words comprise a plurality of aspects such as image granularity, image outline rules, environment complexity, service scenes, emergency degree and the like.

(3) Associating the concept vocabulary. The associated concept vocabulary is the key of the whole link and directly relates to success or failure of the construction of the knowledge graph. The association is mainly carried out by comparing, analyzing and inducing concept vocabularies in the domain knowledge with the assistance of domain experts and associating according to the objective relation of knowledge and power.

(4) The metadata is organized. The objective defect law is summarized and abstracted by using an electric power field system, including but not limited to a conventional contour, an irregular contour, an infrared image contour, a blur, a shelter, an environment similarity and the like.

(5) And storing the knowledge graph. The last link of establishing the knowledge graph stores the facts, rules, concepts and the like related to the power, wherein the facts are description of basic knowledge of the power and are short-term; the rules are extracted from the experience of experts in the power field, and have long-term property.

By means of the electric power image knowledge graph, multi-dimensional features of different target images are analyzed and summarized, and the different target image features are clustered, merged and graded in multiple aspects such as image granularity, image contour rule degree, environment complexity, scene distinguishing and emergency degree, so that the high-quality electric power image knowledge graph is formed, as shown in fig. 6.

Further, various aspects of the power knowledge entity may be represented by circles, such as, but not limited to, "power," "power transmission," "environmental complexity," etc. in fig. 7; the connecting line in the graph represents the attribute relationship between two connected entities; the two entities and the connecting line are constructed into a knowledge structure, and a plurality of knowledge structures are mutually connected through relationship to form a mesh-shaped power knowledge structure. For example, the power knowledge as represented in fig. 7 may be expressed as "environment complexity" including "fuzzy", "clear", and the like knowledge in natural language.

Example 2

Based on the same inventive concept, the invention also provides an intelligent image annotation device based on the power image knowledge graph, as shown in fig. 8, the intelligent image annotation device based on the power image knowledge graph comprises:

the acquisition module is used for acquiring image data of the power target;

the processing module is used for carrying out superpixel segmentation on the image data of the power target and clustering superpixel segmentation results;

and the diagnosis module is used for inputting the clustering result to a pre-constructed ViT power defect target detection model to obtain a defect diagnosis result of the power target output by the pre-constructed ViT power defect target detection model.

Preferably, the super-pixel segmentation of the image data of the power target includes:

and performing super-pixel segmentation on the image data of the power target by adopting an SLIC algorithm.

Preferably, the clustering the super-pixel segmentation results includes:

and clustering the super-pixel segmentation result by adopting a DBSCAN algorithm.

Preferably, the obtaining process of the pre-constructed ViT power defect target detection model includes:

acquiring a sample image from a pre-constructed electric power image knowledge graph;

labeling the sample image;

performing superpixel segmentation on the labeled sample image, and clustering the superpixel segmentation result to obtain a first clustering result;

constructing first training data by using the first clustering result and the corresponding labeling result;

and training the initial deep neural network by using the first training data to obtain a pre-constructed ViT power defect target detection model.

Further, after the training of the initial deep neural network by using the training data is performed to obtain a pre-constructed ViT power deficiency target detection model, the method includes:

acquiring a sample image from a pre-constructed electric power image knowledge graph;

performing superpixel segmentation on the sample image, and clustering superpixel segmentation results to obtain a second clustering result;

inputting the second clustering result into the pre-constructed ViT power defect target detection model to obtain a label corresponding to the sample image output by the pre-constructed ViT power defect target detection model;

constructing second training data by using the second clustering result and the corresponding label;

constructing third training data by the first training data and the second training data according to a preset sampling proportion;

and training the pre-constructed ViT power defect target detection model by using the third training data.

Further, the sampling ratio is 0.2.

Further, the algorithm for labeling the sample image is as follows: a rectangular frame marking algorithm, an example segmentation marking algorithm or human skeleton point line marking.

Example 3

Based on the same inventive concept, the present invention also provides a computer apparatus comprising a processor and a memory, the memory being configured to store a computer program comprising program instructions, the processor being configured to execute the program instructions stored by the computer storage medium. The Processor may be a Central Processing Unit (CPU), or may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable gate array (FPGA) or other Programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., which is a computing core and a control core of the terminal, and is specifically adapted to implement one or more instructions, and specifically adapted to load and execute one or more instructions in a computer storage medium so as to implement a corresponding method flow or a corresponding function, so as to implement the steps of the image intelligent labeling method based on the power image knowledge graph in the foregoing embodiments.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications and equivalents may be made to the embodiments of the invention without departing from the spirit and scope of the invention, which is to be covered by the claims.

Claims (9)

1. An intelligent image labeling method based on an electric power image knowledge graph is characterized by comprising the following steps:

acquiring image data of an electric power target;

performing superpixel segmentation on the image data of the power target, and clustering the superpixel segmentation results;

and inputting the clustering result into a pre-constructed ViT power defect target detection model to obtain a defect diagnosis result of the power target output by the pre-constructed ViT power defect target detection model.

2. The method of claim 1, wherein the superpixel segmenting image data of the power target comprises:

and performing super-pixel segmentation on the image data of the power target by adopting an SLIC algorithm.

3. The method of claim 1, wherein clustering superpixel segmentation results comprises:

and clustering the super-pixel segmentation results by adopting a DBSCAN algorithm.

4. The method of claim 1, wherein the pre-constructed ViT power deficiency target detection model acquisition process comprises:

acquiring a sample image from a pre-constructed electric power image knowledge graph;

labeling the sample image;

performing superpixel segmentation on the marked sample image, and clustering the superpixel segmentation result to obtain a first clustering result;

constructing first training data by using the first clustering result and the corresponding labeling result;

and training the initial deep neural network by using the first training data to obtain a pre-constructed ViT power defect target detection model.

5. The method of claim 4, wherein the training of the initial deep neural network using the training data, after obtaining the pre-constructed ViT power deficiency target detection model, comprises:

acquiring a sample image from a pre-constructed electric power image knowledge graph;

performing superpixel segmentation on the sample image, and clustering superpixel segmentation results to obtain a second clustering result;

inputting the second clustering result into the pre-constructed ViT power defect target detection model to obtain a label corresponding to the sample image output by the pre-constructed ViT power defect target detection model;

constructing second training data by using the second clustering result and the corresponding label;

constructing third training data by the first training data and the second training data according to a preset sampling proportion;

and training the pre-constructed ViT power defect target detection model by using the third training data.

6. The method of claim 5, wherein the sampling ratio is 0.2.

7. The method of claim 4, wherein the algorithm for labeling the sample image is: a rectangular frame marking algorithm, an example segmentation marking algorithm or human skeleton point line marking.

8. An image intelligent labeling device based on power image knowledge graph, which is characterized in that the device comprises:

the acquisition module is used for acquiring image data of the power target;

the processing module is used for carrying out superpixel segmentation on the image data of the power target and clustering superpixel segmentation results;

and the diagnosis module is used for inputting the clustering result to a pre-constructed ViT power defect target detection model to obtain a defect diagnosis result of the power target output by the pre-constructed ViT power defect target detection model.

9. A computer device, comprising: one or more processors;

the processor to store one or more programs;

the one or more programs, when executed by the one or more processors, implement the method for intelligently labeling images based on power imagery knowledge-graphs of any one of claims 1 to 7.

Images