CN115114448B - Intelligent multi-mode fusion power consumption inspection method, device, system, equipment and medium - Google Patents

Intelligent multi-mode fusion power consumption inspection method, device, system, equipment and medium Download PDF

Info

Publication number
CN115114448B
CN115114448B CN202210626040.1A CN202210626040A CN115114448B CN 115114448 B CN115114448 B CN 115114448B CN 202210626040 A CN202210626040 A CN 202210626040A CN 115114448 B CN115114448 B CN 115114448B
Authority
CN
China
Prior art keywords
hidden danger
data
electrical data
safety
electric
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210626040.1A
Other languages
Chinese (zh)
Other versions
CN115114448A (en
Inventor
潘明明
覃剑
田世明
陈宋宋
袁金斗
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Electric Power Research Institute Co Ltd CEPRI
Original Assignee
China Electric Power Research Institute Co Ltd CEPRI
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Electric Power Research Institute Co Ltd CEPRI filed Critical China Electric Power Research Institute Co Ltd CEPRI
Priority to CN202210626040.1A priority Critical patent/CN115114448B/en
Publication of CN115114448A publication Critical patent/CN115114448A/en
Application granted granted Critical
Publication of CN115114448B publication Critical patent/CN115114448B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
    • G06F16/367—Ontology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G06F40/295—Named entity recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Abstract

The invention provides an intelligent multi-mode fusion electricity inspection method, device, system, equipment and medium, comprising the following steps: acquiring a safety management rule of the electric equipment based on the electric equipment related to the electric safety inspection request through hidden danger identification of context awareness; the invention can combine the electric quantity and the non-electric quantity factors to inquire the safety situation of the electric equipment and effectively detect the potential safety hazard of the electric equipment; the invention adopts a semi-supervised non-electric quantity hidden danger identification method to realize the purpose of model training by combining a small amount of marked data with a large amount of unmarked data, and solves the problem of insufficient data of a non-electric quantity sample of a power consumption scene; the invention applies the knowledge graph technology to carry out operation recommendation aiming at the current electric equipment state and potential safety hazard, and can safely guide the next processing.

Description

Intelligent multi-mode fusion power consumption inspection method, device, system, equipment and medium
Technical Field
The invention relates to the field of power supply and power utilization safety, in particular to an intelligent multi-mode fusion power utilization checking method, device, system, equipment and medium.
Background
The electricity safety inspection is an important means for ensuring the normal operation of the power grid, particularly, the requirements of supporting work for ensuring electricity for major activities are stricter, and the faults of the terminal hidden danger are always important factors threatening the safe operation of the overall electricity supply environment, so that the supporting capability of the electric company business personnel for checking the potential risk of the electricity consumption of the client side is more and more critical.
At present, the research is to disassemble the attribute and the state of a monitoring object, group the alarm information, and extract the minimum characteristic value of the alarm information and the mapping relation between the minimum characteristic value and hidden danger based on the electricity utilization characteristic of the monitoring object and an expert knowledge base; analyzing the characteristics, and gathering scattered alarm information to infer hidden trouble conveyed by the scattered alarm information; the method has the advantages that the fragment information expresses exact meaning, so that the information is more valuable, the supporting effect of the whole process of alarm information monitoring, auxiliary decision-making and fault feedback is provided, but the technology only analyzes the electrical quantity information in the electricity utilization process, can not comprehensively diagnose potential safety hazards possibly existing in electric equipment, and has limitation.
Disclosure of Invention
In order to solve the problems existing in the prior art, the invention provides an intelligent multi-mode fusion power consumption checking method, which comprises the following steps:
when an electrical safety check request is received:
acquiring a safety management rule of electric equipment based on the electric equipment related to the electric safety inspection request through hidden danger identification of context awareness;
according to the request type related to the electricity safety inspection request, analyzing and determining an electricity safety inspection request result aiming at the electric equipment based on a safety management rule and electric data and non-electric data of the electric equipment related to the electricity safety inspection request;
the electricity utilization safety check request comprises a request type, electric equipment, electric data and non-electric data of the electric equipment.
Preferably, when the request type related to the electricity security check request includes: when hidden danger inquiry or operation recommendation is performed, the electrical data and non-electrical data of the electrical equipment related to the electrical safety inspection request are analyzed and determined based on a safety management rule, and an electrical safety inspection request result aiming at the electrical equipment is determined, and the method comprises the following steps:
performing first hidden danger identification based on the electrical data; based on the non-electrical data, performing second hidden danger identification by using a pre-trained hidden danger identification task model; fusing the first hidden danger identification result and the second hidden danger identification result to obtain a hidden danger identification result of the electric equipment;
Based on the electrical data, the non-electrical data, the request type, the safety management rules and the hidden danger identification result, processing by utilizing a pre-constructed knowledge graph to obtain an electricity utilization safety inspection request result corresponding to the request type;
the hidden danger identification task model is obtained by training labeled historical non-electrical data and unlabeled historical non-electrical data through semi-supervised detection learning;
the tag includes: hidden danger information of electric equipment; the hidden trouble information comprises: normal, abnormal, hidden trouble.
Preferably, the training of the hidden danger identification task model includes:
training by taking the historical non-electrical data with the tag as a training set to obtain a hidden danger identification task model;
predicting unlabeled historical non-electrical data by utilizing single-stage target detection or double-stage target detection based on the hidden danger identification task model to obtain hidden danger information, confidence corresponding to the hidden danger information, marking loss corresponding to the labeled historical non-electrical data and unlabeled historical non-electrical data;
taking the weighted sum of the tag loss and the no-tag loss as a consistency constraint loss;
Taking hidden danger information corresponding to the maximum confidence coefficient as a pseudo tag of unlabeled historical non-electrical data, adding a training set to train the task model continuously until the task model reaches the optimal;
the label loss and the label-free loss are calculated by a semi-supervised target detection model.
Preferably, the single-stage object detection process includes:
inputting image data of electric equipment in a training set and the inverted image data into a single-stage classifier to be predicted to obtain hidden danger information and confidence coefficient of the hidden danger information;
respectively carrying out target detection loss calculation on the labeled historical non-electrical data and the unlabeled historical non-electrical data;
the historical non-electrical data are image data corresponding to electric equipment.
Preferably, the dual-stage object detection process includes:
taking image data of electric equipment in a training set and the inverted image data as input data, and training by using a CNN (computer network) to obtain hidden danger information; respectively carrying out target detection loss calculation on the labeled historical non-electrical data and the unlabeled historical non-electrical data; the historical non-electrical data are image data corresponding to electric equipment.
Preferably, the processing, based on the electrical data, the non-electrical data, the request type, the security management procedure and the hidden danger identification result, by using a pre-constructed knowledge graph to obtain an electricity utilization security check request result corresponding to the request type includes:
performing disambiguation processing from the electrical data, the non-electrical data, the safety management rules and the hidden danger identification results by utilizing a pre-constructed knowledge graph;
and carrying out rule judgment based on disambiguated electrical data or non-electrical data in combination with a safety regulation, a hidden danger identification result and an electricity safety logic to obtain an electricity safety inspection request result corresponding to the request type.
Preferably, when the request type is hidden danger inquiry, the electricity safety inspection request result is a hidden danger inquiry result;
when the request type is operation recommendation, the electricity utilization safety check request result is a recommendation operation result corresponding to the elimination of electricity utilization hidden danger.
Preferably, the disambiguating processing from the electrical data, the non-electrical data, the safety management procedure and the hidden danger identification result by using the pre-constructed knowledge graph includes:
extracting entity names from the electrical data, the non-electrical data, the safety management rules and the hidden danger identification results;
And (3) respectively performing local disambiguation and global disambiguation by utilizing a pre-established knowledge graph based on each entity reference, and then determining a corresponding target entity.
Preferably, the determining the corresponding target entity after performing local disambiguation and global disambiguation by using the pre-constructed knowledge graph based on each entity reference includes:
the following operations are performed based on each entity designation:
selecting at least one entity from a pre-constructed knowledge graph as a candidate entity based on the entity index;
obtaining local similarity probability of each entity finger and the candidate entity by using a deep learning method;
determining a global association based on the association between all entity designations and the corresponding candidate entities;
multiplying the local similarity probability of the candidate entity by the global association degree of the candidate entity, taking the candidate entity corresponding to the maximum value as a target entity, and designating the entity as a link with the target entity;
the knowledge graph comprises: and the association relation of a plurality of entities and each entity.
Preferably, when the request type is a status query, the analyzing, based on a safety management rule, electrical data and non-electrical data of the electrical equipment related to the electrical safety inspection request to determine an electrical safety inspection request result for the electrical equipment includes:
Collecting the electric operation state and the non-electric operation state of the electric equipment;
and carrying out rule judgment based on the electric operation state and the non-electric operation state and the electricity safety logic to obtain the electric equipment state.
Based on the same inventive concept, the invention also provides an intelligent multi-mode fusion electricity inspection device, which comprises: a hidden danger identification module and a procedure management module for sensing the context;
the context-aware hidden danger identification module is used for, when receiving an electricity safety check request: invoking the procedure management module and receiving a security management procedure returned by the procedure management module; and the module is further used for calling a corresponding module to execute based on the request type related to the electricity safety check request: analyzing the electrical data and the non-electrical data of the electric equipment related to the electric safety inspection request based on a safety management rule to determine an electric safety inspection request result aiming at the electric equipment;
the regulation management module is used for acquiring the safety management regulation of the electric equipment based on the electric equipment related to the electric safety inspection request through the hidden danger identification of the context awareness.
Preferably, when the request type includes: when hidden danger inquiry or operation recommendation is carried out, the calling module comprises: the hidden danger identification module and the knowledge graph module;
The hidden danger identification module is used for carrying out first hidden danger identification based on the electrical data; based on the non-electrical data, performing second hidden danger identification by using a pre-trained hidden danger identification task model; fusing the first hidden danger identification result and the second hidden danger identification result to obtain a hidden danger identification result of the electric equipment;
the knowledge graph module is used for processing by utilizing a pre-constructed knowledge graph based on the electrical data, the non-electrical data, the request type, the safety management rule and the hidden danger identification result to obtain an electricity utilization safety inspection request result corresponding to the request type;
the hidden danger identification task model is obtained by training labeled historical non-electrical data and unlabeled historical non-electrical data through semi-supervised detection learning;
the tag includes: hidden danger information of electric equipment; the hidden trouble information comprises: normal, abnormal, hidden trouble.
Preferably, when the request type is a state query, the calling module is specifically configured to collect an electrical operation state and a non-electrical operation state of the electrical equipment;
the context-aware hidden danger identification module is further configured to: and carrying out rule judgment based on the electric operation state and the non-electric operation state and the electricity safety logic to obtain the electric equipment state.
Based on the same inventive concept, the invention also provides an intelligent multi-mode fusion electricity inspection system, which comprises an intelligent multi-mode fusion electricity inspection device, a visualization device and a data management device;
the visual equipment is an electricity inspection field device and is used for collecting electric data and non-electric data of electric equipment on an electricity utilization field; meanwhile, generating an electricity utilization safety inspection request according to the collected electric data and non-electric data of the electric equipment and the electricity utilization safety inspection request type;
the intelligent multi-mode fusion electricity utilization inspection device acquires an electricity utilization security inspection request initiated by the visual equipment, and executes the intelligent multi-mode fusion electricity utilization inspection method provided by the invention based on management data provided by the data management device;
the data management device is used for storing various types of management data.
Preferably, the visualization device is a mobile intelligent terminal and comprises one or more of the following: mobile computer terminal, mobile phone terminal and AR acquisition terminal.
Preferably, the management data stored by the data management device at least includes one or more of the following management data: electrical data, non-electrical data, knowledge graph;
The electrical data and the non-electrical data are historical data of the electric equipment with labels.
Based on the same inventive concept, the present invention also provides a computer apparatus comprising: one or more processors;
the processor is used for storing one or more programs;
when the one or more programs are executed by the one or more processors, the intelligent multi-mode fusion power consumption checking method provided by the invention is realized.
Based on the same inventive concept, the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed, implements the intelligent multi-mode fusion power consumption checking method provided by the invention.
Compared with the prior art, the invention has the beneficial effects that:
(1) The invention provides an intelligent multi-mode fusion electricity inspection method, device and system, which comprises the following steps: acquiring a safety management rule of electric equipment based on the electric equipment related to the electric safety inspection request through hidden danger identification of context awareness; according to the request type related to the electricity safety inspection request, analyzing and determining an electricity safety inspection request result aiming at the electric equipment based on a safety management rule, wherein the electricity data and the non-electricity data of the electric equipment are related to the electricity safety inspection request;
(2) The invention provides an intelligent multi-mode fusion electricity inspection device, which realizes unified call on various request types by taking a context-aware hidden danger identification module as an interface, judges and returns corresponding electricity utilization security inspection request results according to different request types in the context-aware hidden danger identification module, greatly simplifies the development of a program and provides a convenient interface for the subsequent expansion of the program.
Drawings
FIG. 1 is a flow chart of an intelligent multi-mode fusion power consumption inspection method of the invention;
FIG. 2 is a typical flow chart of hidden trouble shooting according to the present invention;
FIG. 3 is a flow chart of an exemplary state query process of the present invention;
FIG. 4 is a flowchart representative of an exemplary operational recommendation of the present invention;
FIG. 5 is a flowchart of an exemplary hidden trouble recognition function according to the present invention;
FIG. 6 is an overall structure diagram of a consistent semi-supervised objective detection model of the present invention;
FIG. 7 is a schematic diagram of a pseudo tag self-training method of the present invention;
FIG. 8 is a flow chart of entity linking according to the present invention;
fig. 9 is a block diagram of an intelligent multimode fusion electrical inspection system according to the present invention.
Detailed Description
Aiming at the actual requirements of safety detection of electric equipment in a production environment, the intelligent device for detecting the electric parameters in the production environment utilizes the electric parameter measurement capability of the intelligent device for detecting the electric parameters in hardware and the capability of measuring various non-electric information can be expanded, and hardware support is provided for procedure perception map and multi-mode information fusion evidence collection through the technology of the Internet of things. In the aspect of security inspection service, the service platform realizes the hidden danger identification function of context awareness by combining electrical and non-electrical hidden danger identification through map disambiguation and reasoning of rule management. Finally, security check services including hidden danger inquiry, state display, operation recommendation and the like are provided for the user and can be displayed in a visual mode. For a better understanding of the present invention, reference is made to the following description, drawings and examples.
Example 1:
the invention provides an intelligent multi-mode fusion electricity inspection method, which can combine the electric quantity and non-electric quantity factors to inquire the safety situation of electric equipment and effectively detect the potential safety hazard of the electric equipment, as shown in figure 1, and comprises the following steps:
s1, when an electricity safety check request is received:
s2, acquiring a safety management rule of the electric equipment based on the electric equipment related to the electric safety inspection request through hidden danger identification of context awareness;
s3, according to the request type related to the electricity safety inspection request, analyzing and determining an electricity safety inspection request result aiming at the electric equipment based on a safety management rule and electric data and non-electric data of the electric equipment related to the electricity safety inspection request;
the electricity utilization safety check request comprises a request type, electric equipment, electric data and non-electric data of the electric equipment.
The electrical data and the non-electrical data of the electric equipment are collected by the inspection equipment and the sensors in the electric field scene, and are attached to an electric safety inspection request.
The request types related to the electricity safety check request of the invention comprise: the steps related to the hidden danger inquiry, the operation recommendation and the state inquiry are briefly introduced below.
Hidden danger inquiry:
as shown in fig. 2, in the electric equipment checking process, when an electric safety inspector uses intelligent equipment such as AR equipment or mobile terminals and mobile computers with photographing function to send hidden danger inquiry requests, a system server firstly calls a hidden danger identification module perceived by context and requests corresponding safety management rules of the electric equipment from a rule management module; in combination with the safety regulations, a request is sent to the fusion hidden trouble identification, and hidden trouble which possibly exists at present is identified; and sending the fused hidden danger information and the procedure information to the knowledge graph management module, requesting hidden danger inquiry, and finally returning corresponding hidden danger inquiry results sequentially after the knowledge graph management module performs operations such as disambiguation, inquiry, reasoning and the like.
State query:
as shown in fig. 3, in the electric equipment inspection process, an electric safety inspector sends a state inquiry request by using an intelligent device with a photographing function such as an AR device, a mobile terminal and a mobile computer, and then a system server sequentially sends a request to a procedure management module to acquire an electric equipment safety management procedure; sending a request to an electrical data management module to acquire electrical operation condition data; and sending a request to the non-electrical data management module, acquiring non-electrical operation condition data, and finally finishing the inquiry of the state of the electric equipment.
Operation recommendation:
as shown in fig. 4, in the electrical safety inspection process, when an electrical safety inspection personnel uses intelligent equipment such as AR equipment or mobile terminals and mobile computers with photographing function to send out an operation recommendation request, a system server side firstly calls a hidden danger identification module perceived by context and requests a corresponding safety management procedure of the electrical equipment from a procedure management module; in combination with the safety regulations, a request is sent to the fusion hidden trouble identification, and hidden trouble which possibly exists at present is identified; and sending the fusion hidden danger information and the procedure information to the knowledge graph management module, requesting the recommendation operation, and finally returning the recommendation operation results corresponding to the potential power utilization hazards to the inspector in sequence under the current scene after the knowledge graph management module performs disambiguation, inquiry, reasoning and other operations.
The hidden danger identification function related to hidden danger inquiry and operation recommendation is executed as follows:
as shown in fig. 5, after a fused hidden danger identification request is sent, the fused hidden danger identification module is called first, the hidden danger identification request is sent to the electric hidden danger identification module and the non-electric hidden danger identification module respectively, then a data request is sent to the data management module, corresponding data is obtained, the electric hidden danger identification and the non-electric hidden danger identification are respectively carried out, then the fused hidden danger identification module is returned, the fused identification analysis is carried out, and the result is returned.
Thus, when the electrical safety check request involves request types including: when the hidden danger is queried or the operation is recommended, the step S3 of analyzing the electrical data and the non-electrical data of the electrical equipment related to the electrical safety inspection request based on the safety management rule to determine the electrical safety inspection request result for the electrical equipment specifically includes:
performing first hidden danger identification based on the electrical data; based on the non-electrical data, performing second hidden danger identification by using a pre-trained hidden danger identification task model; fusing the first hidden danger identification result and the second hidden danger identification result to obtain a hidden danger identification result of the electric equipment;
based on the electrical data, the non-electrical data, the request type, the safety management rules and the hidden danger identification result, processing by utilizing a pre-constructed knowledge graph to obtain an electricity utilization safety inspection request result corresponding to the request type;
the hidden danger identification task model is obtained by training labeled historical non-electrical data and unlabeled historical non-electrical data through semi-supervised detection learning;
the tag includes: hidden danger information of electric equipment; the hidden trouble information comprises: normal, abnormal, hidden trouble.
When the request type is a status query, the step S3 of analyzing electrical data and non-electrical data of the electrical equipment related to the electrical safety inspection request based on the safety management rule to determine an electrical safety inspection request result for the electrical equipment specifically includes:
collecting the electric operation state and the non-electric operation state of the electric equipment;
and carrying out rule judgment based on the electric operation state and the non-electric operation state and the electricity safety logic to obtain the electric equipment state.
Algorithms involved in hidden trouble inquiry or operation recommendation are specifically described below.
1. Semi-supervised non-electrical quantity hidden danger identification algorithm
As shown in fig. 6, for the actual situation that the safety accident samples in the electric safety are fewer, in order to make the anti-interference capability of the trained model stronger, a model training method based on semi-supervised learning is introduced. In the non-electric potential hazard identification process, the semi-supervised detection model mainly analyzes and identifies the image information of the electric equipment in the actual electricity utilization scene, and a sample library contains a small amount of images with marked labels and a large amount of image data without labels. The identification here may be: whether a fire, smoke or line open, etc., will be described below for better understanding of the scheme, by way of example, but the algorithm of the present invention is not limited to the fire. The tag with the ignition state is used for judging faults such as ignition of electric equipment and identifying hidden dangers of electricity. According to the semi-supervised monitoring model training method, the task model effect of the abnormal condition of the electric equipment in the non-electric quantity of the image is improved through pseudo-label iterative training, the consistency constraint loss is calculated for the non-labeling image on the basis, the non-labeling image and the labeled image are trained together, the fusion result of the consistency constraint loss and the target detection loss is obtained, and the reverse reasoning is carried out, so that the detection precision of the single-stage model SSD is effectively improved, and the problem that the electric field scene video training sample is insufficient is solved.
1) Semi-supervised target detection model
The model structure combines a semi-supervised learning method with a target detection method, and in order to achieve one-to-one correspondence of target objects, an original image and a flipped image thereof are used as input. A pair of object boxes represents the same class and their positioning information must be consistent so that a constraint model can be performed based on the consistency of the detection results of the two input image pairs.
(1) Single-stage detection process, as in the left part of fig. 6: firstly, an image I of electric equipment in a training set and an inverted image are processed
Figure BDA0003677541840000094
In the input single-stage classifier, an image is divided into S multiplied by S grids, each grid predicts B frames, and each frame predicts a confidence level besides the position of the frame to be returned to the frame to represent the credibility of the target and the accuracy of the predicted position. In addition, the conditional probabilities of C types are predicted for each grid, and the prediction results are encoded into a tensor in the dimension of SxSx (B x 5+C) and respectively expressed as f in the graph k (I) And +.>
Figure BDA0003677541840000091
And finally, respectively carrying out loss calculation on the labeled and unlabeled electrical appliance images based on a prediction result, wherein the prediction result is hidden danger information and comprises: normal, abnormal, hidden trouble.
(2) A two-stage detection process, as shown in the right part of fig. 6: firstly, an image I of electric equipment in a training set and an inverted image are processed
Figure BDA0003677541840000092
Inputting the picture into CNN network, extracting picture characteristics to obtain correspondent characteristic map phi (I) and +.>
Figure BDA0003677541840000093
A region suggestion network (Region proposal network, RPN) is then used on the feature map to extract a series of rectangular candidate regionsThereby extracting a region of interest (Regions of Interest, roI), the RoI in the flipped and non-flipped live-view images being represented as
Figure BDA0003677541840000101
And h k The method comprises the steps of carrying out a first treatment on the surface of the Mapping each RoI to obtain a corresponding feature map, and sending the feature map to an RoI pooling layer to unify the feature maps to the same size; and finally, through a full connection layer, finally obtaining an output vector through a classifier, and respectively carrying out loss calculation on the labeled and unlabeled electrical appliance images.
For unlabeled images and labeled images, the patent optimizes them with a constructively different loss function. For non-tag data, by combining optimization of consistency classification loss JS divergence and consistency target positioning loss L2 distance, and superimposing monitoring loss of tag image data (a part of all image data is subjected to tagging operation, and the tag represents whether the image data is in an abnormal state of firing) on the basis, global loss of a semi-monitoring target detection model can be obtained, as follows:
Figure BDA0003677541840000102
In the method, in the process of the invention,
Figure BDA0003677541840000103
detecting global loss of the model for the semi-supervised target; />
Figure BDA0003677541840000104
Classification loss for the original target detector;
Figure BDA0003677541840000105
loss of positioning for the original target detector; w (t) is the weight of the consistency loss in the overall loss; t is a training batch;
Figure BDA0003677541840000106
is a consistency loss of unlabeled data.
In order to train the network better, the w (t) adopts a ramp-up and ramp-down method, and the weight of the overall loss is lost along with the conversion lifting consistency of the training batch, so that the network can be converged more quickly.
2) Semi-supervised Self-Training method
The model adopts a semi-supervised Self-Training method, as shown in fig. 7, the Self-Training process firstly uses the labeled data to train to obtain the model, then predicts the unlabeled data, and the data with high confidence can be used for adding a Training set to continue Training until the model meets the requirements. The patent adopts a pseudo tag training strategy to train a model on tagged and untagged images at the same time, and the training process is as follows:
first, the pre-trained model is trained in a supervised manner using both marked and unmarked data. The weighted sum of the marked and unmarked penalty terms is taken as the total penalty as follows:
Figure BDA0003677541840000107
Wherein: l is the total loss; n is the small batch data amount in the marked data; m is the number in the labeled small batch data; c is the data volume with the label; i is the number in the tagged data;
Figure BDA0003677541840000111
an output unit for m samples in the marked data; />
Figure BDA0003677541840000112
An output tag that is marked data; a () is a balance coefficient; t is training round; n' is the small batch data size in the unmarked data; m' is the number in the unlabeled small lot data; i' is the number in the unlabeled data; />
Figure BDA0003677541840000113
Is an output unit in the unmarked data; />
Figure BDA0003677541840000114
The dummy tag is output for unlabeled data.
And then, predicting a batch of unlabeled images by using the trained model, and identifying whether the electric appliance in the non-electric data of the images catches fire or not, wherein the maximum confidence is used as a pseudo tag.
And finally, fine-tuning the data with the labels and the pseudo labels together until the optimal model is finally obtained.
The invention adopts a semi-supervised non-electric quantity hidden danger identification method to realize the purpose of model training by combining a small amount of marked data with a large amount of unmarked data, and solves the problem of insufficient data of a non-electric quantity sample of a power consumption scene.
2. Map construction and reasoning method for fusing entity disambiguation
1) Atlas entity linking procedure
In the construction process of the business rule knowledge graph, unstructured, redundant and even wrong information exists in massive and complex information of the data sources, and entity names in the text data sources are linked to target entities in a given knowledge base by an entity linking technology; for example, given the text "the electrical resistance of the appliance is 20 ohms", the entity linking system needs to correctly correspond the entity designation "ohm" in the sentence to the entity "ohm" representing the unit of resistance in the real world in the knowledge base, rather than incorrectly linking to the entity "gavage ohm" representing the german physicist. Aiming at the electricity consumption intelligent inspection scene, the disambiguation treatment is required to be carried out on the entities in the electric quantity information (such as electric parameters of voltage value, current value, frequency, impedance, capacitance and the like) and the non-electric quantity (such as electric equipment temperature, pressure and the like) contained in the business text data source, so that the problem of diversity and ambiguity of entity names is solved when a user carries out electricity consumption detection rule search or business rule recommendation, and the real understanding of the user demands is realized.
The process of completing entity linking includes three steps, as shown in fig. 8, entity name identification, candidate entity generation and candidate entity disambiguation, respectively.
First, entity designation recognition is performed in the first step of text processing, and all words possibly representing named designation items, such as proper nouns of transformers, cables, currents, voltages and the like, in the electric service text to be disambiguated are acquired. The system adopts a Stanford NER tool to carry out entity recognition on a given text to be disambiguated, and the characteristics defined in the model comprise a current word, a word before and after the current word, a character n-gram, a part-of-speech and context part-of-speech sequence, a shape of a word, a context shape of a word sequence and the like.
Subsequently, a candidate entity generation procedure is performed on the text, and the entity in the text is determined to refer to a set of entities possibly pointed to, for example, the entity "ohm" in "the resistance of the electric appliance is 20 ohm" can refer to a plurality of entities in the knowledge base, such as "ohm (unit)", "ohm (physicist)". (1) First, the surface names are expanded, some entity names are acronyms or a part of their full names (e.g., the symbol "I" representing current), so that possible expanded variants (e.g., the full name "current") can be identified from documents in which the entity names appear by the surface name expansion technique. And performing surface name expansion by adopting a rule-based method, and generating an entity name query list. Given an entity designation word m, a specific extension rule is as follows:
1) Adding the surface layer character string of m into a query list;
2) And querying other words in the text to be disambiguated, wherein the surface layer character string meeting m is a substring of the word, and adding the longest word as an extension name of m into a query list in a word set meeting the condition. And forming the extended forms into a candidate entity set pointed by the entity.
Through the entity name extension process, a query list can be obtained, candidate entities are searched in the constructed name-candidate entity dictionary for each character string in the query list, and a candidate entity set of all entity names can be obtained. The corresponding relation dictionary of entity index and candidate entity is composed of two columns, namely entity index item and candidate entity set, and forms the entity inquiry dictionary as shown in the following table. And then carrying out noise filtering by utilizing the context information of the entity reference items so as to solve the problems of the number and recall rate generated by the candidate entities.
And finally, performing candidate entity disambiguation and linking work, and dividing a candidate entity disambiguation model into a local model and a global model in order to fully utilize the context information pointed by the entity and the global correlation of each entity in the document in the same electricity inspection field, wherein the two models adopt different disambiguation evidences. The local model uses a deep neural network to measure semantic association between candidate entities and entity index items and contexts thereof, and focuses on single entity index-candidate entity pairs; the global model uses knowledge representation and graph model to measure the association relation strength among all candidate entities pointed by all entities, and focuses on all entity names in the same document, so that the whole relation among the entities is introduced into disambiguation evidence information, and a better entity linking effect is realized.
2) Federated entity linking algorithm
The method for using deep learning by local disambiguation comprises the core steps of firstly carrying out vector representation on a context text of entity names (such as safety regulations, electric faults of electric equipment, potential safety hazards of electric equipment and circuits and the like) in the electric safety inspection field and a description text of candidate entities, respectively obtaining semantic vectors of the entity names and the candidate entities, and taking the semantic vectors as input of a deep neural network to obtain similarity probabilities of the entity names and the candidate entities.
The global disambiguation algorithm constructs an entity association diagram for each input text to be disambiguated, and improves the accuracy of entity linking by means of the link relation of the entities in the electric power field in the knowledge base and the semantic association between candidate entities. In the entity association graph, a candidate entity set corresponding to all entity references in the text is taken as a vertex set of the graph model, edges in the association graph are established between candidate entities pointed by any two different entities, the edges represent the relationship between the entities, the weight of the edges represents the association degree between the starting point candidate entity and the end point candidate entity, and the global association degree of all the candidate entities is calculated to obtain the global association degree.
And combining entity links, namely multiplying the local similarity by the global association after obtaining the local similarity of the entity names and the candidate entities in the electricity inspection field and the global association of the candidate entities, taking the candidate entity corresponding to the maximum value as a target entity, taking the target entity as a link object, combining the local similarity and the global association, enriching evidence information of entity disambiguation and improving the accuracy of entity links.
Through the map entity linking process, the identification, generation and disambiguation processes of the entities in the service text of the electricity safety field are realized, and the accurate entities in the safety electricity utilization field such as safety regulations are connected with the knowledge map, so that the user can realize accurate understanding and response to the user demands when carrying out electricity utilization safety detection or service operation regulation recommendation.
In the construction process of the domain knowledge graph of the electricity safety, a large number of business entities and relations containing the domain electric quantity and non-electric quantity information are imported into the knowledge graph through the steps of data acquisition, information extraction, entity linking and the like, a domain knowledge base is provided, the knowledge base is a set of domain knowledge, and the form of the knowledge base can be text, a database or the knowledge graph. By means of knowledge reasoning capability of the knowledge graph, rule judgment is carried out by combining with knowledge in the fields of safety regulations, electrical faults of electrical equipment and the like, the safety state of the current electrical equipment is analyzed, technical support is provided for functions such as hidden danger prediction and the like in electricity utilization inspection, and intelligent electricity utilization safety inspection is achieved.
The invention applies the knowledge graph technology to carry out operation recommendation aiming at the current electric equipment state and potential safety hazard, and can safely guide the next processing.
Example 2:
based on the same inventive concept, the invention also provides an intelligent multi-mode fusion electricity inspection device, as shown in fig. 9, comprising: a hidden danger identification module and a procedure management module for sensing the context;
the context-aware hidden danger identification module is used for, when receiving an electricity safety check request: invoking the procedure management module and receiving a security management procedure returned by the procedure management module; and the module is further used for calling a corresponding module to execute based on the request type related to the electricity safety check request: analyzing the electrical data and the non-electrical data of the electric equipment related to the electric safety inspection request based on a safety management rule to determine an electric safety inspection request result aiming at the electric equipment; the invention takes the context-aware hidden danger identification module as an interface, realizes unified call on various request types, judges and returns the corresponding electricity utilization security check request result according to different request types in the context-aware hidden danger identification module, greatly simplifies the development of the program and provides a convenient interface for the subsequent expansion of the program;
The regulation management module is used for acquiring the safety management regulation of the electric equipment based on the electric equipment related to the electric safety inspection request through the hidden danger identification of the context awareness.
When the request type includes: when hidden danger inquiry or operation recommendation is carried out, the calling module comprises: the hidden danger identification module and the map disambiguation reasoning module;
the hidden danger identification module is used for carrying out first hidden danger identification based on the electrical data; based on the non-electrical data, performing second hidden danger identification by using a pre-trained hidden danger identification task model; fusing the first hidden danger identification result and the second hidden danger identification result to obtain a hidden danger identification result of the electric equipment;
the map disambiguation reasoning module is used for processing by utilizing a pre-constructed knowledge map based on the electrical data, the non-electrical data, the request type, the safety management rule and the hidden danger identification result to obtain an electricity utilization safety inspection request result corresponding to the request type; when the request type is hidden danger inquiry, the electricity utilization security check request result is a hidden danger inquiry result; when the request type is operation recommendation, the electricity utilization safety check request result is a recommendation operation result corresponding to the elimination of electricity utilization hidden danger.
The hidden danger identification task model is obtained by training labeled historical non-electrical data and unlabeled historical non-electrical data through semi-supervised detection learning;
the tag includes: hidden danger information of electric equipment; the hidden trouble information comprises: normal, abnormal, hidden trouble.
When the request type is state inquiry, the calling module is specifically used for collecting the electric operation state and the non-electric operation state of the electric equipment;
the context-aware hidden danger identification module is further configured to: and carrying out rule judgment based on the electric operation state and the non-electric operation state and the electricity safety logic to obtain the electric equipment state.
Training of the hidden danger identification task model in this embodiment includes:
training by taking the historical non-electrical data with the tag as a training set to obtain a hidden danger identification task model;
predicting unlabeled historical non-electrical data by utilizing single-stage target detection or double-stage target detection based on the hidden danger identification task model to obtain hidden danger information, confidence corresponding to the hidden danger information, marking loss corresponding to the labeled historical non-electrical data and unlabeled historical non-electrical data;
Taking the weighted sum of the tag loss and the no-tag loss as a consistency constraint loss;
taking hidden danger information corresponding to the maximum confidence coefficient as a pseudo tag of unlabeled historical non-electrical data, adding a training set to train the task model continuously until the task model reaches the optimal;
the label loss and the label-free loss are calculated by a semi-supervised target detection model.
1) The single-stage target detection process includes:
inputting image data of electric equipment in a training set and the inverted image data into a single-stage classifier to be predicted to obtain hidden danger information and confidence coefficient of the hidden danger information;
respectively carrying out target detection loss calculation on the labeled historical non-electrical data and the unlabeled historical non-electrical data;
the historical non-electrical data are image data corresponding to electric equipment.
2) The dual-stage target detection process includes:
taking image data of electric equipment in a training set and the inverted image data as input data, and training by using a CNN (computer network) to obtain hidden danger information; respectively carrying out target detection loss calculation on the labeled historical non-electrical data and the unlabeled historical non-electrical data; the historical non-electrical data are image data corresponding to electric equipment.
The map disambiguation reasoning module is specifically used for:
1) Performing disambiguation processing from the electrical data, the non-electrical data, the safety management rules and the hidden danger identification results by utilizing a pre-constructed knowledge graph; the specific process is as follows:
extracting entity names from the electrical data, the non-electrical data, the safety management rules and the hidden danger identification results;
and (3) respectively performing local disambiguation and global disambiguation by utilizing a pre-established knowledge graph based on each entity reference, and then determining a corresponding target entity.
2) And carrying out rule judgment based on disambiguated electrical data or non-electrical data in combination with a safety regulation, a hidden danger identification result and an electricity safety logic to obtain an electricity safety inspection request result corresponding to the request type.
Here, the following operations are performed based on each entity designation, respectively:
selecting at least one entity from a pre-constructed knowledge graph as a candidate entity based on the entity index;
obtaining local similarity probability of each entity finger and the candidate entity by using a deep learning method;
determining a global association based on the association between all entity designations and the corresponding candidate entities;
multiplying the local similarity probability of the candidate entity by the global association degree of the candidate entity, taking the candidate entity corresponding to the maximum value as a target entity, and designating the entity as a link with the target entity;
The knowledge graph comprises: and the association relation of a plurality of entities and each entity.
The specific algorithm in this embodiment may refer to embodiment 1, and will not be described here again.
Example 3
Based on the same inventive concept, the invention also provides an intelligent multimode fusion electricity inspection system, which is a software platform system and is designed into a three-layer architecture, as shown in fig. 9, comprising: an intelligent multi-mode fusion power consumption checking device, a visualization device and a data management device;
the visualization equipment is power utilization inspection field equipment and is used for realizing a security inspection service visualization function, and as shown in the uppermost block diagram of fig. 9, the visualization equipment is specifically used for collecting electric data and non-electric data of electric equipment on a power utilization field; meanwhile, generating an electricity utilization safety inspection request according to the collected electric data and non-electric data of the electric equipment and the electricity utilization safety inspection request type;
the intelligent multi-mode fusion electricity inspection device is used for providing security inspection service, and as shown in a middle frame of fig. 9, the intelligent multi-mode fusion electricity inspection device is used for acquiring an electricity security inspection request initiated by the visual equipment and executing the intelligent multi-mode fusion electricity inspection method provided by the invention based on management data provided by the data management device;
The data management device is used for storing various types of management data as shown in the lowest block diagram of fig. 9.
The visualization apparatus includes: mobile terminal, mobile computer terminal or AR acquisition terminal, etc.
The management data stored by the data management device at least comprises one or more of the following management data: electrical data, non-electrical data, and knowledge graph.
The three-layer service is specifically described as follows:
data management: the data management layer provides data support for the security check service above the data management layer and mainly comprises electric data management, non-electric data management and knowledge graph management. The electrical data is the electricity consumption condition data of the electric equipment, such as analog quantity data of current, voltage, power, frequency and the like collected from an electric equipment sensor; the variety of non-electrical data is more, such as video data collected by an electric field scene camera when the electric equipment operates, and sensor data of various temperatures, pressures, flow rates and the like; the knowledge graph management means that a knowledge base storing graph information is managed, and corresponding operations such as updating, inquiring and the like are completed.
Security check service: the platform system provides hidden danger identification adopting safety flow context sensing, the function is divided into two parts to be realized, firstly, knowledge graph disambiguation and reasoning are carried out on the basis of electric equipment safety regulation management, ambiguity is eliminated for words with a word meaning, and reasoning analysis of potential safety hazards is completed; and secondly, semi-supervised learning is used for collecting and analyzing various electrical data and non-electrical data, and hidden danger is identified. The specific functions are described in embodiment 2 and will not be repeated here.
Security check business visualization: the safety check service provides the user with the functions of inquiring the potential safety hazards of the electric equipment, displaying the state and recommending the operation aiming at the potential safety hazards and the running state of the current electric equipment. The functions are presented through an AR interaction technology, so that multiple persons share the picture and view the data measured in real time on site, and an operator can be guided to solve corresponding problems through remote voice according to the current real-time data and the live real-time picture. The invention realizes service presentation through the AR technology, facilitates remote real-time sharing of site information, and can guide an operator to solve corresponding problems according to current real-time data and site real-time images in a remote voice manner.
Example 4:
based on the same inventive concept, the invention also provides a computer device comprising a processor and a memory for storing a computer program comprising program instructions, the processor for executing the program instructions stored by the computer storage medium. The processor may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application SpecificIntegrated Circuit, ASIC), off-the-shelf Programmable gate arrays (FPGAs) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., which are the computational core and control core of the terminal adapted to implement one or more instructions, in particular to load and execute one or more instructions in a computer storage medium to implement the corresponding method flow or corresponding functions, to implement the steps of an intelligent multimodal fusion electrical inspection method in the above embodiments.
Example 5
Based on the same inventive concept, the present invention also provides a storage medium, in particular, a computer readable storage medium (Memory), which is a Memory device in a computer device, for storing programs and data. It is understood that the computer readable storage medium herein may include both built-in storage media in a computer device and extended storage media supported by the computer device. The computer-readable storage medium provides a storage space storing an operating system of the terminal. Also stored in the memory space are one or more instructions, which may be one or more computer programs (including program code), adapted to be loaded and executed by the processor. The computer readable storage medium herein may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. One or more instructions stored in a computer-readable storage medium may be loaded and executed by a processor to implement the steps of an intelligent multimodal fusion power up checking method in the above embodiments.
It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing is illustrative of the present invention and is not to be construed as limiting thereof, but rather as providing for the use of additional embodiments and advantages of all such modifications, equivalents, improvements and similar to the present invention are intended to be included within the scope of the present invention as defined by the appended claims.

Claims (15)

1. An intelligent multi-mode fusion power consumption inspection method is characterized by comprising the following steps:
When an electrical safety check request is received:
acquiring a safety management rule of electric equipment based on the electric equipment related to the electric safety inspection request through hidden danger identification of context awareness;
according to the request type related to the electricity safety inspection request, analyzing and determining an electricity safety inspection request result aiming at the electric equipment based on a safety management rule and electric data and non-electric data of the electric equipment related to the electricity safety inspection request;
the power utilization safety check request comprises a request type, electric equipment, electric data and non-electric data of the electric equipment;
when the request types related to the electricity safety check request include: when hidden danger inquiry or operation recommendation is performed, the electrical data and non-electrical data of the electrical equipment related to the electrical safety inspection request are analyzed and determined based on a safety management rule, and an electrical safety inspection request result aiming at the electrical equipment is determined, and the method comprises the following steps:
performing first hidden danger identification based on the electrical data; based on the non-electrical data, performing second hidden danger identification by using a pre-trained hidden danger identification task model; fusing the first hidden danger identification result and the second hidden danger identification result to obtain a hidden danger identification result of the electric equipment;
Based on the electrical data, the non-electrical data, the request type, the safety management rules and the hidden danger identification result, processing by utilizing a pre-constructed knowledge graph to obtain an electricity utilization safety inspection request result corresponding to the request type;
the hidden danger identification task model is obtained by training labeled historical non-electrical data and unlabeled historical non-electrical data through semi-supervised detection learning;
the tag includes: hidden danger information of electric equipment; the hidden trouble information comprises: normal, abnormal, hidden trouble exists;
the processing by utilizing a pre-constructed knowledge graph based on the electrical data, the non-electrical data, the request type, the safety management procedure and the hidden danger identification result to obtain an electricity utilization safety inspection request result corresponding to the request type comprises the following steps:
performing disambiguation processing from the electrical data, the non-electrical data, the safety management rules and the hidden danger identification results by utilizing a pre-constructed knowledge graph;
and carrying out rule judgment based on disambiguated electrical data or non-electrical data in combination with a safety regulation, a hidden danger identification result and an electricity safety logic to obtain an electricity safety inspection request result corresponding to the request type.
2. The method of claim 1, wherein the training of the hidden danger identification task model comprises:
training by taking the historical non-electrical data with the tag as a training set to obtain a hidden danger identification task model;
predicting unlabeled historical non-electrical data by utilizing single-stage target detection or double-stage target detection based on the hidden danger identification task model to obtain hidden danger information, confidence corresponding to the hidden danger information, marking loss corresponding to the labeled historical non-electrical data and unlabeled historical non-electrical data;
taking the weighted sum of the tag loss and the no-tag loss as a consistency constraint loss;
taking hidden danger information corresponding to the maximum confidence coefficient as a pseudo tag of unlabeled historical non-electrical data, adding a training set to train the task model continuously until the task model reaches the optimal;
the label loss and the label-free loss are calculated by a semi-supervised target detection model.
3. The method of claim 2, wherein the single-stage object detection process comprises:
inputting image data of electric equipment in a training set and the inverted image data into a single-stage classifier to be predicted to obtain hidden danger information and confidence coefficient of the hidden danger information;
Respectively carrying out target detection loss calculation on the labeled historical non-electrical data and the unlabeled historical non-electrical data;
the historical non-electrical data are image data corresponding to electric equipment.
4. The method of claim 2, wherein the dual-stage object detection process comprises:
taking image data of electric equipment in a training set and the inverted image data as input data, and training by using a CNN (computer network) to obtain hidden danger information; respectively carrying out target detection loss calculation on the labeled historical non-electrical data and the unlabeled historical non-electrical data; the historical non-electrical data are image data corresponding to electric equipment.
5. The method of claim 1, wherein,
when the request type is hidden danger inquiry, the electricity utilization security check request result is a hidden danger inquiry result;
when the request type is operation recommendation, the electricity utilization safety check request result is a recommendation operation result corresponding to the elimination of electricity utilization hidden danger.
6. The method of claim 1, wherein said disambiguating from said electrical data, non-electrical data, security management procedures, and hidden danger identification results using pre-constructed knowledge maps comprises:
Extracting entity names from the electrical data, the non-electrical data, the safety management rules and the hidden danger identification results;
and (3) respectively performing local disambiguation and global disambiguation by utilizing a pre-established knowledge graph based on each entity reference, and then determining a corresponding target entity.
7. The method of claim 1, wherein determining the corresponding target entity after performing local disambiguation and global disambiguation, respectively, using the pre-constructed knowledge-graph based on each entity reference comprises:
the following operations are performed based on each entity designation:
selecting at least one entity from a pre-constructed knowledge graph as a candidate entity based on the entity index;
obtaining local similarity probability of each entity finger and the candidate entity by using a deep learning method;
determining a global association based on the association between all entity designations and the corresponding candidate entities;
multiplying the local similarity probability of the candidate entity by the global association degree of the candidate entity, taking the candidate entity corresponding to the maximum value as a target entity, and designating the entity as a link with the target entity;
the knowledge graph comprises: and the association relation of a plurality of entities and each entity.
8. The method of claim 1, wherein when the request type is a status query, the analyzing electrical data and non-electrical data of the powered device involved in the electrical safety inspection request based on a safety management procedure to determine an electrical safety inspection request result for the powered device comprises:
collecting the electric operation state and the non-electric operation state of the electric equipment;
and carrying out rule judgment based on the electric operation state and the non-electric operation state and the electricity safety logic to obtain the electric equipment state.
9. An intelligent multimode fusion power consumption inspection device, characterized by comprising: a hidden danger identification module and a procedure management module for sensing the context;
the context-aware hidden danger identification module is used for, when receiving an electricity safety check request: invoking the procedure management module and receiving a security management procedure returned by the procedure management module; and the module is further used for calling a corresponding module to execute based on the request type related to the electricity safety check request: analyzing the electrical data and the non-electrical data of the electric equipment related to the electric safety inspection request based on a safety management rule to determine an electric safety inspection request result aiming at the electric equipment;
The regulation management module is used for acquiring the safety management regulation of the electric equipment based on the electric equipment related to the electric safety inspection request through the hidden danger identification of the context awareness;
the request types include: when hidden danger inquiry or operation recommendation is carried out, the called modules comprise: the hidden danger identification module and the map disambiguation reasoning module;
the hidden danger identification module is used for carrying out first hidden danger identification based on the electrical data; based on the non-electrical data, performing second hidden danger identification by using a pre-trained hidden danger identification task model; fusing the first hidden danger identification result and the second hidden danger identification result to obtain a hidden danger identification result of the electric equipment;
the map disambiguation reasoning module is used for processing by utilizing a pre-constructed knowledge map based on the electrical data, the non-electrical data, the request type, the safety management rule and the hidden danger identification result to obtain an electricity utilization safety inspection request result corresponding to the request type;
the hidden danger identification task model is obtained by training labeled historical non-electrical data and unlabeled historical non-electrical data through semi-supervised detection learning;
the tag includes: hidden danger information of electric equipment; the hidden trouble information comprises: normal, abnormal, hidden trouble exists;
The processing by utilizing a pre-constructed knowledge graph based on the electrical data, the non-electrical data, the request type, the safety management procedure and the hidden danger identification result to obtain an electricity utilization safety inspection request result corresponding to the request type comprises the following steps:
performing disambiguation processing from the electrical data, the non-electrical data, the safety management rules and the hidden danger identification results by utilizing a pre-constructed knowledge graph;
and carrying out rule judgment based on disambiguated electrical data or non-electrical data in combination with a safety regulation, a hidden danger identification result and an electricity safety logic to obtain an electricity safety inspection request result corresponding to the request type.
10. The apparatus of claim 9, wherein the invoked module is specifically configured to collect an electrical operating state and a non-electrical operating state of the powered device when the request type is a status query;
the context-aware hidden danger identification module is further configured to: and carrying out rule judgment based on the electric operation state and the non-electric operation state and the electricity safety logic to obtain the electric equipment state.
11. An intelligent multi-mode fusion electricity inspection system is characterized by comprising an intelligent multi-mode fusion electricity inspection device, a visualization device and a data management device;
The visual equipment is an electricity inspection field device and is used for collecting electric data and non-electric data of electric equipment on an electricity utilization field; meanwhile, generating an electricity utilization safety inspection request according to the collected electric data and non-electric data of the electric equipment and the electricity utilization safety inspection request type;
the intelligent multi-mode fusion electricity inspection device acquires an electricity security inspection request initiated by the visual equipment and executes the intelligent multi-mode fusion electricity inspection method according to any one of claims 1-8 based on management data provided by the data management device;
the data management device is used for storing and managing various types of management data.
12. The system of claim 11, wherein the visualization device is a mobile smart terminal comprising one or more of: mobile computer terminal, mobile phone terminal and AR acquisition terminal.
13. The system of claim 11, wherein the management data stored and managed by the data management device includes at least one or more of the following management data: electrical data, non-electrical data, knowledge graph;
the electrical data and the non-electrical data are historical data of the electric equipment with labels.
14. A computer device, comprising: one or more processors;
the processor is used for storing one or more programs;
an intelligent multimodal fusion power up checking method as recited in any one of claims 1-8, when the one or more programs are executed by the one or more processors.
15. A computer readable storage medium, characterized in that a computer program is stored thereon, which computer program, when executed, implements an intelligent multimodal fusion power up checking method as claimed in any of claims 1-8.
CN202210626040.1A 2022-06-02 2022-06-02 Intelligent multi-mode fusion power consumption inspection method, device, system, equipment and medium Active CN115114448B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210626040.1A CN115114448B (en) 2022-06-02 2022-06-02 Intelligent multi-mode fusion power consumption inspection method, device, system, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210626040.1A CN115114448B (en) 2022-06-02 2022-06-02 Intelligent multi-mode fusion power consumption inspection method, device, system, equipment and medium

Publications (2)

Publication Number Publication Date
CN115114448A CN115114448A (en) 2022-09-27
CN115114448B true CN115114448B (en) 2023-07-14

Family

ID=83326287

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210626040.1A Active CN115114448B (en) 2022-06-02 2022-06-02 Intelligent multi-mode fusion power consumption inspection method, device, system, equipment and medium

Country Status (1)

Country Link
CN (1) CN115114448B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117009589B (en) * 2023-08-03 2024-05-03 上海南洋宏优智能科技有限公司 Security check management method and system based on multi-mode intelligent service
CN117688503B (en) * 2024-02-04 2024-04-16 国网天津市电力公司滨海供电分公司 Electricity safety inspection system based on mobile terminal

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102075992A (en) * 2011-01-07 2011-05-25 大唐移动通信设备有限公司 Method and equipment for determining terminal anomaly

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104167817B (en) * 2014-08-20 2017-05-24 深圳供电局有限公司 Power equipment real-time information integration system and method
US10586156B2 (en) * 2015-06-25 2020-03-10 International Business Machines Corporation Knowledge canvassing using a knowledge graph and a question and answer system
US20190198137A1 (en) * 2017-12-26 2019-06-27 International Business Machines Corporation Automatic Summarization of Patient Data Using Medically Relevant Summarization Templates
EP3637386A1 (en) * 2018-10-12 2020-04-15 Thales Machine learning on big data in avionics
CN113553420B (en) * 2020-04-22 2023-11-24 国网电力科学研究院有限公司 Power grid fault processing rule recommendation method and system based on knowledge graph
CN111898852A (en) * 2020-06-15 2020-11-06 广州智能科技发展有限公司 Method for constructing knowledge graph of hidden troubles and accidents of enterprise, electronic equipment and storage medium
CN112329914B (en) * 2020-10-26 2024-02-02 华翔翔能科技股份有限公司 Fault diagnosis method and device for buried transformer substation and electronic equipment
CN113064999B (en) * 2021-03-19 2023-12-15 南方电网调峰调频发电有限公司信息通信分公司 Knowledge graph construction algorithm, system, equipment and medium based on IT equipment operation and maintenance
CN114383652A (en) * 2021-12-06 2022-04-22 中国电力科学研究院有限公司 Method, system and device for identifying potential fault online risk of power distribution network
CN114282010A (en) * 2021-12-30 2022-04-05 国网电力科学研究院有限公司 Power grid operation fault identification method and system based on knowledge graph and storage medium

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102075992A (en) * 2011-01-07 2011-05-25 大唐移动通信设备有限公司 Method and equipment for determining terminal anomaly

Also Published As

Publication number Publication date
CN115114448A (en) 2022-09-27

Similar Documents

Publication Publication Date Title
CN115114448B (en) Intelligent multi-mode fusion power consumption inspection method, device, system, equipment and medium
KR102288249B1 (en) Information processing method, terminal, and computer storage medium
CN110362660A (en) A kind of Quality of electronic products automatic testing method of knowledge based map
CN109933785A (en) Method, apparatus, equipment and medium for entity associated
CN110795527B (en) Candidate entity ordering method, training method and related device
CN109710739B (en) Information processing method and device and storage medium
CN114332680A (en) Image processing method, video searching method, image processing device, video searching device, computer equipment and storage medium
CN113657100A (en) Entity identification method and device, electronic equipment and storage medium
CN111507285A (en) Face attribute recognition method and device, computer equipment and storage medium
CN116010586A (en) Method, device, equipment and storage medium for generating health advice
CN116821318A (en) Business knowledge recommendation method, device and storage medium based on large language model
CN110991183A (en) Method, device, equipment and storage medium for determining predicate of problem
CN111507137A (en) Action understanding method and device, computer equipment and storage medium
CN114329004A (en) Digital fingerprint generation method, digital fingerprint generation device, data push method, data push device and storage medium
CN113536784A (en) Text processing method and device, computer equipment and storage medium
CN113515589A (en) Data recommendation method, device, equipment and medium
CN115797795B (en) Remote sensing image question-answer type retrieval system and method based on reinforcement learning
CN111782676A (en) Probability prediction model training method, probability prediction method and probability prediction device
CN115906863A (en) Emotion analysis method, device and equipment based on comparative learning and storage medium
CN115132324A (en) Mental health prediction method and device, electronic equipment and storage medium
CN114547313A (en) Resource type identification method and device
CN114708429A (en) Image processing method, image processing device, computer equipment and computer readable storage medium
KR20190064288A (en) Sketch-based media plagiarism inspection method and apparatus
CN114238663B (en) Knowledge graph analysis method and system for material data, electronic device and medium
CN113392221B (en) Method and related device for processing thin entity

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant