CN115510253A - Transformer substation equipment AR auxiliary maintenance method and terminal based on three-dimensional knowledge graph - Google Patents
Transformer substation equipment AR auxiliary maintenance method and terminal based on three-dimensional knowledge graph Download PDFInfo
- Publication number
- CN115510253A CN115510253A CN202211035697.7A CN202211035697A CN115510253A CN 115510253 A CN115510253 A CN 115510253A CN 202211035697 A CN202211035697 A CN 202211035697A CN 115510253 A CN115510253 A CN 115510253A
- Authority
- CN
- China
- Prior art keywords
- fault
- maintenance
- real
- infrared thermal
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012423 maintenance Methods 0.000 title claims abstract description 186
- 238000000034 method Methods 0.000 title claims abstract description 60
- 239000013598 vector Substances 0.000 claims abstract description 84
- 230000005236 sound signal Effects 0.000 claims abstract description 39
- 238000004590 computer program Methods 0.000 claims description 22
- 230000000007 visual effect Effects 0.000 claims description 15
- 238000000605 extraction Methods 0.000 claims description 10
- 230000003190 augmentative effect Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000012163 sequencing technique Methods 0.000 claims description 4
- 238000000547 structure data Methods 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 claims description 3
- 230000008439 repair process Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 description 13
- 230000006870 function Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012800 visualization Methods 0.000 description 3
- 239000011521 glass Substances 0.000 description 2
- 238000013024 troubleshooting Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/483—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/43—Querying
- G06F16/432—Query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/43—Querying
- G06F16/432—Query formulation
- G06F16/433—Query formulation using audio data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/43—Querying
- G06F16/432—Query formulation
- G06F16/434—Query formulation using image data, e.g. images, photos, pictures taken by a user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/43—Querying
- G06F16/438—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/20—Administration of product repair or maintenance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Business, Economics & Management (AREA)
- Mathematical Physics (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- Health & Medical Sciences (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Library & Information Science (AREA)
- Marketing (AREA)
- Entrepreneurship & Innovation (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Computational Linguistics (AREA)
- Computer Graphics (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a transformer substation equipment AR auxiliary maintenance method and terminal based on a three-dimensional knowledge graph. The method comprises the following steps: acquiring a sound signal, a vibration signal and an infrared thermal image of a fault occurrence position; respectively extracting feature vectors of the sound signal, the vibration signal and the infrared thermal image to obtain fault feature vectors; respectively comparing the fault characteristic vector with a plurality of sample characteristic vectors in a fault three-dimensional knowledge graph to calculate similarity; determining a fault reason and a target maintenance scheme according to the similarity and the fault three-dimensional knowledge graph, and visually displaying the target maintenance scheme; and acquiring real-time image information of a fault site, generating a virtual-real combined image according to the real-time image information and a target maintenance scheme, and inputting the virtual-real combined image into a portable AR equipment display module for display so as to assist maintenance. The method can realize rapid acquisition of the maintenance scheme based on the three-dimensional knowledge map, and improve the field maintenance efficiency of workers in an AR auxiliary maintenance mode.
Description
Technical Field
The invention relates to the technical field of power system monitoring, in particular to a transformer substation equipment AR auxiliary maintenance method and a terminal based on a three-dimensional knowledge graph.
Background
A substation is an important electric power facility, and is mainly responsible for converting voltage, receiving and distributing electric energy, controlling the flow of electric power, and adjusting voltage in an electric power system. With the increase of the scale of the power grid and the complexity of the structure, workers cannot know and solve all the problems, and in many cases, the workers need to inquire data and a digital book library to obtain fault problems or solutions. How to facilitate the staff to obtain the relevant data of the fault or the maintenance scheme in time in the overhaul operation process of the transformer substation equipment is a problem to be solved urgently to improve the field maintenance efficiency of the staff.
Disclosure of Invention
The embodiment of the invention provides an AR auxiliary maintenance method and terminal for substation equipment based on a three-dimensional knowledge graph, and aims to solve the problem of how to improve the field maintenance efficiency of workers.
In a first aspect, an embodiment of the present invention provides a method for assisting maintenance of a substation device AR based on a three-dimensional knowledge graph, including:
acquiring a sound signal, a vibration signal and an infrared thermal image of a fault occurrence position;
respectively extracting feature vectors of the sound signal, the vibration signal and the infrared thermal image to obtain fault feature vectors;
respectively comparing the fault characteristic vector with a plurality of sample characteristic vectors in a fault three-dimensional knowledge graph to calculate similarity;
determining a fault reason and a target maintenance scheme according to the similarity and the fault three-dimensional knowledge graph, and carrying out visual display on the target maintenance scheme;
acquiring real-time image information of a fault site, generating a virtual-real combined image according to the real-time image information and the target maintenance scheme, and inputting the virtual-real combined image into a display module of portable AR equipment for display so as to assist maintenance; wherein, the fault three-dimensional knowledge graph comprises: fault causes, and three-dimensional models and sample characteristic vectors of fault parts, maintenance schemes and maintenance tools corresponding to the fault causes; the sample feature vector includes: a sound signal sample feature vector, a vibration signal sample feature vector, and an infrared thermal image sample feature vector.
In a possible implementation manner, the generating a virtual-real combined image according to the real-time image information and the target scheme includes:
determining a three-dimensional model of a corresponding maintenance tool according to the target maintenance scheme, and generating a virtual image aiming at the three-dimensional model of the maintenance tool through an augmented reality technology;
determining a fault part corresponding to the fault occurrence position and the position of the fault part according to the real-time image information and the target maintenance scheme;
and generating a virtual-real combined image according to the real-time image information, the virtual image, the fault part and the position of the fault part.
In a possible implementation manner, before the acquiring the sound signal, the vibration signal and the infrared thermal image of the fault occurrence position, the method further includes:
acquiring maintenance data in a substation equipment maintenance record table and a maintenance manual;
extracting vocabularies which represent the relationship between the entities in the maintenance data in a knowledge extraction mode, and constructing entity-relationship-entity triple structure data;
collecting a sound signal, a vibration signal and an infrared thermal image of a fault occurrence position when equipment is in fault, respectively extracting characteristic vectors, and constructing a triple of fault reasons and the corresponding characteristic vectors;
extracting the feature vector of the fault part corresponding to the fault occurrence position, and constructing a triple of the fault part and the corresponding feature vector;
performing three-dimensional modeling on maintenance tools of different specifications to generate a maintenance tool three-dimensional model, uniquely numbering the maintenance tools, and constructing a triple of numbers of the fault parts and the maintenance tools according to the characteristic matching of the equipment parts and the maintenance tools;
and leading all constructed triples into a graph database to construct a fault three-dimensional knowledge graph.
In a possible implementation manner, the determining a fault cause and a target maintenance scheme according to the similarity and the fault three-dimensional knowledge graph includes:
performing descending sorting on the plurality of similarity degrees obtained by calculation;
determining a fault reason according to the sample feature vectors corresponding to one or more similarity values sequenced in the front and the fault three-dimensional knowledge graph;
determining one or more corresponding maintenance schemes as recommended maintenance schemes according to the fault reasons, and carrying out visual display on the recommended maintenance schemes;
and acquiring a target maintenance scheme selection instruction, and determining a target maintenance scheme from the recommended maintenance schemes according to the target maintenance scheme selection instruction.
In a possible implementation manner, the visually displaying the recommended maintenance solution includes:
and when a plurality of recommended maintenance schemes exist, sequencing the recommended maintenance schemes according to the historical selection times, and carrying out visual display in sequence according to the selection times from the most to the least.
In one possible implementation manner, after the virtual-real combined image is input to a display module of a portable AR device for display, the method further includes:
and generating voice prompt information according to the maintenance steps of the target maintenance scheme.
In a possible implementation manner, after the virtual and real combined images are input into a display module of the portable AR device for display, the method further includes:
acquiring a real-time infrared thermal image of a fault site;
identifying whether the fault field equipment is electrified or not according to the real-time infrared thermal image;
and generating visual information according to the recognition result, and displaying the visual information in the set display area through a display module of the portable AR equipment.
In one possible implementation, the method further includes:
when the maintenance operation is finished, acquiring the sound signal, the vibration signal and the infrared thermal image of the fault occurrence position again;
respectively extracting feature vectors of the sound signal, the vibration signal and the infrared thermal image to obtain temporary feature vectors;
respectively comparing the temporary feature vectors with a plurality of sample feature vectors in a normally-operated three-dimensional knowledge graph to calculate similarity;
and determining that the maintenance is successful when the similarity is larger than a set threshold value.
In a third aspect, an embodiment of the present invention provides a three-dimensional knowledge graph-based auxiliary maintenance device for substation equipment AR, including:
the acquisition module is used for acquiring a sound signal, a vibration signal and an infrared thermal image of a fault occurrence position;
the characteristic extraction module is used for respectively extracting characteristic vectors of the sound signal, the vibration signal and the infrared thermal image to obtain a fault characteristic vector;
the calculation module is used for respectively comparing the fault characteristic vector with a plurality of sample characteristic vectors in a fault three-dimensional knowledge map to calculate similarity;
the visualization module is used for determining a fault reason and a target maintenance scheme according to the similarity and the fault three-dimensional knowledge map, and visually displaying the target maintenance scheme;
the acquisition module is also used for acquiring real-time image information of a fault site;
the virtual-real combination module is used for generating a virtual-real combination image according to the real-time image information and the target maintenance scheme, and inputting the virtual-real combination image into a portable AR equipment display module for display so as to assist maintenance; wherein, the fault three-dimensional knowledge graph comprises: fault causes, and three-dimensional models and sample characteristic vectors of fault parts, maintenance schemes and maintenance tools corresponding to the fault causes; the sample feature vector includes: the system comprises a sound signal sample feature vector, a vibration signal sample feature vector and an infrared thermal image sample feature vector.
In a third aspect, an embodiment of the present invention provides a terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method according to the first aspect or any one of the possible implementation manners of the first aspect when executing the computer program.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored, and the computer program, when executed by a processor, implements the steps of the method according to the first aspect or any one of the possible implementation manners of the first aspect.
The embodiment of the invention provides a transformer substation equipment AR auxiliary maintenance method and a terminal based on a three-dimensional knowledge graph, which can realize accurate identification of a fault occurrence position and a fault occurrence reason by acquiring a sound signal, a vibration signal and an infrared thermal image of the fault occurrence position. The sound signal, the vibration signal and the infrared thermal image are respectively subjected to feature vector extraction to obtain fault feature vectors, the fault feature vectors are respectively compared with a plurality of sample feature vectors in a fault three-dimensional knowledge map to calculate the similarity, the automatic identification of fault occurrence and fault occurrence reasons is improved through the three-dimensional knowledge map and feature comparison mode, and the problems of poor accuracy and low efficiency of manual fault identification are solved. And determining a fault reason and a target maintenance scheme according to the similarity and the fault three-dimensional knowledge graph, and visually displaying the target maintenance scheme. The real-time image information of a fault site is acquired, a virtual-real combined image is generated according to the real-time image information and the target maintenance scheme, and the virtual-real combined image is input into a portable AR equipment display module to be displayed so as to assist maintenance, so that maintenance prompt is carried out for workers, and the maintenance efficiency and safety are improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a flowchart illustrating an implementation of an AR auxiliary maintenance method for a substation device based on a three-dimensional knowledge graph according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an auxiliary maintenance device for a substation equipment AR based on a three-dimensional knowledge graph according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a terminal according to an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
Optionally, the method provided in the embodiment of the present invention is implemented by a portable AR device, for example: the implementation of AR glasses, AR helmets and the like, namely the determination process of the maintenance scheme and the virtual and real combined image display are realized by portable AR equipment, and the structure and the communication cost of the transformer substation equipment AR auxiliary device based on the three-dimensional knowledge map are simplified.
Optionally, the method provided in the embodiment of the present invention is executed by a signal processing device in communication connection with the portable AR device, that is, the portable AR device only executes subsequent virtual-real combined image display operation, and the determination of the maintenance scheme is executed by the signal processing device, so that the operation pressure and energy consumption of the portable AR device are reduced, and the determination efficiency of the maintenance scheme and the fault maintenance efficiency are improved.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following description is made by way of specific embodiments with reference to the accompanying drawings.
Fig. 1 is an application scene diagram of an auxiliary maintenance method for a substation equipment AR based on a three-dimensional knowledge graph according to an embodiment of the present invention. As shown in fig. 1, the method comprises the following steps:
and S101, acquiring a sound signal, a vibration signal and an infrared thermal image of a fault occurrence position.
Wherein the sound signal, vibration signal and infrared thermal image are detected or acquired by the corresponding devices. The execution subject in step S101, such as the portable AR device or a signal processing device communicatively connected to the portable AR device, receives the sound signal, the vibration signal, and the infrared thermal image transmitted by the corresponding device, so as to achieve the expertise and accuracy of acquiring the sound signal, the vibration signal, and the infrared thermal image.
Optionally, one or more detection devices of the sound signal, the vibration signal and the infrared thermal image are built in the execution main body in the step S101, or are obtained by a corresponding module configured by the execution main body in the step S101, so that the device cost in the auxiliary maintenance process is reduced.
S102, respectively extracting feature vectors of the sound signal, the vibration signal and the infrared thermal image to obtain fault feature vectors.
S103, comparing the fault characteristic vector with a plurality of sample characteristic vectors in the fault three-dimensional knowledge graph respectively to calculate similarity.
And S104, determining a fault reason and a target maintenance scheme according to the similarity and the fault three-dimensional knowledge graph, and visually displaying the target maintenance scheme.
The target maintenance scheme is sorted and divided according to the steps of the target maintenance scheme, and the target maintenance scheme after being sorted and divided is visually displayed, so that a user can check the maintenance scheme more clearly.
And S105, acquiring real-time image information of a fault site, generating a virtual-real combined image according to the real-time image information and a target maintenance scheme, and inputting the virtual-real combined image into a display module of the portable AR equipment for display so as to assist maintenance.
The real-time image information is used as a background, the target maintenance scheme is combined to display the counterweight region on the basis of the real-time image information, so that the position related to the target maintenance scheme can be conveniently and quickly locked by a worker, and the troubleshooting and maintenance work can be carried out, so that the field maintenance efficiency of the worker can be improved.
In the embodiment, the fault occurrence position and the fault occurrence reason are accurately identified by acquiring the sound signal, the vibration signal and the infrared thermal image of the fault occurrence position. The sound signal, the vibration signal and the infrared thermal image are respectively subjected to feature vector extraction to obtain fault feature vectors, the fault feature vectors are respectively compared with a plurality of sample feature vectors in a fault three-dimensional knowledge map to calculate the similarity, the automatic identification of fault occurrence and fault occurrence reasons is improved through the three-dimensional knowledge map and feature comparison mode, and the problems of poor accuracy and low efficiency of manual fault identification are solved. And determining a fault reason and a target maintenance scheme according to the similarity and the fault three-dimensional knowledge graph, and visually displaying the target maintenance scheme. The method comprises the steps of obtaining real-time image information of a fault site, generating a virtual-real combined image according to the real-time image information and a target maintenance scheme, inputting the virtual-real combined image into a portable AR equipment display module for display, assisting maintenance, carrying out maintenance prompt for workers, and improving maintenance efficiency and safety.
In a possible implementation manner, the generating a virtual-real combined image according to the real-time image information and the target scheme in step S105 includes the following steps:
s1051, determining a corresponding three-dimensional model of the maintenance tool according to the target maintenance scheme, and generating a virtual image aiming at the three-dimensional model of the maintenance tool through an augmented reality technology;
s1052, determining a fault part corresponding to the fault occurrence position and the position of the fault part according to the real-time image information and the target maintenance scheme;
and S1053, generating a virtual-real combined image according to the real-time image information, the virtual image, the fault part and the position of the fault part.
The real-time image information is used as a background, and the AR display is carried out on the positions of the fault parts and the fault parts on the basis of the real-time image information by combining the target maintenance scheme, so that the positions of the fault parts related to the target maintenance scheme can be conveniently and quickly locked by workers, and the troubleshooting and maintenance work can be carried out, so that the field maintenance efficiency of the workers can be improved.
In a possible implementation manner, before acquiring the sound signal, the vibration signal and the infrared thermal image of the fault occurrence location in step S101, the method further includes:
s106, collecting maintenance data in a substation equipment maintenance record table and a maintenance manual;
s107, extracting vocabularies representing the relationship between the entities in the maintenance data in a knowledge extraction mode, and constructing entity-relationship-entity triple structure data;
the example that a certain maintenance record is "air in the oil pump causes abnormal sound of the oil pump, and the maintenance method is to remove air in the oil pump" is described, and a plurality of sets of triple structure data are constructed based on the maintenance record, for example: triple such as oil pump abnormal sound-fault parts-oil pump, oil pump abnormal sound-fault reasons-air in the pump, air in the pump-solution-oil pump air discharge and the like can be constructed.
In a possible implementation manner, after step S107, the method further includes: synonym classification and ambiguous word assignment is performed on words representing entities and relationships using knowledge fusion techniques (e.g., coreference elimination, entity disambiguation).
S108, collecting sound signals, vibration signals and infrared thermal images of a fault occurrence position when equipment is in fault, respectively extracting characteristic vectors, and constructing triples of fault reasons and the corresponding characteristic vectors;
s109, extracting the feature vectors of the fault parts corresponding to the fault occurrence positions, and constructing triples of the fault parts and the corresponding feature vectors;
s110, performing three-dimensional modeling on maintenance tools with different specifications to generate a maintenance tool three-dimensional model, performing unique numbering, and constructing a triple of a fault part and a maintenance tool number according to the characteristic matching of the equipment part and the maintenance tool;
and S111, importing all the constructed triples into a graph database to construct a fault three-dimensional knowledge graph.
In one embodiment, the triad arrangement of the constructed fault causes and the corresponding feature vectors is shown in table 1 below:
TABLE 1
In a possible implementation manner, the determining the fault cause and the target maintenance plan according to the similarity and the fault three-dimensional knowledge graph in step S104 includes:
s1041, sequencing the calculated multiple similarities in a descending order;
s1042, determining a fault reason according to the sample feature vectors and the fault three-dimensional knowledge graph corresponding to one or more similarity values sequenced in the front;
s1043, determining one or more corresponding maintenance schemes as recommended maintenance schemes according to the failure reasons, and visually displaying the recommended maintenance schemes;
s1044, obtaining a target maintenance scheme selection instruction, and determining a target maintenance scheme from the recommended maintenance schemes according to the target maintenance scheme selection instruction.
Referring to table 1 above, for the same fault phenomenon, multiple fault causes may exist, and for reasons, in order to improve the efficiency of identifying the fault causes, multiple signals of the sound signal, the vibration signal and the infrared thermal image are acquired, so as to investigate the fault causes, determine one or more fault causes with higher similarity, and further determine a corresponding maintenance scheme.
In a possible implementation manner, in step S104, the visual display of the recommended maintenance plan includes:
and when a plurality of recommended maintenance schemes exist, sequencing the recommended maintenance schemes according to the historical selection times, and carrying out visual display in sequence according to the selection times from the most to the least.
Wherein, because different staff experience is different, can adopt different maintenance means to maintain, the corresponding step is different in quantity and complexity. Visual display is carried out based on the historical selection times, and a user can conveniently obtain a maintenance scheme which is relatively simple and easy to operate or has a high success rate.
In one possible implementation manner, after inputting the virtual and real combined images into the display module of the portable AR device for display, the method further includes:
and S112, generating voice prompt information according to the maintenance steps of the target maintenance scheme.
This embodiment has realized that the characters show and the voice broadcast through the maintenance step carry out augmented reality supplementary maintenance, effectively indicates the staff to carry out the maintenance task according to the maintenance scheme step, improves maintenance security and maintenance effect.
In one possible implementation manner, after inputting the virtual and real combined images into the display module of the portable AR device for display, the method further includes:
s113, acquiring a real-time infrared thermal image of a fault site;
s114, identifying whether the fault field equipment is electrified or not according to the real-time infrared thermal image;
and S115, generating visual information according to the recognition result, and displaying the visual information in the set display area through the display module of the portable AR equipment.
Wherein, for avoiding showing information interference staff's the field of vision, portable AR equipment can carry out safety suggestion in setting for the display area, for example: AR field of view edge region.
In this embodiment, can timely effectual carry out the safety suggestion when trouble field device is electrified, avoid the staff to electrocute, improve the security.
In a specific embodiment, whether the equipment is charged or not is judged according to the infrared thermal image, the AR glasses identify the charged area as red, and the safety area as green, so that the personal safety of workers is guaranteed.
Optionally, when the charged area is identified, the worker is prompted to pay attention by vibration or voice.
Optionally, steps S112 and S113 to S115 provided in the foregoing embodiments are implemented in different embodiments, that is, in some embodiments, only step S112 is executed, and in other embodiments, only steps S113 to S115 are executed. Optionally, steps S112 and S113 to S115 provided in the foregoing embodiment are implemented in the same embodiment, and may be executed in parallel.
The foregoing embodiments mainly describe the process of determining the maintenance schedule and the process of performing the maintenance operation based on the maintenance schedule, and do not describe the process after the maintenance operation is finished.
In one possible implementation, the method further includes:
after the maintenance operation is finished, acquiring the sound signal, the vibration signal and the infrared thermal image of the fault position again;
respectively extracting feature vectors of the sound signal, the vibration signal and the infrared thermal image to obtain temporary feature vectors;
respectively comparing the temporary feature vectors with a plurality of sample feature vectors in a normally-operated three-dimensional knowledge graph to calculate similarity;
and determining that the maintenance is successful when the similarity is larger than the set threshold.
In a possible implementation manner, after the maintenance is determined to be successful, the maintenance process is added into the graph database, and the steps S106 to S111 are executed to enrich and perfect the fault three-dimensional knowledge graph, so that the subsequent maintenance for the related fault is facilitated.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by functions and internal logic of the process, and should not limit the implementation process of the embodiments of the present invention in any way.
The following are embodiments of the apparatus of the invention, reference being made to the corresponding method embodiments described above for details which are not described in detail therein.
Fig. 2 is a schematic structural diagram of a three-dimensional knowledge-graph-based auxiliary maintenance device for substation equipment AR according to an embodiment of the present invention, as shown in fig. 2, for convenience of description, only a part related to the embodiment of the present invention is shown, and as shown in fig. 2, the device includes: the system comprises an acquisition module 201, a feature extraction module 202, a calculation module 203, a visualization module 204 and a virtual-real combination module 205.
The acquiring module 201 is configured to acquire a sound signal, a vibration signal, and an infrared thermal image of a fault occurrence location.
A feature extraction module 202, configured to perform feature vector extraction on the sound signal, the vibration signal, and the infrared thermal image, respectively, to obtain a fault feature vector.
And the calculating module 203 is configured to compare the fault feature vector with a plurality of sample feature vectors in a fault three-dimensional knowledge graph respectively to calculate similarity.
And the visualization module 204 is used for determining a fault reason and a target maintenance scheme according to the similarity and the fault three-dimensional knowledge graph, and visually displaying the target maintenance scheme.
The obtaining module 201 is further configured to obtain real-time image information of a fault site.
And the virtual-real combination module 205 is configured to generate a virtual-real combination image according to the real-time image information and the target maintenance scheme, and input the virtual-real combination image into a portable AR device display module for display to assist in maintenance. Wherein, the fault three-dimensional knowledge graph comprises: fault causes, and three-dimensional models and sample characteristic vectors of fault parts, maintenance schemes and maintenance tools corresponding to the fault causes; the sample feature vector includes: a sound signal sample feature vector, a vibration signal sample feature vector, and an infrared thermal image sample feature vector.
In the embodiment, the fault occurrence position and the fault occurrence reason are accurately identified by acquiring the sound signal, the vibration signal and the infrared thermal image of the fault occurrence position. The sound signal, the vibration signal and the infrared thermal image are respectively subjected to feature vector extraction to obtain fault feature vectors, the fault feature vectors are respectively compared with a plurality of sample feature vectors in a fault three-dimensional knowledge map to calculate the similarity, the automatic identification of fault occurrence and fault occurrence reasons is improved through the three-dimensional knowledge map and feature comparison mode, and the problems of poor accuracy and low efficiency of manual fault identification are solved. And determining a fault reason and a target maintenance scheme according to the similarity and the fault three-dimensional knowledge graph, and carrying out visual display on the target maintenance scheme. The method comprises the steps of obtaining real-time image information of a fault site, generating a virtual-real combined image according to the real-time image information and a target maintenance scheme, inputting the virtual-real combined image into a portable AR equipment display module for display, assisting maintenance, carrying out maintenance prompt for workers, and improving maintenance efficiency and safety.
Fig. 3 is a schematic structural diagram of a terminal according to an embodiment of the present invention. As shown in fig. 3, the terminal 3 of this embodiment includes: a processor 30, a memory 31 and a computer program 32 stored in said memory 31 and executable on said processor 30. The processor 30, when executing the computer program 32, implements the steps in each of the above-described embodiments of the three-dimensional knowledge-graph-based auxiliary repair method for the substation equipment AR, such as the steps S101 to S105 shown in fig. 2. Alternatively, the processor 30, when executing the computer program 32, implements the functions of each module/unit in each device embodiment described above, for example, the functions of the modules 201 to 205 shown in fig. 2.
Illustratively, the computer program 32 may be partitioned into one or more modules/units that are stored in the memory 31 and executed by the processor 30 to implement the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions for describing the execution of the computer program 32 in the terminal 3. For example, the computer program 32 may be divided into the modules 201 to 205 shown in fig. 2.
The terminal 3 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The terminal 3 may include, but is not limited to, a processor 30, a memory 31. It will be appreciated by those skilled in the art that fig. 3 is merely an example of a terminal 3 and does not constitute a limitation of the terminal 3, and may include more or fewer components than shown, or some components may be combined, or different components, e.g. the terminal may also include input output devices, network access devices, buses, etc.
The Processor 30 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 31 may be an internal storage unit of the terminal 3, such as a hard disk or a memory of the terminal 3. The memory 31 may also be an external storage device of the terminal 3, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) and the like provided on the terminal 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the terminal 3. The memory 31 is used for storing the computer program and other programs and data required by the terminal. The memory 31 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only used for distinguishing one functional unit from another, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal and method may be implemented in other ways. For example, the above-described apparatus/terminal embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated module/unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the method according to the embodiment of the present invention may be implemented by instructing relevant hardware by a computer program, where the computer program may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps in each embodiment of the method for assisting maintenance of the substation equipment AR based on the three-dimensional knowledge graph. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U.S. disk, removable hard disk, magnetic diskette, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signal, telecommunications signal, and software distribution medium, etc.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (10)
1. A transformer substation equipment AR auxiliary maintenance method based on a three-dimensional knowledge graph is characterized by comprising the following steps:
acquiring a sound signal, a vibration signal and an infrared thermal image of a fault occurrence position;
respectively extracting feature vectors of the sound signal, the vibration signal and the infrared thermal image to obtain fault feature vectors;
respectively comparing the fault characteristic vector with a plurality of sample characteristic vectors in a fault three-dimensional knowledge graph to calculate similarity;
determining a fault reason and a target maintenance scheme according to the similarity and the fault three-dimensional knowledge graph, and visually displaying the target maintenance scheme;
acquiring real-time image information of a fault site, generating a virtual-real combined image according to the real-time image information and the target maintenance scheme, and inputting the virtual-real combined image into a portable AR equipment display module for display so as to assist maintenance; wherein, the fault three-dimensional knowledge graph comprises: fault causes, and three-dimensional models and sample characteristic vectors of fault parts, maintenance schemes and maintenance tools corresponding to the fault causes; the sample feature vector includes: a sound signal sample feature vector, a vibration signal sample feature vector, and an infrared thermal image sample feature vector.
2. The method of claim 1, wherein generating a virtual-real combined image from the real-time image information and the target solution comprises:
determining a three-dimensional model of a corresponding maintenance tool according to the target maintenance scheme, and generating a virtual image aiming at the three-dimensional model of the maintenance tool through an augmented reality technology;
determining fault parts and positions of the fault parts corresponding to the fault occurrence positions according to the real-time image information and the target maintenance scheme;
and generating a virtual-real combined image according to the real-time image information, the virtual image, the fault part and the position of the fault part.
3. The method according to claim 1 or 2, wherein before the acquiring the sound signal, the vibration signal and the infrared thermal image of the fault occurrence position, further comprising:
acquiring maintenance data in a substation equipment maintenance record table and a maintenance manual;
extracting vocabularies representing the entities and the relationships among the entities in the maintenance data in a knowledge extraction mode, and constructing entity-relationship-entity triple structure data;
collecting a sound signal, a vibration signal and an infrared thermal image of a fault occurrence position when equipment is in fault, respectively extracting characteristic vectors, and constructing a triple of fault reasons and the corresponding characteristic vectors;
extracting the characteristic vector of the fault part corresponding to the fault occurrence position, and constructing a triple of the fault part and the corresponding characteristic vector;
three-dimensional modeling is carried out on maintenance tools with different specifications, a maintenance tool three-dimensional model is generated, unique numbering is carried out, and a triple of the numbers of the fault parts and the maintenance tools is constructed according to the characteristic matching of the equipment parts and the maintenance tools;
and leading all the constructed triples into a graph database to construct a fault three-dimensional knowledge graph.
4. The method of claim 1, wherein determining a fault cause and a target repair plan based on the similarity and the fault three-dimensional knowledge-graph comprises:
performing descending sorting on the plurality of similarity degrees obtained by calculation;
determining a fault reason according to the sample feature vectors corresponding to one or more similarity values sequenced in the front and the fault three-dimensional knowledge graph;
determining one or more corresponding maintenance schemes as recommended maintenance schemes according to the fault reasons, and carrying out visual display on the recommended maintenance schemes;
and acquiring a target maintenance scheme selection instruction, and determining a target maintenance scheme from the recommended maintenance schemes according to the target maintenance scheme selection instruction.
5. The method of claim 4, wherein the visually presenting the recommended repair solutions comprises:
and when a plurality of recommended maintenance schemes exist, sequencing the recommended maintenance schemes according to the historical selection times, and carrying out visual display in sequence according to the selection times from the most to the least.
6. The method of claim 1, wherein after inputting the virtual-real combined image into a display module of a portable AR device for presentation, further comprising:
and generating voice prompt information according to the maintenance steps of the target maintenance scheme.
7. The method of claim 1, wherein after inputting the virtual-real combined image into a display module of a portable AR device for presentation, further comprising:
acquiring a real-time infrared thermal image of a fault site;
identifying whether the fault field equipment is electrified or not according to the real-time infrared thermal image;
and generating visual information according to the recognition result, and displaying the visual information in the set display area through a display module of the portable AR equipment.
8. The method of claim 1, further comprising:
when the maintenance operation is finished, acquiring the sound signal, the vibration signal and the infrared thermal image of the fault occurrence position again;
respectively extracting feature vectors of the sound signal, the vibration signal and the infrared thermal image to obtain temporary feature vectors;
respectively comparing the temporary feature vectors with a plurality of sample feature vectors in a normally-operated three-dimensional knowledge graph to calculate similarity;
and determining that the maintenance is successful when the similarity is larger than the set threshold.
9. A terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of the preceding claims 1 to 8 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211035697.7A CN115510253A (en) | 2022-08-26 | 2022-08-26 | Transformer substation equipment AR auxiliary maintenance method and terminal based on three-dimensional knowledge graph |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211035697.7A CN115510253A (en) | 2022-08-26 | 2022-08-26 | Transformer substation equipment AR auxiliary maintenance method and terminal based on three-dimensional knowledge graph |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115510253A true CN115510253A (en) | 2022-12-23 |
Family
ID=84502623
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211035697.7A Pending CN115510253A (en) | 2022-08-26 | 2022-08-26 | Transformer substation equipment AR auxiliary maintenance method and terminal based on three-dimensional knowledge graph |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115510253A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117253129A (en) * | 2023-11-16 | 2023-12-19 | 国网山东省电力公司枣庄供电公司 | Deep learning substation equipment monitoring and analyzing system based on AR technology |
| CN118135143A (en) * | 2024-05-07 | 2024-06-04 | 成都市技师学院(成都工贸职业技术学院、成都市高级技工学校、成都铁路工程学校) | AR-based aeroengine maintenance modeling method |
| CN118133958A (en) * | 2024-05-07 | 2024-06-04 | 陕西万禾数字科技有限公司 | Military combat command system and method based on augmented reality and knowledge graph |
-
2022
- 2022-08-26 CN CN202211035697.7A patent/CN115510253A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117253129A (en) * | 2023-11-16 | 2023-12-19 | 国网山东省电力公司枣庄供电公司 | Deep learning substation equipment monitoring and analyzing system based on AR technology |
| CN118135143A (en) * | 2024-05-07 | 2024-06-04 | 成都市技师学院(成都工贸职业技术学院、成都市高级技工学校、成都铁路工程学校) | AR-based aeroengine maintenance modeling method |
| CN118133958A (en) * | 2024-05-07 | 2024-06-04 | 陕西万禾数字科技有限公司 | Military combat command system and method based on augmented reality and knowledge graph |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115510253A (en) | Transformer substation equipment AR auxiliary maintenance method and terminal based on three-dimensional knowledge graph | |
| CN110221145B (en) | Power equipment fault diagnosis method and device and terminal equipment | |
| CN104794136A (en) | Fault analysis method and device | |
| CN113592019B (en) | Fault detection method, device, equipment and medium based on multi-model fusion | |
| CN112115602B (en) | Coal mill pipeline blocking state determining method and device, storage medium and computing equipment | |
| US11734954B2 (en) | Face recognition method, device and electronic equipment, and computer non-volatile readable storage medium | |
| CN112288883B (en) | Method and device for prompting operation guide information, electronic equipment and storage medium | |
| CN111798162A (en) | Risk monitoring method and device based on neural network | |
| CN115730605A (en) | Data analysis method based on multi-dimensional information | |
| CN111695979A (en) | Method, device and equipment for analyzing relation between raw material and finished product | |
| CN113743725B (en) | Ascending qualification detection method and system for ascending operators based on edge calculation | |
| CN113192028B (en) | Quality evaluation method and device for face image, electronic equipment and storage medium | |
| CN111325572B (en) | Data processing method and device | |
| CN115578551A (en) | Household ammeter state checking method based on image recognition | |
| CN111507570A (en) | Engineering project construction process monitoring method and system, intelligent terminal and storage medium | |
| CN110414792A (en) | Component centralized procurement management system and Related product based on BIM and big data | |
| CN112084092B (en) | Method, device, equipment and storage medium for determining diagnosis rule | |
| CN113033469A (en) | Tool damage identification method, device, equipment, system and readable storage medium | |
| CN112732773A (en) | Uniqueness checking method and system for relay protection defect data | |
| US20190384272A1 (en) | System section data management device and method thereof | |
| CN104238419A (en) | Handheld energy-saving monitoring intelligent working system and working method | |
| CN116307407B (en) | Enterprise data visualization processing system and method based on cloud computing | |
| CN116881335B (en) | Multi-mode data intelligent analysis system and method | |
| CN111639857B (en) | Order intelligent auditing method and device, electronic equipment and storage medium | |
| CN118337534A (en) | Data monitoring system for determining abnormal flow |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |