CN111679740A - Method for carrying out remote intelligent diagnosis on power station equipment by using Augmented Reality (AR) technology - Google Patents

Method for carrying out remote intelligent diagnosis on power station equipment by using Augmented Reality (AR) technology Download PDF

Info

Publication number
CN111679740A
CN111679740A CN202010508416.XA CN202010508416A CN111679740A CN 111679740 A CN111679740 A CN 111679740A CN 202010508416 A CN202010508416 A CN 202010508416A CN 111679740 A CN111679740 A CN 111679740A
Authority
CN
China
Prior art keywords
technology
remote
diagnosis
equipment
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010508416.XA
Other languages
Chinese (zh)
Inventor
徐创学
冯庭有
吴波
江永
李振兴
周明
赵骥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian Thermal Power Research Institute Co Ltd
Huaneng Group Technology Innovation Center Co Ltd
Original Assignee
Xian Thermal Power Research Institute Co Ltd
Huaneng Group Technology Innovation Center Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian Thermal Power Research Institute Co Ltd, Huaneng Group Technology Innovation Center Co Ltd filed Critical Xian Thermal Power Research Institute Co Ltd
Priority to CN202010508416.XA priority Critical patent/CN111679740A/en
Publication of CN111679740A publication Critical patent/CN111679740A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
    • G06F16/367—Ontology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/43—Querying
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/0486—Drag-and-drop
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/20—Administration of product repair or maintenance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/06—Energy or water supply
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04806—Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20—Indexing scheme for editing of 3D models
    • G06T2219/2016—Rotation, translation, scaling
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Business, Economics & Management (AREA)
  • Software Systems (AREA)
  • Human Resources & Organizations (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Hardware Design (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Computer Graphics (AREA)
  • Evolutionary Computation (AREA)
  • General Business, Economics & Management (AREA)
  • Tourism & Hospitality (AREA)
  • Geometry (AREA)
  • Architecture (AREA)
  • Quality & Reliability (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Operations Research (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Public Health (AREA)
  • Water Supply & Treatment (AREA)
  • Primary Health Care (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Computational Linguistics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The method for carrying out remote intelligent diagnosis on power station equipment by utilizing Augmented Reality (AR) technology fuses a virtual world and a real world by utilizing AR holographic technology, realizes that remote diagnosis parties carry out face-to-face communication and exchange through instant messaging technology, and solves the problem by assisting with a handle; the complete AR object connection consisting of AR visual positioning, registration, recognition and tracking is utilized, and then the 'man-machine' connection of remote diagnosis is realized through the direct recognition and control of gestures and postures; the connection application of data is realized by utilizing end-cloud and AI data modeling and visualization technologies; the physical environment and the virtual environment are fused by utilizing AR enhanced simulation and projection technology, and all parties cooperate to share the picture of the first visual angle in multiple ways through AR, so that a scene in which the virtual environment and the real world are superposed is provided, cooperation is more accurate and efficient, and the method has positive significance for improving safety, reliability and availability of power station equipment and reducing diagnosis and maintenance cost.

Description

Method for carrying out remote intelligent diagnosis on power station equipment by using Augmented Reality (AR) technology
Technical Field
The invention relates to a remote intelligent diagnosis method, in particular to a method for carrying out remote intelligent diagnosis on power station equipment by utilizing an Augmented Reality (AR) technology, which realizes human-machine-number-ring-known cyclic interaction to carry out remote fault diagnosis based on the AR technology.
Background
Power plants (power stations) are capital and technology intensive enterprises, and the safe and efficient operation of their equipment is a constant goal pursued by the enterprises. In the process of power generation, any fault in the operation process of important equipment not only directly affects the yield and quality of electric energy, but also can cause serious equipment and personal accidents. Process state monitoring and fault diagnosis are important and effective technical means for the safe, reliable and economical operation of unit equipment. In addition, due to the requirements of the unit on flexible operation, environmental protection emission and the like, professionals with abundant experience and knowledge are needed to perform technical service in the process of analyzing and diagnosing the equipment operation, maintenance, reliability and economy of the power plant. With the increase of the digitization degree of enterprises and the enlargement of equipment, the equipment fault diagnosis needs more and more resources and more processed data, needs to be supported by a large amount of scientific calculation and knowledge experience, and enables a single enterprise not to solve all problems.
The traditional field technical service mode has the defects of long project period, high execution cost, high human resource occupancy rate, high labor intensity, poor emergency and the like. The field diagnosis and analysis of the equipment are easily limited by manpower, technology and regions, and the sharing and reusing of historical knowledge are low.
With the rapid development of computer network technology, the remote diagnosis of devices is more and more widely regarded. However, the traditional remote diagnosis has technical and spatial obstacles in flexible data acquisition, field device connection, environment perception, personnel communication and remote knowledge transmission, and restricts the practical application effect of the remote diagnosis.
Disclosure of Invention
In order to solve the problems faced by the traditional remote diagnosis and cater to the development trend of intelligent power stations, the invention aims to provide a method for carrying out remote intelligent diagnosis on power station equipment by using an Augmented Reality (AR) technology, and a method for carrying out remote intelligent diagnosis on power station equipment developed by using the AR technology.
In order to achieve the purpose, the technical scheme of the invention is as follows:
a method for carrying out remote intelligent diagnosis on power station equipment by utilizing an Augmented Reality (AR) technology comprises the following steps:
1. the AR holographic technology and the instant messaging technology are utilized to connect the on-site maintenance staff with the remote expert in a face-to-face way, so that the remote diagnosis is realized
The method specifically comprises the following steps: on the basis of instant messaging, a field maintenance worker and a remote expert beyond thousands of miles away are virtualized in a room position by an AR holographic imaging display technology, a depth-of-field sensing camera and a digital projector are used for creating a virtual space, the virtual world and the real world are fused together, the field maintenance worker and the remote expert see each other from various different angles through holographic digital images, the two parties communicate and communicate face to face, the problem is solved by the aid of a handle, and the two parties can perform graphic drawing and show working principle flows and solutions on the same working whiteboard;
2. human-machine connection of on-site overhaul machine and remote expert by utilizing AR equipment identification tracking and control perception technology
Firstly, an ARAR device is utilized to position, register, identify and track a field machine to form complete AR object connection, and data are subjected to three-dimensional perspective on the machine through 2D, 3D and 4D image technologies of AR by means of a computer;
in the aspect of object control perception, the gesture and limb interaction with diagnostic machine equipment is realized through direct recognition and control of the gesture and the posture, the click, touch, sliding and hand posture feedback are realized through tracking a plurality of joints of a hand, in addition, voice synthesis control is also provided, the human-computer interaction function is controlled through will, a remote expert can know which part has a problem only by looking up and down at equipment in operation, and therefore, the user can be helped to understand the structure and the operation state of the equipment and further check and find the problem by directly dragging, rotating, zooming and operating the perspective view of the field equipment and the parts in a 3D virtual space;
3. data modeling visual connection of field operation information and remote diagnosis center by using 'end-cloud' technology
The remote diagnosis needs to collect a large amount of field operation data including real-time data and historical data, the data has the characteristics of real-time mass and is transmitted and stored to a diagnosis data center, an artificial intelligence AI is used for finding potential rules from the large amount of operation data, extracting equipment characteristic data and modeling operation rules, an AR is used for performing high-dimensional display of models of the characteristic parameters and providing auxiliary decision for a diagnostician, and the quantity value comprises: maximum value, minimum value, peak value, mean square value, variance, square root amplitude, average amplitude and average peak value;
4. the physical environment and the virtual environment are fused by utilizing AR enhanced simulation and projection technology, and the on-site environment is virtualized to a remote diagnosis center for environment simulation connection
The AR enables all parties to share the pictures at the first visual angle in a coordinated manner, a scene in which a virtual environment is overlapped with the real world is provided for a remote expert, the physical space and the virtual space are calibrated, fused and controlled, the visual field presented to people is not only seen by the eyes of overhaul staff, but also virtual space objects, data and models, so that the whole remote diagnosis work is immersed in the scene of augmented reality, the remote expert directly knows the content seen by the on-site overhaul staff, the on-site work and maintenance risk coefficient can be known more clearly, and the guidance result of the remote expert is synchronously displayed in the visual field of an on-site user;
5. method for realizing equipment operation characteristic parameter extraction and knowledge graph establishment by using AR remote diagnosis
The power station equipment diagnosis is an equipment state evaluation technology based on real-time/historical data mining component characteristics, unit characteristic parameters are obtained from a large amount of historical data through an artificial intelligent calculation method comprising an artificial neural network, a genetic algorithm, mode identification and mathematical statistics, and the performance and fault diagnosis is carried out on the operation state of main equipment in a mode matching mode; in the fault diagnosis, one-hand data of a site is directly recorded and stored, the task state is tracked in real time, and meanwhile, various types of knowledge map data including documents, pictures, videos and data models can be inquired; the acquisition and analysis of field equipment operation big data are realized through AR technology, so that the precipitation of an empirical method and the optimization of a flow are carried out, and the establishment of an equipment diagnosis knowledge graph is realized.
The invention has the advantages that:
the invention carries out remote fault diagnosis according to the circulation interaction of 5 characteristic elements of 'man-machine-number-ring-know', fuses a virtual world and a real world by utilizing an AR holographic technology, realizes remote diagnosis and multi-party face-to-face 'communication and communication by an instant messaging (information expressed by characters, voice, pictures and videos) technology, and assists in solving problems by a' hand; the complete AR object connection consisting of ARAR visual positioning, registration, recognition and tracking is utilized, and then the 'man-machine' connection of remote diagnosis is realized through the direct recognition and control of gestures and postures; the connection application of data is realized by utilizing end-cloud and AI data modeling and visualization technologies; the physical environment and the virtual environment are fused by utilizing AR enhanced simulation and projection technology, and all parties cooperate to share the picture of the first visual angle in multiple ways through AR, so that a scene in which the virtual environment and the real world are superposed is provided, and cooperation is more accurate and efficient. And extending remote diagnostics from desktop level to environmental level; the AR + AI remote diagnosis is applied to realize the extraction of the equipment operation characteristic parameters and the establishment of the knowledge graph. Based on the remote diagnosis of the AR technology, the remote diagnosis can really realize the cooperation of multiple experts, multiple means and multiple tools, and can realize regional and even global diagnosis resource sharing and rapid fault diagnosis; the intelligent power station intelligent monitoring system provides diversified inspirations for rapid development of intelligent power stations, and has positive significance for improving safety, reliability and availability of power station equipment and reducing diagnosis and maintenance cost.
Drawings
FIG. 1 is a diagram of the relationship between features and remote diagnosis of power plant equipment using Augmented Reality (AR) technology.
FIG. 2 is a connection block diagram of the present invention for performing remote intelligent diagnosis of power station equipment by using Augmented Reality (AR) technology.
Detailed Description
The invention will be further described with reference to fig. 1 and 2 of the specification.
As shown in fig. 1, the method for performing remote intelligent diagnosis on power station equipment by using Augmented Reality (AR) technology of the present invention is a method for performing remote fault diagnosis by implementing human-machine-digital-loop-known loop interaction, and has 5 specific elements as shown in fig. 1:
1. the AR holographic technology and the instant messaging technology are utilized to connect the on-site maintenance staff with the remote expert in a face-to-face way, so that the remote diagnosis is realized
The remote diagnosis is often performed by remote assistance, and currently, a telephone, a video and a chat tool can provide remote assistance, but experience is unsatisfactory, for example, the telephone cannot represent scene pictures of two parties in conversation, the video remote assistance can provide a scene picture, but it is difficult to perform hand-held assistance, and the remote assistance of the chat tool is generally unidirectional (such as QQ).
The method comprises the following steps: on the basis of instant messaging (information expressed by characters, voice, pictures and videos), on-site maintenance staff and remote experts far away from thousands of miles are virtually located in one room through an AR holographic imaging display technology. In the past, own information can only be expressed through characters, voice, pictures and videos of a screen, a powerful AR virtual holographic imaging technology is applied, a depth-of-field sensing camera and a digital projector are used for creating a virtual space, the virtual world and the real world are integrated, and an unprecedented interactive mode is provided by utilizing the technologies of powerful real-time 3D computing capacity, accurate instant positioning, gesture recognition technology, three-dimensional head-up reality technology and the like of the AR. The field maintenance staff and the remote expert can see each other from various angles through the holographic digital image, the two parties can communicate and communicate with each other approximately in a face-to-face mode, the problems are solved by assistance of audio and video chat interaction and a hand, and the two parties can draw graphs and display working principle flows and solutions on the same working white board.
The remote expert can circle the problem in the screen by using a finger or a pen, and the opposite side can also see the circled picture at the same time, so that the problem in practice is solved. Thus, the remote expert can guide the user to operate in a 'field' mode so as to solve the problems encountered in the actual work.
The AR multi-person communication supports various scenes such as one-to-many, many-to-one, many-to-many and the like, and all parties participating in diagnosis can freely select the type of the terminal equipment accessing the communication.
2. Human-machine connection of on-site overhaul machine and remote expert by utilizing AR equipment identification tracking and control perception technology
By utilizing the AR visual positioning, registration, identification and tracking of the complete AR object connection technology. The complex data are subjected to three-dimensional perspective on a machine through 2D, 3D and 4D image technologies of AR by means of a computer, and the method is a precondition of man-machine interaction connection.
In the aspect of object control perception, the system can realize the interaction with the gestures and limbs of the diagnosis machine equipment through the direct recognition and control of the gestures and postures, can realize the clicking, touching, sliding and hand posture feedback through common gestures, can also be controlled by voice and graphic markers, and can realize the direct dragging, rotating, zooming and operating of the field equipment and the perspective view of parts thereof in a 3D virtual space. Therefore, a remote expert can know which part is in problem only by 'looking' at the running equipment from top to bottom, and can even judge whether the equipment needs to be stopped for maintenance.
Through tracking a plurality of joints of hand, realize clicking, the touch, slide and feedback hand gesture, in addition there is stronger human-computer interaction ability function: speech synthesis control, idea control, etc. This directly drags, rotates, zooms, manipulates perspective views of the field device and its components in 3D virtual space, helping the user understand the device structure and operating state for further inspection and problem discovery.
3. Data modeling visual connection of field operation information and remote diagnosis center by using 'end-cloud' technology
As shown in fig. 2, the remote diagnosis center provides data cloud services (IaaS/PaaS/SaaS), the AR terminal is deployed on site, and the two terminals are connected in a "end-cloud" manner.
Remote diagnosis requires a large amount of field operation data, and the data has the characteristic of real-time mass. According to the data and modeling connection requirements, the cross-platform transmission and diversified display of the data are realized based on the characteristics of XML language expandability, high structuralization and convenience for network transmission, and the marking, the attribute and the description method are customized, so that the data connection can be timely performed for remote diagnosis personnel with high mobility of a working place.
And for the received big data set, adopting an AR + AI working processing mode. The power station equipment diagnosis is an equipment state evaluation technology based on real-time/historical data mining component characteristics, potential rules are found from a large amount of operation data by using artificial intelligence AI, equipment characteristic data are extracted, and operation rule modeling is performed, and the AR performs high-dimensional display on the magnitude values (including maximum value, minimum value, peak value, mean square value, variance, square root amplitude value, mean amplitude value and mean peak value) of the characteristic parameters, so as to assist a diagnostician in decision making.
The AR terminal is deployed on site, data and models are stored in the cloud, end cloud connection interaction is carried out through the AR + AI technology, and the integration of computing power and an algorithm and the combination of the technology and an application scene are achieved.
4. The physical environment and the virtual environment are fused by utilizing AR enhanced simulation and projection technology. Virtualizing on-site environment to remote diagnosis center for environment simulation connection
The AR enables all the cooperative parties to share the pictures through the first visual angle in a multi-party mode, a scene in which a virtual environment is overlapped with the real world is provided for a remote expert, as shown in figure 2, the physical space and the virtual space are calibrated, fused and controlled, so that the whole remote diagnosis work is immersed in the scene of augmented reality, the remote expert can not only see the contents seen by the eyes of overhaul personnel, but also directly know the contents seen by an on-site overhaul worker, the on-site work and maintenance risk coefficients can be known more clearly, and the guidance result of the remote expert is synchronously displayed in the visual field of an on-site user. The screen freezing and real-time marking technology can be realized through one key, the on-the-spot maintenance guidance is provided, and the cooperation is more accurate and efficient.
The screen well known in daily life is replaced by the AR environment projection mapping mode, 360-degree projection immersive environment experience is achieved, remote diagnosis is extended from a desktop level to an environment level, namely, a traditional desktop remote connection mode is interactively developed to a large scene of a field physical world, and a ubiquitous user interaction interface is achieved.
5. The AR remote diagnosis is applied to realize the extraction of the characteristic value parameters of the equipment operation and the establishment of the knowledge graph
In the industrial field, large-scale equipment has complex mechanism, complex overhaul operation, long flow and high requirements on operation specification, efficiency and safety. The pressure of field inspection and maintainers is high.
The power station equipment diagnosis is an equipment state evaluation technology based on real-time/historical data mining component characteristics, unit characteristic parameters are obtained from a large amount of historical data through an artificial intelligence calculation method (artificial neural network, genetic algorithm), mode identification and mathematical statistics, and performance and fault diagnosis is carried out on the operation state of main equipment in a mode matching mode.
In the AR diagnosis process, one-hand data of a site can be directly recorded and stored, and the task state is tracked in real time, so that the information management and the information tracing are facilitated. And meanwhile, the knowledge graph data of various types (including documents, pictures, videos, data models and the like) can be inquired.
The AR technology is used for realizing the collection and analysis of the large running data of the field equipment, so that the precipitation of the empirical method and the optimization of the process are carried out. Meanwhile, an expert data model and a related intelligent algorithm are constructed, and real remote intelligent diagnosis is realized.
The invention discloses a method for carrying out remote intelligent diagnosis on power station equipment by utilizing Augmented Reality (AR) technology, wherein related personnel carry out previous case analysis and judgment by utilizing an AR intelligent terminal, such as AR intelligent glasses, a built-in software algorithm and a work auxiliary system by utilizing the AR technology, an optimal working scheme is iterated, and standard steps needing real operation are superposed in the real world through three-dimensional images and emerge in front of the eyes. All steps are clear, and the error rate is greatly reduced.
With the development of the industrial internet of things technology, the remote fault diagnosis and maintenance technology can realize quick, timely and correct diagnosis and maintenance of a complex system. The remote diagnosis not only can timely carry out analysis and diagnosis on the unit and reduce the processing time of faults, but also can provide equipment state evaluation, fault early warning, performance degradation analysis, energy-saving analysis and operation guidance for power generation enterprises, so that the operation safety, reliability and availability of the equipment can be improved.
The remote diagnosis based on the AR technology is the combination of a computer, a modern information technology and a traditional engineering technology service, and can realize the cooperation of multiple experts, multiple means and multiple tools and realize regional and even global diagnosis resource sharing and rapid fault diagnosis in the real sense through the human-machine-digital-ring-known circular interaction. The invention has the advantages that the information flow instead of the human body is moved in the diagnosis process, the development trend of the Internet plus technology is met, the method becomes one of the directions of industrial intelligent development, diversified inspirations are provided for the rapid development of intelligent power stations, and the invention has good application prospect.
The system architecture, main contents and main features of the present invention have been shown and described. It will be appreciated by those skilled in the art that the present invention is not limited by the examples of application, which are set forth herein only to illustrate the principles and compositions of the present invention, but that various changes and modifications may be made without departing from the spirit and scope of the invention, which is intended to be covered by the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.

Claims (6)

1. The method for carrying out remote intelligent diagnosis on the power station equipment by utilizing the augmented reality AR technology is characterized by comprising the following steps:
(1) the AR holographic technology and the instant messaging technology are utilized to connect the on-site maintenance staff with the remote expert in a face-to-face manner, so that remote diagnosis is realized;
(2) the on-site maintenance machine is in man-machine connection with a remote expert by utilizing an AR equipment identification tracking and control perception technology;
(3) carrying out data modeling visual connection on the field operation information and the remote diagnosis center by using a 'terminal-cloud' technology;
(4) fusing a physical environment and a virtual environment by utilizing an AR enhanced simulation and projection technology, and virtualizing a field environment to a remote diagnosis center for environment simulation connection;
(5) and realizing equipment operation characteristic parameter extraction and knowledge graph establishment by using AR remote diagnosis.
2. The method for remote intelligent diagnosis of power plant equipment using Augmented Reality (AR) technology as claimed in claim 1,
the method is characterized in that the AR holographic technology and the instant messaging technology are utilized to connect field overhaul personnel with remote experts in a face-to-face manner, so that remote diagnosis is realized, and the method specifically comprises the following steps: on the basis of instant messaging, a field maintenance worker and a remote expert far away from thousands of miles are virtualized in a room position by an AR holographic imaging display technology, a virtual space is created by using a depth-of-field sensing camera and a digital projector, the virtual world and a real world are fused together, the field maintenance worker and the remote expert can see each other from various different angles through holographic digital images, two parties communicate and communicate face to face, the problem is solved by the aid of a handle, and the two parties can perform graphic drawing and show a working principle flow and a solution on the same working whiteboard.
3. The method for remote intelligent diagnosis of power plant equipment using Augmented Reality (AR) technology as claimed in claim 1,
the method is characterized in that the on-site maintenance machine is in man-machine connection with a remote expert by utilizing an AR equipment identification tracking and control perception technology, and specifically comprises the following steps:
firstly, an ARAR device is utilized to position, register, identify and track a field machine to form complete AR object connection, and data are subjected to three-dimensional perspective on the machine through 2D, 3D and 4D image technologies of AR by means of a computer;
in the aspect of object control perception, the gesture and the limb interaction of diagnosis machine equipment are realized through direct recognition and control of the gesture and the posture, the click, touch, sliding and hand posture feedback are realized through tracking a plurality of joints of a hand, in addition, voice synthesis control is also realized, the human-computer interaction function is controlled through will, a remote expert can know which part has a problem only by looking up and down at equipment in operation at a glance, and therefore, the user is helped to understand the equipment structure and the operation state and further check and find the problem by directly dragging, rotating, zooming and operating the field equipment and the perspective views of the parts in a 3D virtual space.
4. The method for remote intelligent diagnosis of power plant equipment using Augmented Reality (AR) technology as claimed in claim 1,
the method is characterized in that data modeling visual connection is carried out on the field operation information and the remote diagnosis center by using an end-cloud technology, and specifically comprises the following steps:
the remote diagnosis needs to collect a large amount of field operation data including real-time data and historical data, the data has the characteristics of real-time mass and is transmitted and stored to a diagnosis data center, an artificial intelligence AI is used for finding potential rules from the large amount of operation data, extracting equipment characteristic data and modeling operation rules, an AR is used for performing high-dimensional display of models of the characteristic parameters and providing auxiliary decision for a diagnostician, and the quantity value comprises: maximum, minimum, peak, mean, variance, square root amplitude, average peak.
5. The method for remote intelligent diagnosis of power plant equipment using Augmented Reality (AR) technology as claimed in claim 1,
the method is characterized in that the physical environment and the virtual environment are fused by utilizing the AR enhancement simulation and projection technology, and the on-site environment is virtualized to the remote diagnosis center for environment simulation connection, and specifically comprises the following steps:
through AR, all the cooperative parties share the pictures at the first visual angle in a multi-party mode, and a scene in which a virtual environment is overlapped with the real world is provided for a remote expert; through carrying out calibration, integration and control with physical space and virtual space, the field of vision that presents in front of everybody is not only what overhaul personnel's eyes seen, and virtual space object, data and model in addition, and whole work of remote diagnosis is immersed in augmented reality's scene like this, and remote expert directly knows the content that on-the-spot maintainer saw, can know on-the-spot work more clearly, maintenance risk coefficient, and remote expert's guidance result will show in on-the-spot user field of vision in step.
6. The method for remote intelligent diagnosis of power plant equipment using Augmented Reality (AR) technology as claimed in claim 1,
the AR remote diagnosis realizes the equipment operation characteristic parameter extraction and the knowledge graph establishment specifically as follows:
the equipment state evaluation technology based on real-time/historical data mining component characteristics obtains unit characteristic parameters from a large amount of historical data through an artificial intelligent calculation method comprising an artificial neural network, a genetic algorithm, pattern recognition and mathematical statistics, and performs performance and fault diagnosis on the running state of main equipment in a pattern matching mode; in the fault diagnosis, one-hand data of a site is directly recorded and stored, the task state is tracked in real time, and meanwhile, various types of knowledge map data including documents, pictures, videos and data models can be inquired; the acquisition and analysis of field equipment operation big data are realized through AR technology, so that the precipitation of an empirical method and the optimization of a flow are carried out, and the establishment of an equipment diagnosis knowledge graph is realized.
CN202010508416.XA 2020-06-06 2020-06-06 Method for carrying out remote intelligent diagnosis on power station equipment by using Augmented Reality (AR) technology Pending CN111679740A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010508416.XA CN111679740A (en) 2020-06-06 2020-06-06 Method for carrying out remote intelligent diagnosis on power station equipment by using Augmented Reality (AR) technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010508416.XA CN111679740A (en) 2020-06-06 2020-06-06 Method for carrying out remote intelligent diagnosis on power station equipment by using Augmented Reality (AR) technology

Publications (1)

Publication Number Publication Date
CN111679740A true CN111679740A (en) 2020-09-18

Family

ID=72435233

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010508416.XA Pending CN111679740A (en) 2020-06-06 2020-06-06 Method for carrying out remote intelligent diagnosis on power station equipment by using Augmented Reality (AR) technology

Country Status (1)

Country Link
CN (1) CN111679740A (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112181848A (en) * 2020-10-21 2021-01-05 滁州市精美家电设备股份有限公司 Industrial Internet 5G remote guarantee and diagnosis full-automatic complete equipment operation system
CN112330818A (en) * 2020-11-03 2021-02-05 北京卫星环境工程研究所 Unmanned aerial vehicle part manual layering auxiliary system and method based on augmented reality
CN112434575A (en) * 2020-11-12 2021-03-02 广东电网有限责任公司 Electric power Internet of things equipment remote control method and system based on attitude recognition
CN112562092A (en) * 2020-12-22 2021-03-26 国网上海市电力公司 Electrical test intelligent interaction system based on AR technology
CN112616047A (en) * 2020-11-13 2021-04-06 贵州电网有限责任公司 Machine room operation field guidance and remote assistance scheduling system based on AR
CN112710426A (en) * 2020-12-17 2021-04-27 大唐东北电力试验研究院有限公司 AR power station steam turbine dynamic balance method and device based on virtual reality
CN112865317A (en) * 2021-01-28 2021-05-28 深圳供电局有限公司 Remote virtual control method, device and system for intelligent power station
CN112947881A (en) * 2021-04-02 2021-06-11 浙江德维迪亚数字科技有限公司 Remote screen freezing guidance method based on instant messaging
CN113011723A (en) * 2021-03-04 2021-06-22 北京计算机技术及应用研究所 Remote equipment maintenance system based on augmented reality
CN113114971A (en) * 2021-06-16 2021-07-13 国能信控互联技术有限公司 Head display equipment assisting remote cooperation, remote cooperation system and control method thereof
CN113222337A (en) * 2021-04-06 2021-08-06 盈机技术服务(北京)有限公司 Remote vulnerability assessment method and device and storage medium
CN113783305A (en) * 2021-09-27 2021-12-10 国能陕西水电有限公司 AR-based power station integrated management method, system and server
CN115171447A (en) * 2022-07-11 2022-10-11 广东电网有限责任公司东莞供电局 Operation tutoring system based on augmented reality technology
CN115357873A (en) * 2022-10-20 2022-11-18 苏州元硕自动化科技有限公司 AR equipment point inspection auxiliary system based on Vuforia technology

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108879440A (en) * 2018-06-20 2018-11-23 国网山东省电力公司济宁供电公司 Intelligent examination and repair system and method based on wearable augmented reality display terminal and cloud platform
CN110233841A (en) * 2019-06-11 2019-09-13 上海文景信息科技有限公司 A kind of long-distance education data interaction system and method based on AR holographic glasses
CN110442486A (en) * 2019-08-05 2019-11-12 北京远舢智能科技有限公司 A kind of remote device diagnostics system and method based on mixed reality technology

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108879440A (en) * 2018-06-20 2018-11-23 国网山东省电力公司济宁供电公司 Intelligent examination and repair system and method based on wearable augmented reality display terminal and cloud platform
CN110233841A (en) * 2019-06-11 2019-09-13 上海文景信息科技有限公司 A kind of long-distance education data interaction system and method based on AR holographic glasses
CN110442486A (en) * 2019-08-05 2019-11-12 北京远舢智能科技有限公司 A kind of remote device diagnostics system and method based on mixed reality technology

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112181848A (en) * 2020-10-21 2021-01-05 滁州市精美家电设备股份有限公司 Industrial Internet 5G remote guarantee and diagnosis full-automatic complete equipment operation system
CN112330818B (en) * 2020-11-03 2021-06-22 北京卫星环境工程研究所 Unmanned aerial vehicle part manual layering auxiliary system and method based on augmented reality
CN112330818A (en) * 2020-11-03 2021-02-05 北京卫星环境工程研究所 Unmanned aerial vehicle part manual layering auxiliary system and method based on augmented reality
CN112434575A (en) * 2020-11-12 2021-03-02 广东电网有限责任公司 Electric power Internet of things equipment remote control method and system based on attitude recognition
CN112434575B (en) * 2020-11-12 2023-09-01 广东电网有限责任公司 Gesture recognition-based remote control method and system for electric power Internet of things equipment
CN112616047A (en) * 2020-11-13 2021-04-06 贵州电网有限责任公司 Machine room operation field guidance and remote assistance scheduling system based on AR
CN112710426A (en) * 2020-12-17 2021-04-27 大唐东北电力试验研究院有限公司 AR power station steam turbine dynamic balance method and device based on virtual reality
CN112562092A (en) * 2020-12-22 2021-03-26 国网上海市电力公司 Electrical test intelligent interaction system based on AR technology
CN112865317A (en) * 2021-01-28 2021-05-28 深圳供电局有限公司 Remote virtual control method, device and system for intelligent power station
CN113011723A (en) * 2021-03-04 2021-06-22 北京计算机技术及应用研究所 Remote equipment maintenance system based on augmented reality
CN113011723B (en) * 2021-03-04 2024-03-01 北京计算机技术及应用研究所 Remote equipment maintenance system based on augmented reality
CN112947881A (en) * 2021-04-02 2021-06-11 浙江德维迪亚数字科技有限公司 Remote screen freezing guidance method based on instant messaging
CN113222337A (en) * 2021-04-06 2021-08-06 盈机技术服务(北京)有限公司 Remote vulnerability assessment method and device and storage medium
CN113222337B (en) * 2021-04-06 2022-07-29 盈机技术服务(北京)有限公司 Remote vulnerability assessment method and device and storage medium
CN113114971A (en) * 2021-06-16 2021-07-13 国能信控互联技术有限公司 Head display equipment assisting remote cooperation, remote cooperation system and control method thereof
CN113783305A (en) * 2021-09-27 2021-12-10 国能陕西水电有限公司 AR-based power station integrated management method, system and server
CN113783305B (en) * 2021-09-27 2023-10-20 国能陕西水电有限公司 AR-based power station integrated management method, system and server
CN115171447A (en) * 2022-07-11 2022-10-11 广东电网有限责任公司东莞供电局 Operation tutoring system based on augmented reality technology
CN115357873A (en) * 2022-10-20 2022-11-18 苏州元硕自动化科技有限公司 AR equipment point inspection auxiliary system based on Vuforia technology

Similar Documents

Publication Publication Date Title
CN111679740A (en) Method for carrying out remote intelligent diagnosis on power station equipment by using Augmented Reality (AR) technology
US11532127B2 (en) Virtual object driving method, apparatus, electronic device, and readable storage medium
WO2021179399A1 (en) Mixed reality-based remote operation guidance system and method
CN111260084B (en) Remote system and method based on augmented reality cooperative assembly maintenance
CN108509026B (en) Remote maintenance support system and method based on enhanced interaction mode
CN104656893B (en) The long-distance interactive control system and method in a kind of information physical space
CN111600297B (en) Online analysis method and device for operation data of power grid equipment under cloud-edge cooperation
Li et al. A framework and method for human-robot cooperative safe control based on digital twin
CN112306233A (en) Inspection method, inspection system and inspection management platform
CN108828996A (en) A kind of the mechanical arm remote control system and method for view-based access control model information
CN113752264A (en) Mechanical arm intelligent equipment control method and system based on digital twins
CN109395375A (en) A kind of 3d gaming method of interface interacted based on augmented reality and movement
CN115469745A (en) Machine room equipment fault processing method and system based on AR technology
CN110142769B (en) ROS platform online mechanical arm demonstration system based on human body posture recognition
CN107538492A (en) Intelligent control system, method and the intelligence learning method of mobile robot
CN110751734A (en) Mixed reality assistant system suitable for job site
CN114169546A (en) MR remote cooperative assembly system and method based on deep learning
Gao et al. Real-time visual representations for mixed reality remote collaboration
CN112069873A (en) LeapMotion gesture recognition-based screen control system and method
Zenati et al. A new AR interaction for collaborative E-maintenance system
CN113965550B (en) Intelligent interactive remote auxiliary video system
Narducci et al. Enabling consistent hand-based interaction in mixed reality by occlusions handling
CN111660294B (en) Augmented reality control system of hydraulic heavy-duty mechanical arm
CN207432213U (en) The Intelligent control system of mobile robot
Simón et al. The development of an advanced maintenance training programme utilizing augmented reality

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200918