CN114360087A - Mining AR system of patrolling and examining - Google Patents
Mining AR system of patrolling and examining Download PDFInfo
- Publication number
- CN114360087A CN114360087A CN202111615633.XA CN202111615633A CN114360087A CN 114360087 A CN114360087 A CN 114360087A CN 202111615633 A CN202111615633 A CN 202111615633A CN 114360087 A CN114360087 A CN 114360087A
- Authority
- CN
- China
- Prior art keywords
- information
- terminal
- inspection
- diagnosis
- communication
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007689 inspection Methods 0.000 claims abstract description 132
- 238000003745 diagnosis Methods 0.000 claims abstract description 70
- 230000006854 communication Effects 0.000 claims abstract description 67
- 238000004891 communication Methods 0.000 claims abstract description 66
- 238000005065 mining Methods 0.000 claims abstract description 28
- 230000000007 visual effect Effects 0.000 claims description 25
- 238000013473 artificial intelligence Methods 0.000 claims description 10
- 239000012086 standard solution Substances 0.000 claims description 9
- 239000003245 coal Substances 0.000 abstract description 23
- 238000012423 maintenance Methods 0.000 abstract description 9
- 230000003190 augmentative effect Effects 0.000 abstract description 4
- 238000000034 method Methods 0.000 description 18
- 230000008569 process Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 7
- 238000004519 manufacturing process Methods 0.000 description 6
- 230000007547 defect Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 239000002131 composite material Substances 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- 238000011179 visual inspection Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000009412 basement excavation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000011248 coating agent Substances 0.000 description 1
- 238000000576 coating method Methods 0.000 description 1
- 230000005662 electromechanics Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 238000009423 ventilation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C1/00—Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people
- G07C1/10—Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people together with the recording, indicating or registering of other data, e.g. of signs of identity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B1/00—Details of transmission systems, not covered by a single one of groups H04B3/00 - H04B13/00; Details of transmission systems not characterised by the medium used for transmission
- H04B1/38—Transceivers, i.e. devices in which transmitter and receiver form a structural unit and in which at least one part is used for functions of transmitting and receiving
- H04B1/3827—Portable transceivers
- H04B1/385—Transceivers carried on the body, e.g. in helmets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a mining AR (augmented reality) inspection system which comprises an AR terminal and a central control platform, wherein the AR terminal is in communication connection with the central control platform through a communication network; the central control platform comprises: the operation auxiliary information system is used for responding to the received related information of the AR terminal and providing operation auxiliary information corresponding to the related information for the AR terminal; the recording system is used for recording the routing inspection recording information and the operation auxiliary information uploaded by the AR terminal; the diagnosis system is used for acquiring the routing inspection record information of the record system and the diagnosis result information corresponding to the operation auxiliary information; and the report generating system is used for generating the patrol report according to the patrol record information, the operation auxiliary information and the diagnosis result information provided by the diagnosis system. The invention is used for automatically classifying and summarizing various inspection records to form a report, improves the accuracy of the report, and can assist mine field maintenance personnel to quickly diagnose coal mine problems and mining equipment faults during routing inspection.
Description
Technical Field
The invention relates to the technical field of mining inspection equipment, in particular to a mining AR inspection system.
Background
With the increasing requirements of intelligent coal mining, how to ensure the safety of coal mining production becomes more and more important in coal mining management work. At present, in the coal mining process, the conditions of excavation, ventilation, electromechanics and the like need to be inspected, and once the coal mine safety has problems, the mining progress and the mining benefit are greatly influenced, and even the personnel safety is endangered. Therefore, a large amount of maintenance personnel are required to be equipped to carry out timely and effective fault diagnosis and maintenance on the fault equipment in the coal mine production field.
The traditional inspection method is that an inspection route is determined according to the basic situation of a coal mine, each inspector records problems found on the inspection route by using characters, pictures, audio records, video records and the like, and after the inspection is finished, the inspection records are manually gathered to form an inspection report. And because the materials carried by the polling personnel are limited, the polling personnel can not timely acquire the required auxiliary polling information resources in the polling process, the coal mine problem and the mining equipment fault can not be quickly diagnosed, and the efficiency is low.
Disclosure of Invention
The invention provides an AR (augmented reality) inspection system for a mine, which is used for acquiring inspection information in various forms, automatically classifying and summarizing the inspection information to form a report, acquiring auxiliary information required by fault diagnosis in real time and improving the efficiency and quality of equipment and coal mine problem diagnosis.
The technical scheme of the invention is as follows:
a mining AR patrol system comprises an AR terminal and a central control platform, wherein the AR terminal is in communication connection with the central control platform through a communication network; the central control platform comprises:
the operation auxiliary information system is used for responding to the received related information of the AR terminal and providing operation auxiliary information corresponding to the related information for the AR terminal;
the recording system is used for recording the routing inspection recording information and the operation auxiliary information uploaded by the AR terminal;
the diagnosis system is used for acquiring the routing inspection record information of the record system and the diagnosis result information corresponding to the operation auxiliary information;
and the report generating system is used for generating the patrol report according to the patrol record information, the operation auxiliary information and the diagnosis result information provided by the diagnosis system.
Further, the central control platform further comprises a knowledge base associated with the operation auxiliary information system, and the knowledge base is used for providing at least one of corresponding equipment information, facility information and standard solutions for the AR terminal according to the retrieval content submitted by the AR terminal.
Further, the central control platform further comprises an expert system associated with the operation auxiliary information system, and the expert system is used for providing operation guide information for the AR terminal; the expert system comprises an expert guidance platform and an AI diagnosis system, wherein the expert guidance platform is used for connecting an expert appointed by the AR terminal and carrying out remote communication; and the AI diagnosis system is used for acquiring intelligent diagnosis information through an artificial intelligence algorithm according to the problem information provided by the AR terminal and feeding back the intelligent diagnosis information to the AR terminal.
Further, the central control platform further comprises a collaboration system for collaborative operation, the collaboration system is used for providing a communication platform for communication among the plurality of AR terminals, and the collaboration system is associated with the operation auxiliary information system; and the cooperation system stores the communication information in the communication platform to a database associated with the recording system.
Furthermore, the diagnosis system constructs all the routing inspection record information into an inspection record tree composed of root nodes and child nodes for display according to the problem types and routing inspection places in the routing inspection record information.
Further, the recording system is associated with a plurality of databases, and is respectively used for storing the inspection record information, the operation auxiliary information and the operation guide information.
Further, the AR terminal comprises a head-mounted main body, wherein the head-mounted main body is provided with a semitransparent visual display, a communication module, a main control circuit, a camera device, a sound collection device, a positioning module, an earphone and a power supply; the semi-transparent visual display, the communication module, the camera device, the sound collection device, the positioning module and the earphone are all connected with the main control circuit, and the power supply supplies power to the semi-transparent visual display, the communication module, the camera device, the sound collection device, the positioning module and the earphone through the main control circuit.
Furthermore, a point cloud scanning device connected with the main control circuit is also arranged on the head-wearing main body.
Furthermore, the head-wearing main body is also provided with an environmental information acquisition device and an alarm device which are connected with the main control circuit.
Furthermore, the portable electronic device further comprises a handheld indicating device, wherein the handheld indicating device is in communication connection with the communication module and provides indication for the semitransparent visual display.
The technical scheme provided by the embodiment of the invention at least has the following beneficial effects:
(1) the AR polling system can facilitate field personnel to input polling information, obtain auxiliary information required by fault diagnosis in real time, and improve the efficiency and quality of coal mine problem diagnosis of equipment. Specifically, visual inspection is carried out by combining an augmented reality technology, various inspection information is conveniently collected, inspection record information is conveniently recorded, and an inspection report is formed and called. The field personnel can obtain corresponding virtual visual operation auxiliary information based on the real scene original information and preset auxiliary information required by online search, and the inspection personnel are guided by the operation auxiliary information, so that inspection accuracy and efficiency are improved. By combining the live-action information and the virtual information, the equipment fault diagnosis efficiency, the inspection system expandability and the mining scene applicability can be improved.
(2) The auxiliary inspection resources including the knowledge base, the expert system, the cooperation system and the like are arranged on the central control platform, inspection guidance is provided for inspection personnel more comprehensively, particularly, the expert system makes up the defect that preset auxiliary information cannot comprehensively cope with different fault problems, the guidance of a remote expert is obtained in time, the fault diagnosis time and the error rate of field inspection personnel are reduced, and the inspection effect is improved.
(3) Through the point cloud acquisition device, a non-planar object identification tracking technology is adopted, dependence of traditional plane-based identification tracking on object plane characteristic information is eliminated, cross-view angle identification tracking is achieved, and the method is more suitable for complex environments of operation and maintenance of mining equipment.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention and are not to be construed as limiting the invention.
Fig. 1 is a block diagram of a mining AR inspection system according to an embodiment of the present invention.
FIG. 2 is a schematic diagram of an editable interface of a diagnostic system according to an embodiment of the invention.
FIG. 3 is a schematic diagram of another editable interface of the diagnostic system according to the embodiment of the invention.
FIG. 4 is a schematic diagram of yet another editable interface of a diagnostic system according to an embodiment of the invention.
FIG. 5 is a schematic diagram of yet another editable interface of a diagnostic system according to an embodiment of the invention.
FIG. 6 is a schematic diagram of yet another editable interface of a diagnostic system according to an embodiment of the invention.
FIG. 7 is a schematic diagram of yet another editable interface of a diagnostic system according to an embodiment of the invention.
FIG. 8 is a schematic diagram of yet another editable interface of a diagnostic system according to an embodiment of the invention.
Fig. 9 is a block diagram of a mining AR inspection system in accordance with another embodiment of the present invention.
Fig. 10 is a schematic diagram of an AR terminal according to an embodiment of the present invention.
Fig. 11 is a structural diagram of an AR terminal according to an embodiment of the present invention.
In the figure, the position of the upper end of the main shaft,
a head-mounted body 100; a visual display 110; an upward flipping mechanism 120; an illumination device 130; a sound collection device 140; a restraint device 150.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first", "second", etc. in the present invention are used for distinguishing similar objects, and are not necessarily used for describing a particular order or sequence. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.
As shown in fig. 1, the embodiment provides a mining AR inspection system, which is used for inspection of a mining area, and includes an AR terminal and a central control platform, where the AR terminal is in communication connection with the central control platform through a communication network; the central control platform comprises:
the operation auxiliary information system is used for responding to the received related information of the AR terminal and providing operation auxiliary information corresponding to the related information for the AR terminal;
the recording system is used for recording the routing inspection recording information and the operation auxiliary information uploaded by the AR terminal;
the diagnosis system is used for acquiring the routing inspection record information of the record system and the diagnosis result information corresponding to the operation auxiliary information;
and the report generating system is used for generating the patrol report according to the patrol record information, the operation auxiliary information and the diagnosis result information provided by the diagnosis system.
The recording system can collect the patrol inspection record information to form patrol inspection records, and the patrol inspection records are stored in the database, so that omission or repetition of manual records is avoided.
The operation auxiliary information system is connected with the AR terminal through a communication network and displays the operation auxiliary information through the AR terminal; the inspection record uploaded by the AR terminal at least comprises one or more of original information of an image, video or audio and auxiliary operation information aiming at the inspection record; the recording system is provided with an independent database for respectively recording the original information and the operation auxiliary information.
And the display interface of the diagnosis system simultaneously displays the inspection record called in the diagnosis process and an editable interface in the report production system. The diagnosis system calls the inspection record, generates an inspection report from the inspection record according to the meta information such as the inspection route, the problem type and the like, and improves the relevance between the inspection report and the inspection record. The diagnosis system displays the patrol record, can visually obtain information such as coal mine basic conditions, equipment operation and maintenance operation contents and the like in real time during diagnosis, and simultaneously displays an editable interface, thereby being more convenient to input diagnosis result information. The diagnosis system comprises original information and operation auxiliary information in the calling and recording system, and can display a composite image of routing inspection information and operation auxiliary information, or display a single image of routing inspection information or operation auxiliary information.
The central control center control platform also comprises a display system which is used for displaying the polling records and the details thereof called in the diagnosis process of the diagnosis system and reporting an editable interface in the production system.
The diagnostic system further includes an information overlay system for displaying a composite image of the original information overlay work assistance information via the display system, and a selection system for displaying a single image of the original information or the work assistance information via the display system.
The AR terminal can record various original information, makes the record information variety pluralism of patrolling and examining, and the record of patrolling and examining that forms is more comprehensive, diagnostic system's information stack display system is through overlapping the original information that will gather and the operation auxiliary information based on mining equipment operation and maintenance scene needs, can directly obtain corresponding visual operation auxiliary information based on original information to improve coal mine problem diagnosis efficiency, diagnostic system's selection system can make display system single display original information or operation auxiliary information, and the field personnel can switch according to actual application needs, has improved system scalability and mining scene suitability of patrolling and examining.
The using process of the recording system in the mining AR patrol inspection system is as follows: the field personnel patrol along the arranged inspection route, the route comprises a plurality of inspection places, namely key nodes, if the route 1 is 'A-B-C', A, B, C is the key node, the route 2 is 'D-B-E', D, B, E is the key node, and the cross node of the route 1 and the route 2 is B; the method comprises the steps that field personnel record problems found in inspection through an AR terminal, each problem forms inspection record information, the inspection record information at least comprises original information with contents such as inspectors, inspection places, problem types, problem descriptions and relevant evidences, and the inspection record information also comprises operation auxiliary information with contents such as processing bases and processing opinions, and the inspection record information comprises characters, images, videos and sound records. After the inspection is finished, the field personnel arrange the inspection record information in the AR terminal and submit the inspection record information to the recording system of the central control platform. Through mining AR system of patrolling and examining, can make things convenient for the field personnel to type and read equipment and patrol and examine information, obtain the required auxiliary information of failure diagnosis in real time, improve equipment colliery problem diagnosis efficiency and quality.
Furthermore, the diagnosis system constructs all the routing inspection record information into an inspection record tree composed of root nodes and child nodes for display according to the problem types and routing inspection places in the routing inspection record information.
The use process of the diagnosis system in the mining AR patrol system is as follows: the original information and the auxiliary operation information in the recording system are called by the diagnosis system, and the routing inspection record information submitted by all field personnel is automatically constructed into a 'inspection record tree' according to the problem type and the inspection place in the original information, wherein the 'inspection record tree' comprises three layers, the first layer of nodes are of the problem type, listing the problem types in all the routing inspection record information in the layer, checking the lower nodes with the problem types at the positions, all places where the problem occurs are listed under each problem type, lower nodes recorded as the inspection places are inspected, according to different problem types and inspection places, the inspection record information is listed in different branches, and if a plurality of field personnel submit the inspection record information aiming at the same inspection place and the same problem type, the inspection record information is listed side by side so as to be convenient for comparison and check in editing.
Referring to fig. 2 to 8, the background personnel check the polling record information in the diagnostic system calling record system in the editable interface, edit the production polling report according to the polling record information, and finally generate the system production polling report through the report. The editable interface is divided into three columns, the left column (the left area) is the inspection record tree, the middle column (the middle area) is the inspection record information, and the right column (the right side) is the inspection report editor. The diagnosis system calls the original information and the auxiliary operation information in the recording system, automatically generates an inspection report frame, fills the information of the basic situation, the route and the like of the coal mine into the inspection report, initializes the problems, and automatically generates sections according to the first-layer nodes of the inspection record tree, wherein the title of each section is one of the problem types.
When the diagnosis system is used for editing, whether the routing inspection record information is added into an inspection report or not can be judged according to the check condition of the check box in the left area before the inspection record; after one or more pieces of inspection record information are selected, the details of the selected inspection record information are displayed in a middle area, the details comprise original information and operation auxiliary information submitted by field personnel, when the selected inspection record information is not added to an inspection report, a button of 'adding to the inspection report' appears at the bottom of the middle area, and a system executes the following actions after clicking: adding the problem description and the processing opinion in the selected routing inspection record information into a corresponding problem type section of the right area, creating a problem and automatically numbering; adding links corresponding to the names of field personnel submitting the routing inspection record information below the problem, positioning the routing inspection record information in a left area by clicking the links, and displaying the details of the routing inspection record information in a middle area; checking a check box in the left area before the selected routing inspection record information to indicate that the routing inspection record information is added; adding a link after each inspection record information detail in the middle area to associate the section and the problem label of the inspection report corresponding to the inspection record information, skipping to the corresponding part of the inspection report by clicking the link, adding a 'disassociation' button to the link, and disconnecting the association between the inspection record information and the inspection report by clicking; and hiding an 'add to inspection report' button in the middle area, finishing the editing of the inspection report of the piece of inspection record information, and finally generating the inspection report through a report generation system.
In another embodiment of the present invention, as shown in fig. 9, the central control platform further includes a knowledge base associated with the job auxiliary information system, and the knowledge base is configured to provide at least one of corresponding device information, facility information and standard solution for the AR terminal according to the search content submitted by the AR terminal.
The knowledge base is connected with the AR terminal through the operation auxiliary information system and displays the equipment information, the facility information and the standard solution through the AR terminal. And the knowledge base can be provided with operation and maintenance auxiliary information such as a 3D model of the mining equipment, manufacturer information, an operation flow, maintenance key points and the like. When field personnel overhaul equipment, the AR terminal can display equipment information, facility information and a standard solution of the knowledge base in real time, so that the knowledge base is visualized, the field personnel can obtain auxiliary information required for judging coal mine problems and equipment fault diagnosis, the time wasted by searching correct information is reduced, data can be prevented from being switched back and forth during operation, and the diagnosis work efficiency is improved.
The operation auxiliary information can be information searched by an operator according to needs, and can also be operation auxiliary information actively pushed by an AR terminal according to positioning information and shot scenes, for example, in the process of polling personnel, navigation instructions are carried out on a polling route, and the observed equipment is marked on glasses, so that the AR function is realized.
As shown in fig. 9, the central control platform further includes an expert system associated with the work assistance information system, where the expert system is configured to provide work guidance information for the AR terminal; the expert system comprises an expert guidance platform and an AI diagnosis system, wherein the expert guidance platform is used for connecting an expert appointed by the AR terminal and carrying out remote communication; and the AI diagnosis system is used for acquiring intelligent diagnosis information through an artificial intelligence algorithm according to the problem information provided by the AR terminal and feeding back the intelligent diagnosis information to the AR terminal.
The expert system is connected with the AR terminal through the operation auxiliary information system and displays operation guide information through the AR terminal; the routing inspection record uploaded by the AR terminal also comprises operation guidance information; the recording system sets an independent database to record the operation guidance information.
The expert guidance platform and the AI diagnosis system can provide operation guidance information through original inspection information to assist field personnel in judging coal mine problems and repairing equipment, and the defect that preset equipment information, facility information and a standard solution cannot comprehensively cope with different fault problems is overcome. Meanwhile, the recording system also records the operation guidance information so as to record the whole process of the collaborative diagnosis, and an administrator can check and retrieve the operation guidance information, trace back the collaborative diagnosis process, and perform analysis and management.
In subsequent routing inspection, the operation guide information can be used as operation auxiliary information to assist field personnel in diagnosis, so that the knowledge learning of the AI diagnosis system on coal mine problems and fault diagnosis is facilitated, more problems can be diagnosed, and the task time and the error rate are reduced.
The knowledge base and the operation guidance information provided by the expert system can be used as part of the operation auxiliary information, the knowledge base refers to standardized information which is input into the system, such as a three-dimensional model of equipment, technical parameters and the like, and the expert guidance refers to various instructions given on a manual site or Artificial Intelligence (AI).
The information superposition system is also used for superposing and displaying the operation guide information, and the selection system is also used for switching and displaying the operation guide information.
As shown in fig. 9, the central control platform further includes a collaboration system for collaborative operations, the collaboration system is configured to provide a communication platform for communication among the plurality of AR terminals, and the collaboration system is associated with the operation auxiliary information system; and the cooperation system stores the communication information in the communication platform to a database associated with the recording system.
The cooperative operation refers to a platform for direct mutual communication of mine workers, and communication among front-line workers at different positions and different posts is needed under many conditions in a mine.
Part of AR terminals, a knowledge base and an expert system are accessed into a cooperation system to operate together, the cooperation system is connected with the AR terminals through an operation auxiliary information system and displays operation auxiliary information, equipment information, facility information and a standard solution scheme, and operation guide information through the AR terminals; the collaboration system uploads job auxiliary information, equipment information, facility information, standard solutions, and job guidance information to the recording system.
The cooperation system adopts a C/S structure, has distributed use capability, and by means of SLAM technology, remote experts perform cooperation guidance through an expert system, can receive a field real-time picture in real time and directly perform space marking in a real-time communication picture at a 'first visual angle' to form operation guidance information, wherein the operation guidance information is not limited to characters, pictures, coating information and models, the operation auxiliary information, equipment/facility information, a standard solution scheme and the operation guidance information are overlapped with a field physical environment and a real object through the cooperation system, and field personnel can directly see the corresponding information and guidance results of the equipment/facility through an AR terminal so as to improve the accuracy and efficiency of equipment fault diagnosis.
Further, the recording system is associated with a plurality of databases, which are respectively used for storing the inspection record information, the operation auxiliary information and the operation guide information. The plurality of databases correspondingly keep different information, the data volume is small, and the searching efficiency is high.
It should be noted that the recording system, the diagnosis system, the work assistance information system, the expert system, the collaboration system, and the like are independent systems, but are also related, and a port accessed through the main control platform can be developed in a modular manner, and the systems can be operated independently or in a systematic manner.
As shown in fig. 6, the communication network further includes distributed communication terminals, the distributed communication terminals are disposed at different positions of the mining area, the multiple AR terminals are in direct communication with the communication network or are indirectly connected with the communication network through the distributed communication terminals, the multiple AR terminals can communicate with each other, and any one AR terminal selects to communicate with the communication network directly or indirectly connected with the communication network through the distributed communication terminals or indirectly connected with the distributed communication terminals or the communication network through other AR terminals according to the network communication quality. Because the coal mine environment network is generally poor, in order to ensure the definition of audio and video transmitted between the AR terminal and the central control platform and the synchronism of labels, the distributed communication terminals are arranged, the AR terminal has various modes of being connected with the communication network, so that the AR terminal can be accessed nearby according to the current network environment, the fluency and synchronism of remote communication and labels of the AR terminal in the weak network environment are improved, in the process of connecting the AR terminal and the communication network to transmit data, the technologies of echo cancellation, jitter buffering and the like can be matched to carry out buffer control on data packet delay in the data communication process, and meanwhile, the communication quality is ensured.
As shown in fig. 10 and 11, the AR terminal includes a head-mounted main body 100, and the head-mounted main body is configured with a semitransparent visual display, a communication module, a main control circuit, a camera device, a sound collection device, a positioning module, an earphone, and a power supply; the semi-transparent visual display, the communication module, the camera device, the sound collection device, the positioning module and the earphone are all connected with the main control circuit, and the power supply supplies power to the semi-transparent visual display, the communication module, the camera device, the sound collection device, the positioning module and the earphone through the main control circuit. The camera device comprises an image acquisition device and a video acquisition device, and the image acquisition device, the video acquisition device and the sound acquisition device can be collectively called acquisition devices according to functions.
The front side of the head-wearing main body 100 is provided with a visual display 110, the head-wearing main body 100 is internally provided with a communication module and a main control circuit, the head-wearing main body 100 is provided with an image/video acquisition device and a sound acquisition device 140, and the two sides of the head-wearing main body 100 are provided with earphones. The AR terminal is connected with a communication network through the communication module or connected with other AR terminals. Wherein the visual display 110 is a translucent visual display.
The use process of the AR terminal is as follows: the image acquisition device/video acquisition device (camera device) and the sound acquisition device 140 are used for acquiring original information such as images, videos and audios and transmitting the original information to the central control platform through a communication network, the central control platform is used for transmitting cooperation information such as operation auxiliary information and operation guide information in various characters, images and videos to the visual display 110 of the AR terminal through the communication network after overlapping the original information, the central control platform can also transmit corresponding cooperation information in a voice form to the earphone of the AR terminal through the communication network for playing, after a field worker wears the AR terminal, the corresponding cooperation information can be obtained through the visual display 110 and the earphone, and the field worker is quickly and efficiently assisted to enter routing inspection record information and diagnose difficult problems of various equipment operations by combining real scenes and virtual cooperation information. The positioning device is used for positioning the position of field personnel, and can judge the state of a coal mine field more quickly and accurately by combining the patrol and inspection recording information in the recording system.
Furthermore, a point cloud scanning device connected with the main control circuit is also arranged on the head-wearing main body.
Furthermore, the head-wearing main body is also provided with an environmental information acquisition device and an alarm device which are connected with the main control circuit.
The point cloud scanning device is a 3D tracking technology based on 3D point cloud characteristic information, can break plane limitation, recognize and track non-planar objects, realizes cross-view recognition and tracking, and is more suitable for complex environments of coal mine sites. The environment information acquisition device can detect the current data of the environment where the site personnel are located, the data can be used as original information to be recorded into the recording system, a patrol report is formed through the diagnosis system, when the diagnosis system judges that danger is about to occur, the alarm device operates to remind the site personnel of evacuating, or when the site personnel are in danger, the alarm device can be started to give an alarm, and the safety of the operation of the site personnel in the coal mine is improved.
The head-mounted main body 100 is further provided with an illuminating device 130, and the power supply supplies power to the illuminating device 130, so that the operation of field personnel is facilitated in an environment with weak illumination, and images and videos collected by the image/video collecting device can be clearer.
Furthermore, the portable electronic device further comprises a handheld indicating device, wherein the handheld indicating device is in communication connection with the communication module and provides indication for the semitransparent visual display.
The handheld indicating device is connected with the communication module and can be a cursor indicating device or an AR indicating and recognizing device, the accuracy of capturing coal mine environment coordinates by the AR terminal can be improved, corresponding cooperation information can be displayed on corresponding equipment more quickly, and the problem that when field personnel move quickly or move and change speed, the AR terminal captures the picture again to cause overlong picture missing time or the picture needs to wait for response to cause blurring is avoided.
The head-mounted main body 100 is of a helmet type structure and is provided with the anti-seismic lining and the binding device 150, so that people can wear the head-mounted main body more stably, the head-mounted main body is not easy to move in the operation process or the moving process, and images and videos collected by the AR terminal are more stable and clear. The visual display 110 is connected to the front side of the head-mounted body 100 through the upward turning mechanism 120, and the position of the visual display 110 can be adjusted by turning the visual display upward according to the operation requirement, so as to facilitate the operation of a person.
The AR polling system can facilitate field personnel to input polling information, obtain auxiliary information required by fault diagnosis in real time, and improve the efficiency and quality of coal mine problem diagnosis of equipment. Specifically, visual inspection is carried out by combining an augmented reality technology, various inspection information is conveniently collected, inspection record information is conveniently recorded, and an inspection report is formed and called. The field personnel can obtain corresponding virtual visual operation auxiliary information based on the real scene original information and preset auxiliary information required by online search, and the inspection personnel are guided by the operation auxiliary information, so that inspection accuracy and efficiency are improved. By combining the live-action information and the virtual information, the equipment fault diagnosis efficiency, the inspection system expandability and the mining scene applicability can be improved. The auxiliary inspection resources including the knowledge base, the expert system, the cooperation system and the like are arranged on the central control platform, inspection guidance is provided for inspection personnel more comprehensively, particularly, the expert system makes up the defect that preset auxiliary information cannot comprehensively cope with different fault problems, the guidance of a remote expert is obtained in time, the fault diagnosis time and the error rate of field inspection personnel are reduced, and the inspection effect is improved. Through the point cloud acquisition device, a non-planar object identification tracking technology is adopted, dependence of traditional plane-based identification tracking on object plane characteristic information is eliminated, cross-view angle identification tracking is achieved, and the method is more suitable for complex environments of operation and maintenance of mining equipment.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (10)
1. The mining AR inspection system is characterized by comprising an AR terminal and a central control platform, wherein the AR terminal is in communication connection with the central control platform through a communication network; the central control platform comprises:
the operation auxiliary information system is used for responding to the received related information of the AR terminal and providing operation auxiliary information corresponding to the related information for the AR terminal;
the recording system is used for recording the routing inspection recording information and the operation auxiliary information uploaded by the AR terminal;
the diagnosis system is used for acquiring the routing inspection record information of the record system and the diagnosis result information corresponding to the operation auxiliary information;
and the report generating system is used for generating the patrol report according to the patrol record information, the operation auxiliary information and the diagnosis result information provided by the diagnosis system.
2. The system of claim 1, wherein the central control platform further comprises a knowledge base associated with the work assistance information system, the knowledge base being configured to provide at least one of corresponding device information, facility information, and standard solutions to the AR terminal according to the retrieved content submitted by the AR terminal.
3. The system of claim 1, wherein the central control platform further comprises an expert system associated with the work assistance information system, the expert system configured to provide work guidance information for the AR terminal; the expert system comprises an expert guidance platform and an AI diagnosis system, wherein the expert guidance platform is used for connecting an expert appointed by the AR terminal and carrying out remote communication; and the AI diagnosis system is used for acquiring intelligent diagnosis information through an artificial intelligence algorithm according to the problem information provided by the AR terminal and feeding back the intelligent diagnosis information to the AR terminal for operation.
4. The system according to claim 1, wherein the central control platform further comprises a collaboration system for collaborative work, the collaboration system is used for providing a communication platform for communication among the plurality of AR terminals, and the collaboration system is associated with the work assistance information system; and the cooperation system stores the communication information in the communication platform to a database associated with the recording system.
5. The system of claim 1, wherein the diagnosis system constructs all the patrol record information into a patrol record tree composed of a root node and child nodes for display according to the type of the problem and the patrol location in the patrol record information.
6. The system of claim 3, wherein the logging system is associated with a plurality of databases for storing the routing inspection log information, the job assistance information, and the job guidance information, respectively.
7. The system of claim 1, wherein the AR terminal comprises a head-mounted main body, and the head-mounted main body is configured with a translucent visual display, a communication module, a main control circuit, a camera device, a sound collection device, a positioning module, an earphone and a power supply; the semi-transparent visual display, the communication module, the camera device, the sound collection device, the positioning module and the earphone are all connected with the main control circuit, and the power supply supplies power to the semi-transparent visual display, the communication module, the camera device, the sound collection device, the positioning module and the earphone through the main control circuit.
8. The system of claim 7, wherein the head-mounted body is further configured with a point cloud scanning device connected to the master control circuit.
9. The system of claim 7, wherein the head-mounted body is further provided with an environment information acquisition device and an alarm device which are connected with the main control circuit.
10. The system of claim 7, further comprising a handheld pointing device communicatively coupled to the communication module to provide an indication to the translucent visual display.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111615633.XA CN114360087A (en) | 2021-12-27 | 2021-12-27 | Mining AR system of patrolling and examining |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111615633.XA CN114360087A (en) | 2021-12-27 | 2021-12-27 | Mining AR system of patrolling and examining |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114360087A true CN114360087A (en) | 2022-04-15 |
Family
ID=81102647
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111615633.XA Pending CN114360087A (en) | 2021-12-27 | 2021-12-27 | Mining AR system of patrolling and examining |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114360087A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116563967A (en) * | 2023-04-13 | 2023-08-08 | 中冶南方（武汉）自动化有限公司 | AR-based intelligent auxiliary inspection method and system for underground comprehensive pipe rack |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845502A (en) * | 2017-01-23 | 2017-06-13 | 东南大学 | It is a kind of to visualize guidance method for the Wearable servicing unit of overhaul of the equipments and overhaul of the equipments |
| CN107160388A (en) * | 2017-04-25 | 2017-09-15 | 国网浙江省电力公司 | Robot used for intelligent substation patrol monitoring system |
| CN206672041U (en) * | 2017-03-14 | 2017-11-24 | 北京大合创新科技发展有限公司 | A kind of mining equipment intelligent patrol detection based on augmented reality and safeguard accessory system |
| CN107885709A (en) * | 2017-09-29 | 2018-04-06 | 深圳市共济科技股份有限公司 | A kind of report form generation method, system and storage medium |
| CN110865917A (en) * | 2018-08-27 | 2020-03-06 | 国网天津市电力公司 | AR technology-based electric power machine room inspection operation method, system and application |
| CN112018892A (en) * | 2020-09-04 | 2020-12-01 | 南京太司德智能电气有限公司 | Electric power operation and maintenance remote guidance system |
-
2021
- 2021-12-27 CN CN202111615633.XA patent/CN114360087A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845502A (en) * | 2017-01-23 | 2017-06-13 | 东南大学 | It is a kind of to visualize guidance method for the Wearable servicing unit of overhaul of the equipments and overhaul of the equipments |
| CN206672041U (en) * | 2017-03-14 | 2017-11-24 | 北京大合创新科技发展有限公司 | A kind of mining equipment intelligent patrol detection based on augmented reality and safeguard accessory system |
| CN107160388A (en) * | 2017-04-25 | 2017-09-15 | 国网浙江省电力公司 | Robot used for intelligent substation patrol monitoring system |
| CN107885709A (en) * | 2017-09-29 | 2018-04-06 | 深圳市共济科技股份有限公司 | A kind of report form generation method, system and storage medium |
| CN110865917A (en) * | 2018-08-27 | 2020-03-06 | 国网天津市电力公司 | AR technology-based electric power machine room inspection operation method, system and application |
| CN112018892A (en) * | 2020-09-04 | 2020-12-01 | 南京太司德智能电气有限公司 | Electric power operation and maintenance remote guidance system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116563967A (en) * | 2023-04-13 | 2023-08-08 | 中冶南方（武汉）自动化有限公司 | AR-based intelligent auxiliary inspection method and system for underground comprehensive pipe rack |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12079439B2 (en) | Shared augmented reality session recording and playback | |
| US10878634B2 (en) | Methods for augmented reality applications | |
| CN106340217B (en) | Manufacturing equipment intelligence system and its implementation based on augmented reality | |
| CN104157021B (en) | Intelligent inspection system and intelligent polling method | |
| CN111722714A (en) | Digital substation metering operation inspection auxiliary method based on AR technology | |
| CN110865917A (en) | AR technology-based electric power machine room inspection operation method, system and application | |
| CN109638959B (en) | Power equipment remote signaling function debugging method and system based on AR and deep learning | |
| CN112578907A (en) | Method and device for realizing remote guidance operation based on AR | |
| CN103034755A (en) | Visual inspection method based on virtual reality technology for transformer substation | |
| CN104850945A (en) | Geological record acquisition system, and management and control method based on network and android system | |
| CN114218788A (en) | Transformer substation digital twinning system and application method and system thereof | |
| CN112150596B (en) | Holographic reconstruction method and system for substation equipment fault | |
| CN114662714A (en) | Machine room operation and maintenance management system and method based on AR equipment | |
| CN112585661A (en) | Method and system for generating virtual reality training sessions | |
| CN109034404B (en) | AR (augmented reality) -glasses-based multi-user cooperative maintenance system and method for thermal power plant and AR glasses | |
| CN112306233A (en) | Inspection method, inspection system and inspection management platform | |
| CN109878760A (en) | A kind of general aviation aircraft fault remote assists platform and restorative procedure | |
| CN112929384A (en) | AR intelligent point inspection system based on space anchor point | |
| CN114023138A (en) | Emergency drilling simulation system | |
| CN111710032B (en) | Method, device, equipment and medium for constructing three-dimensional model of transformer substation | |
| CN114360087A (en) | Mining AR system of patrolling and examining | |
| CN111757166A (en) | VR and AR remote detection technology guidance method and system based on 5G transmission | |
| CN113506383A (en) | Airplane strength test inspection method and system | |
| CN105279591B (en) | Man-machine interaction system supporting single-person flow operation instruction and verification | |
| KR20230056006A (en) | Apparatus for artificial intelligence based safety diagnosis through 3d model and method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |