CN114510027A - Underground coal mine equipment inspection method and device based on AR technology - Google Patents
Underground coal mine equipment inspection method and device based on AR technology Download PDFInfo
- Publication number
- CN114510027A CN114510027A CN202111582173.5A CN202111582173A CN114510027A CN 114510027 A CN114510027 A CN 114510027A CN 202111582173 A CN202111582173 A CN 202111582173A CN 114510027 A CN114510027 A CN 114510027A
- Authority
- CN
- China
- Prior art keywords
- underground
- equipment
- personnel
- inspection
- control center
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 130
- 238000000034 method Methods 0.000 title claims abstract description 71
- 239000003245 coal Substances 0.000 title claims abstract description 47
- 238000005516 engineering process Methods 0.000 title claims abstract description 27
- 238000010586 diagram Methods 0.000 claims abstract description 24
- 239000011521 glass Substances 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 17
- 230000002159 abnormal effect Effects 0.000 claims description 11
- 230000004044 response Effects 0.000 claims description 6
- 230000003993 interaction Effects 0.000 claims description 5
- 210000001145 finger joint Anatomy 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 4
- 238000005065 mining Methods 0.000 description 9
- 238000004519 manufacturing process Methods 0.000 description 5
- 239000000463 material Substances 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000005520 cutting process Methods 0.000 description 3
- 239000010687 lubricating oil Substances 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to an AR technology-based underground coal mine equipment inspection method, which comprises the steps that a control center obtains position and real-time state information of underground equipment to be inspected and underground personnel, and plans an inspection task and an inspection route; the underground terminal displays the inspection task and the inspection route to underground personnel; the method comprises the steps that an underground terminal identifies underground equipment to be inspected, generates an underground equipment problem interface diagram and displays the underground equipment problem interface diagram to underground personnel, wherein the underground terminal is integrated on a safety helmet; if the underground personnel can solve the problem of the underground equipment by themselves, the underground terminal shoots and uploads a video for solving the problem by the control center; if the underground personnel can not solve the underground equipment problem by themselves, the underground terminal calls the solving method of the underground equipment with the similar problem from the control center and displays the solving method to the underground personnel; if the control center does not have a method for solving the similar problems of the underground equipment, remote video connection is established, and the problems of the underground equipment are solved through remote guidance. The invention also provides an equipment inspection device.
Description
Technical Field
The invention relates to the technical field of underground coal mines, in particular to an underground coal mine equipment inspection method and device based on an AR technology.
Background
Coal mine workers enter a mine to perform inspection and maintenance, which is an important link in daily production and operation of the coal mine and can ensure stable and efficient production and safe operation of the coal mine. With the increasing level of coal mine intelligence, various underground equipment devices become increasingly complex, and for example, a coal mining face needs to operate cooperatively with equipment such as a coal mining machine, a scraper conveyor, a hydraulic support and the like.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art: because different equipment is developed and matched by different units, each set of equipment is very complex, once equipment failure or other difficult problems occur, coal miners are difficult to solve the problems at the first time, and often need to contact technical experts of related equipment development units to solve the problems. This mode seriously affects the production efficiency of the coal mine.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, the invention aims to provide a method for inspecting underground coal mine equipment, which can improve the underground inspection efficiency.
The invention aims to provide an underground coal mine equipment inspection device for improving underground inspection efficiency.
In order to achieve the above object, in a first aspect, the present invention provides a method for inspecting underground coal mine equipment based on AR technology, including:
the control center acquires the position and real-time state information of underground equipment and underground personnel to be patrolled and examined, and plans a patrolling and examining task and a patrolling and examining route;
the underground terminal displays the inspection task and the inspection route to underground personnel;
the method comprises the steps that an underground terminal identifies underground equipment to be inspected, generates an underground equipment problem interface diagram and displays the underground equipment problem interface diagram to underground personnel, wherein the underground terminal is integrated on a safety helmet;
if the underground personnel can solve the problem of the underground equipment by themselves, the underground terminal shoots and uploads a video for solving the problem by the control center;
if the underground personnel can not solve the underground equipment problem by themselves, the underground terminal calls the solving method of the underground equipment with the similar problem from the control center and displays the solving method to the underground personnel;
if the control center does not have a method for solving the similar problems of the underground equipment, remote video connection is established, and the problems of the underground equipment are solved through remote guidance.
Compared with the prior art, the underground coal mine equipment inspection method based on the AR technology has the advantages that the inspection process and the inspection task are digitized, and the inspection route is dynamically planned, so that the inspection threshold is reduced, the inspection efficiency is improved, and the safety of mine production is enhanced; 2. the method can assist the actual recognition result of the underground inspection personnel in comprehensively judging the equipment problems, and improve the recognition efficiency and the recognition accuracy of the equipment problems; 3. according to the method, corresponding experts can be intelligently matched from the expert database according to the equipment model and the equipment fault, the field problems are remotely marked and guided through the experts, and inspection personnel are assisted to overhaul the equipment, so that the threshold of the inspection personnel is reduced, and the efficiency of solving the problems is improved.
According to one embodiment of the invention, the underground coal mine equipment inspection method based on the AR technology further comprises the following steps:
the control center is communicably coupled with the underground equipment and the underground personnel, acquires the position and real-time state information of the underground equipment to be inspected and the underground personnel, and establishes inspection processes for at least one underground equipment, wherein each inspection process comprises a plurality of inspection steps;
in response to the abnormal state of the underground equipment, the control center replans the inspection task and the inspection route and displays the inspection task and the inspection route to the underground personnel, wherein the inspection route is realized by a dynamic programming algorithm;
and the underground terminal projects the re-planned inspection task and the re-planned inspection route to the eyes of the underground personnel in a virtual picture mode.
According to one embodiment of the invention, the underground coal mine equipment inspection method based on the AR technology further comprises the following steps:
the underground terminal identifies the two-dimensional code of the underground equipment to be inspected, acquires the state information of the underground equipment to be inspected, generates an underground equipment problem interface diagram, and projects the underground equipment problem interface diagram in the form of a virtual picture to the front of eyes of underground personnel.
According to one embodiment of the invention, the underground coal mine equipment inspection method based on the AR technology further comprises the following steps:
the underground terminal receives the gestures of the underground personnel, and obtains the depth information of each frame of image of the shot video to obtain a depth map;
capturing characteristic points of finger joints of the underground personnel according to each frame of depth map, drawing finger contours and identifying position information of fingertips;
color marking and connecting the position information among the continuous multi-frame depth maps, and marking the fault position of the underground equipment in a video;
and uploading the video with the labeling information to the control center.
According to one embodiment of the invention, the method for inspecting the underground coal mine equipment based on the AR technology, wherein the response to the abnormal state of the underground coal mine equipment comprises the following steps:
and judging that the state of the underground equipment is abnormal in response to the fact that data collected by a sensor installed on the underground equipment exceeds a preset threshold value.
According to one embodiment of the invention, the method for patrolling underground coal mine equipment based on the AR technology, the establishing of the remote video connection and the solving of the underground equipment problem through remote guidance comprise the following steps:
according to the model and the problem of the underground equipment, the expert in the field is matched from the database of the control center, and the expert in the field is remotely connected through video or voice.
In a second aspect, an underground coal mine equipment inspection device based on AR technology is provided, including:
the control center is used for acquiring the position and real-time state information of underground equipment and underground personnel to be patrolled and examined, and planning a patrolling task and a patrolling and examining route;
the underground terminal is used for displaying the inspection task and the inspection route to underground personnel; identifying the underground equipment to be inspected, generating an underground equipment problem interface diagram, and displaying the underground equipment problem interface diagram to underground personnel, wherein the underground terminal is integrated on a safety helmet; if the underground personnel can solve the problem of the underground equipment by themselves, shooting and uploading a video for solving the problem by the control center; and if the underground personnel can not solve the underground equipment problem by themselves, calling the solving method of the underground equipment with the same problem from the control center and displaying the solving method to the underground personnel.
According to one embodiment of the invention, the downhole terminal is AR glasses, the AR glasses comprising:
the camera is used for acquiring a video of the underground equipment, identifying the underground equipment and uploading the video;
the time-of-flight sensor is used for measuring the distance between an object and the time-of-flight sensor, obtaining the depth information of an image, carrying out three-dimensional recognition on the gestures of underground personnel and providing interaction between a virtual picture and real actions;
a micro projector for projecting video and images into the eyes of downhole personnel, wherein the video and images are superimposed with the real environment.
According to one embodiment of the present invention, further comprising an uphole terminal communicatively coupled with the control center for interacting with an expert in the field.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flow chart of a method for inspecting underground coal mine equipment based on an AR technology according to an embodiment of the present invention.
Fig. 2 is a schematic flow chart of a well coal mine equipment inspection method according to a preferred embodiment of the invention.
Fig. 3 is a schematic diagram of a coal mining machine inspection procedure according to a preferred embodiment of the present invention.
Fig. 4 is a schematic diagram of communication between a control center and a remote expert and a downhole inspection worker according to a preferred embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention. On the contrary, the embodiments of the invention include all changes, modifications and equivalents coming within the spirit and terms of the claims appended hereto.
Fig. 1 is a schematic flow chart of a method for inspecting underground coal mine equipment based on an AR technology according to an embodiment of the present invention.
Referring to fig. 1, a control center of a well coal mine equipment inspection method based on an AR technology comprises the following steps:
and S102, acquiring the position and real-time state information of underground equipment and underground personnel to be inspected, and planning an inspection task and an inspection route.
And step S104, the underground terminal displays the inspection task and the inspection route to underground personnel.
And S106, identifying the underground equipment to be inspected by the underground terminal, generating an underground equipment problem interface diagram, and displaying the underground equipment problem interface diagram to underground personnel, wherein the underground terminal is integrated on a safety helmet.
And S108, if the underground personnel can solve the problem of the underground equipment by themselves, the underground terminal shoots and uploads a video for solving the problem by the control center.
And S110, if the underground personnel can not solve the underground equipment problem by themselves, the underground terminal calls the solving method of the underground equipment with the same problem from the control center and displays the solving method to the underground personnel.
And S112, if the control center does not have a method for solving the similar problem downhole equipment, establishing remote video connection, and solving the problem of the downhole equipment through remote guidance.
Through the steps, the method digitalizes the inspection process and the inspection task, and dynamically plans the inspection route, so that the inspection threshold is reduced, the inspection efficiency is improved, and the safety of mine production is enhanced; the method can assist the actual recognition result of the underground inspection personnel in comprehensively judging the equipment problems, and improve the recognition efficiency and the recognition accuracy of the equipment problems; 3. according to the method, corresponding experts can be intelligently matched from the expert database according to the equipment model and the equipment fault, the field problems are remotely marked and guided through the experts, and inspection personnel are assisted to overhaul the equipment, so that the threshold of the inspection personnel is reduced, and the efficiency of solving the problems is improved.
Based on the above embodiments, the control center is communicatively coupled with downhole equipment and downhole personnel.
In some embodiments, step S102 includes:
and S1021, the control center acquires the position and real-time state information of the underground equipment to be inspected and the underground personnel, and establishes an inspection flow for at least one underground equipment, wherein each inspection flow comprises a plurality of inspection steps.
And step S1022, in response to the abnormal state of the underground equipment, the control center replans the inspection task and the inspection route and displays the inspection task and the inspection route to underground personnel, wherein the inspection route is realized through a dynamic programming algorithm.
The definition standard of the abnormal state of the underground equipment is as follows: and judging that the state of the underground equipment is abnormal when the data acquired by the sensor arranged on the underground equipment exceeds a preset threshold value.
And step S1023, the underground terminal projects the re-planned inspection task and the inspection route to the eyes of underground personnel in a virtual picture mode.
Step S106 includes:
and S1061, identifying the two-dimensional code of the underground equipment to be inspected by the underground terminal, acquiring the state information of the underground equipment to be inspected, generating an underground equipment problem interface diagram, and projecting the underground equipment problem interface diagram to the eyes of underground personnel in a virtual picture mode.
In some embodiments, establishing a remote video connection and solving the downhole equipment problem by remote guidance in step S112 includes:
according to the model and the problem of the underground equipment, the expert in the field is matched from the database of the control center, and the expert in the field is remotely connected through video or voice.
In some embodiments, step S112 further includes:
and S1121, receiving the gesture of the underground personnel by the underground terminal, and acquiring the depth information of each frame of image of the shot video to obtain a depth map.
S1122, according to each frame of depth map, capturing characteristic points of finger joints of underground personnel, drawing finger contours and identifying position information of fingertips.
S1123, color labeling and connecting are carried out on the position information among the continuous multi-frame depth maps, and the fault position of the underground equipment is labeled in the video.
And S1124, uploading the video with the annotation information to a control center.
In view of the above, according to a second aspect of the embodiments of the present invention, there is provided an inspection device for underground coal mine equipment based on AR technology, including: a control center and a downhole terminal. And the control center is used for acquiring the position and real-time state information of the underground equipment and underground personnel to be inspected, and planning an inspection task and an inspection route. The control center is typically located on the well. The underground terminal is used for displaying the inspection task and the inspection route to underground personnel; identifying the underground equipment to be inspected, generating an underground equipment problem interface diagram, and displaying the underground equipment problem interface diagram to underground personnel, wherein the underground terminal is integrated on a safety helmet; if the underground personnel can solve the problem of the underground equipment by themselves, shooting and uploading a video for solving the problem by the control center; and if the underground personnel can not solve the underground equipment problem by themselves, calling the solving method of the underground equipment with the same problem from the control center and displaying the solving method to the underground personnel.
As a possible implementation, the downhole terminal is AR glasses, and the AR glasses include: camera, time-of-flight sensor and miniature projecting apparatus. The camera is used for acquiring videos of underground equipment, identifying the underground equipment and uploading the videos; the time-of-flight sensor (ToF sensor) is used for measuring the distance between an object and the time-of-flight sensor, obtaining the depth information of an image, carrying out three-dimensional recognition on the gestures of underground personnel and providing interaction between a virtual picture and real actions; a micro projector for projecting video and images into the eyes of downhole personnel, wherein the video and images are superimposed with the real environment.
Optionally, in order to enable the expert in the field to work on the spot, the device of the invention further comprises an aboveground terminal, which is communicatively coupled with the control center for interacting with the expert in the field.
The contents of the above embodiments will be described with reference to a preferred embodiment.
Referring to fig. 2, the method for inspecting the underground coal mine equipment based on the AR technology according to the preferred embodiment includes the following steps:
s201, a patrol worker wears and opens AR glasses;
s202, the control center pushes a polling task;
s203, the inspection personnel confirm the inspection task through AR virtual interaction;
s204, displaying the specific routing inspection route and the routing inspection time of each routing inspection place by the AR glasses;
s205, displaying the inspection tool to be carried by the AR glasses, and prompting an inspector to confirm;
s205, routing the polling personnel to a first polling place according to the real-time navigation of the AR glasses;
s206, after the mobile terminal arrives at the inspection field, the AR glasses identify the two-dimensional code of the specific inspection equipment, and state information of the equipment to be inspected is obtained;
s207, according to the state information, the AR glasses push the specific inspection steps aiming at the equipment to inspection personnel one by one, so that the inspection personnel can inspect the components of the equipment one by one strictly according to the inspection steps;
s208, when the equipment problem is checked, if the polling personnel can solve the problem by themselves, the AR glasses shoot and upload a solution video for subsequent recording and checking; if the polling personnel can not solve the problem by themselves, the AR glasses call a solution from the database according to the equipment problem and push the solution to the polling personnel; if the problems of the related equipment can not be solved or can not be retrieved after the pushing, retrieving the corresponding experts from the database according to the equipment type and the fault type, establishing remote video connection, and carrying out equipment maintenance under the guidance of the experts. Through the three-dimensional gesture recognition function of AR glasses, the routing inspection personnel can mark and explain the virtual picture of the shot equipment by utilizing gestures, and the equipment picture with the mark is transmitted to a remote expert in real time through the AR glasses. In the whole routing inspection process, the AR glasses shoot routing inspection videos and upload the routing inspection videos, the traceability of the routing inspection process is ensured, and routing inspection problems and a solution method can be arranged in a database.
If the control center receives the abnormal state information or alarm information with higher priority in the process of carrying out a certain inspection task, the control center intelligently matches the optimal inspection personnel according to the position of each inspection personnel in the well and the inspection task execution progress of each inspection personnel, pushes a new inspection task with high priority to the inspection personnel, temporarily interrupts and stores the inspection task in progress, and returns to the step S202.
Fig. 3 is a schematic diagram of an inspection process using a coal mining machine as an example. The inspection of the coal mining machine sequentially comprises the following steps:
s301, preparing tools and materials such as a screwdriver, a wrench, a universal meter and lubricating oil;
s302, checking the conditions of working surfaces such as roof supports, coal wall protecting walls and the like;
s303, the equipment is shut down and powered off in sequence;
s304, checking the states of a motor and a gear box of the traction part;
s305, checking the states of a motor and a gear box of the cutting part;
s306, checking the electrical connection condition and judging whether the electrical leakage risk exists or not;
s306, checking the state of the cutting head, and judging whether the cutting head has broken teeth, looseness and other conditions;
s307, cleaning the inspection site, and confirming that no tools, materials and the like are left on the site;
s308, recovering the power supply of the coal mining machine;
s309, the coal mining machine operates again.
Each of the operation steps has more specific and detailed prompt information, and specific information such as the type of a screwdriver, the type of a wrench, the amount of lubricating oil and the like which need to be carried can be noted in detail in the prompt information according to the actual situation of the coal mining machine in step S301, so that detailed prompt is ensured, and the inspection requirement is met. And pushing different forms of each assembly flow is carried out, so that the pushing flow is ensured to be correct and complete. The specific information such as screwdriver model, spanner model, the volume of lubricating oil ensures that the suggestion is detailed, satisfies the demand of patrolling and examining. And pushing different forms of each assembly flow is carried out, so that the pushing flow is ensured to be correct and complete.
In the process that the patrol personnel perform the patrol task on the coal mining machine, when the running states of underground equipment and equipment fed back by the personnel are abnormal, the control center can warn the patrol personnel and push a new patrol task and a corresponding patrol process according to the completion progress and the distance of the patrol task, meanwhile, the current patrol task can be subjected to simple temporary ending and recording, after the emergency task is processed, the recording point can be returned to, and the remaining steps of the previous patrol task can be continuously performed.
Fig. 4 is a schematic diagram of communication between a control center and a remote expert and a downhole inspection worker according to a preferred embodiment of the present invention. The expert system is built by utilizing a database technology, and all experts related to relevant problems in the inspection equipment can be recorded into the expert database. Meanwhile, the common routing inspection problems corresponding to each routing inspection device can be classified and recorded into a routing inspection problem database. According to the technical characteristics and the excellence direction of each expert in the database, the expert database and the routing inspection problem database are connected with each other in a many-to-many mode. After the patrol inspection personnel judge the category of the patrol inspection problem, the AR glasses end can automatically search out the expert adaptive to the patrol inspection problem according to the database and project the expert in front of the patrol inspection personnel in a virtual picture mode. After the inspection personnel select the corresponding experts, video contact can be remotely established with the experts, and the maintenance process is completed under the guidance of remote videos.
The interaction between the patrol personnel and the virtual picture generated by the AR glasses is realized through the ToF sensor. ToF sensors support multiple modes. Under a common recording mode, the ToF sensor records and uploads a patrol video to a control center in real time; in the gesture labeling mode, the ToF sensor acquires depth information of each frame of shot image in real time by using the principle that reflected light reaches the sensor at different object distances in different time periods in the process of shooting a video. For each frame of depth map, tens of characteristic points of finger joints can be captured by utilizing a three-dimensional gesture recognition technology, so that finger contours are drawn, and the position information of fingertips is recognized. The method comprises the steps that color marking and connection are carried out on fingertip position information of continuous multi-frame pictures, so that the fault position can be marked in a real-time recorded video, the video is uploaded to a control center in a unified mode, and a video stream with marking information is transmitted to an expert end and is filed in a local server by the control center.
It should be noted that, in the description of the present invention, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In addition, in the description of the present invention, "a plurality" means two or more unless otherwise specified.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; may be mechanically coupled, may be electrically coupled or may be in communication with each other; they may be directly connected or indirectly connected through intervening media, or they may be connected internally or in any other suitable relationship, unless expressly stated otherwise. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, the first feature "on" or "under" the second feature may be directly contacting the first and second features or indirectly contacting the first and second features through an intermediate. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature may be directly under or obliquely under the first feature, or may simply mean that the first feature is at a lesser elevation than the second feature.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (9)
1. An underground coal mine equipment inspection method based on AR technology is characterized by comprising the following steps:
the control center acquires the position and real-time state information of underground equipment and underground personnel to be patrolled and examined, and plans a patrolling and examining task and a patrolling and examining route;
the underground terminal displays the inspection task and the inspection route to underground personnel;
the method comprises the steps that an underground terminal identifies underground equipment to be inspected, generates an underground equipment problem interface diagram and displays the underground equipment problem interface diagram to underground personnel, wherein the underground terminal is integrated on a safety helmet;
if the underground personnel can solve the problem of the underground equipment by themselves, the underground terminal shoots and uploads a video for solving the problem by the control center;
if the underground personnel can not solve the underground equipment problem by themselves, the underground terminal calls the solving method of the underground equipment with the similar problem from the control center and displays the solving method to the underground personnel;
if the control center does not have a method for solving the similar problems of the underground equipment, remote video connection is established, and the problems of the underground equipment are solved through remote guidance.
2. The method for inspecting the underground coal mine equipment based on the AR technology as claimed in claim 1, comprising:
the control center is communicably coupled with the underground equipment and the underground personnel, acquires the position and real-time state information of the underground equipment to be inspected and the underground personnel, and establishes inspection processes for at least one underground equipment, wherein each inspection process comprises a plurality of inspection steps;
in response to the abnormal state of the underground equipment, the control center replans the inspection task and the inspection route and displays the inspection task and the inspection route to the underground personnel, wherein the inspection route is realized by a dynamic programming algorithm;
and the underground terminal projects the re-planned inspection task and the re-planned inspection route to the eyes of the underground personnel in a virtual picture mode.
3. The method for inspecting the underground coal mine equipment based on the AR technology in claim 1 is characterized by comprising the following steps:
the underground terminal identifies the two-dimensional code of the underground equipment to be inspected, acquires the state information of the underground equipment to be inspected, generates an underground equipment problem interface diagram, and projects the underground equipment problem interface diagram in the form of a virtual picture to the front of eyes of underground personnel.
4. The method for inspecting the underground coal mine equipment based on the AR technology in claim 1 is characterized by comprising the following steps:
the underground terminal receives the gestures of the underground personnel, and obtains the depth information of each frame of image of the shot video to obtain a depth map;
capturing characteristic points of finger joints of the underground personnel according to each frame of depth map, drawing finger contours and identifying position information of fingertips;
color marking and connecting the position information among the continuous multi-frame depth maps, and marking the fault position of the underground equipment in a video;
and uploading the video with the labeling information to the control center.
5. The method for inspecting underground coal mine equipment based on the AR technology in claim 2, wherein the responding to the abnormal state of the underground equipment comprises the following steps:
and judging that the state of the underground equipment is abnormal in response to the fact that data collected by a sensor installed on the underground equipment exceeds a preset threshold value.
6. The method for inspecting underground coal mine equipment based on the AR technology, as claimed in claim 1, wherein the establishing of the remote video connection and the solving of the underground equipment problem through the remote guidance comprise:
according to the model and the problem of the underground equipment, the expert in the field is matched from the database of the control center, and the expert in the field is remotely connected through video or voice.
7. The utility model provides a well worker coal mine equipment inspection device based on AR technique which characterized in that includes:
the control center is used for acquiring the position and real-time state information of underground equipment and underground personnel to be patrolled and examined, and planning a patrolling task and a patrolling and examining route;
the underground terminal is used for displaying the inspection task and the inspection route to underground personnel; identifying the underground equipment to be inspected, generating an underground equipment problem interface diagram, and displaying the underground equipment problem interface diagram to underground personnel, wherein the underground terminal is integrated on a safety helmet; if the underground personnel can solve the problem of the underground equipment by themselves, shooting and uploading a video for solving the problem by the control center; and if the underground personnel can not solve the underground equipment problem by themselves, calling the solving method of the underground equipment with the same problem from the control center and displaying the solving method to the underground personnel.
8. The underground coal mine equipment inspection device based on the AR technology, as claimed in claim 7, wherein the underground terminal is AR glasses, the AR glasses include:
the camera is used for acquiring a video of the underground equipment, identifying the underground equipment and uploading the video;
the time-of-flight sensor is used for measuring the distance between an object and the time-of-flight sensor, obtaining the depth information of an image, carrying out three-dimensional recognition on the gestures of underground personnel and providing interaction between a virtual picture and real actions;
a micro projector for projecting video and images into the eyes of downhole personnel, wherein the video and images are superimposed with the real environment.
9. The underground coal mine equipment inspection device based on the AR technology as claimed in claim 7, further comprising an aboveground terminal communicatively coupled with the control center for interacting with experts in the field.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111582173.5A CN114510027A (en) | 2021-12-22 | 2021-12-22 | Underground coal mine equipment inspection method and device based on AR technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111582173.5A CN114510027A (en) | 2021-12-22 | 2021-12-22 | Underground coal mine equipment inspection method and device based on AR technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114510027A true CN114510027A (en) | 2022-05-17 |
Family
ID=81548310
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111582173.5A Pending CN114510027A (en) | 2021-12-22 | 2021-12-22 | Underground coal mine equipment inspection method and device based on AR technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114510027A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115100285A (en) * | 2022-08-25 | 2022-09-23 | 深圳市信润富联数字科技有限公司 | Wind power sensor installation method, device, equipment and readable storage medium |
| CN115311752A (en) * | 2022-06-14 | 2022-11-08 | 中铁建网络信息科技有限公司 | Routing inspection task scheduling method based on safety helmet with AR function and safety helmet |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107277451A (en) * | 2017-07-14 | 2017-10-20 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of remote guide scene investigation failure |
| CN107610269A (en) * | 2017-09-12 | 2018-01-19 | 国网上海市电力公司 | A kind of power network big data intelligent inspection system and its intelligent polling method based on AR |
| CN109360290A (en) * | 2018-10-31 | 2019-02-19 | 深圳供电局有限公司 | Power Line Inspection System |
| CN109685248A (en) * | 2018-11-20 | 2019-04-26 | 上海大学 | A kind of underground coal mine security staff polling path and the efficient overall planning method of task |
| WO2019174009A1 (en) * | 2018-03-15 | 2019-09-19 | 深圳市中联创新自控系统有限公司 | Machine room management method and dynamic environment system |
| CN110400388A (en) * | 2019-07-02 | 2019-11-01 | 上海德衡数据科技有限公司 | A kind of intelligent inspection system based on augmented reality |
| CN111119887A (en) * | 2020-02-19 | 2020-05-08 | 太原理工大学 | Remote intervention AR inspection system of mine fully-mechanized mining working face under holographic technology |
| CN112434880A (en) * | 2020-12-10 | 2021-03-02 | 清研灵智信息咨询(北京)有限公司 | Patrol route planning and patrol personnel management system based on deep learning |
| CN113768236A (en) * | 2021-09-28 | 2021-12-10 | 中煤科工开采研究院有限公司 | Wearable intelligent individual soldier device used in coal mine |
-
2021
- 2021-12-22 CN CN202111582173.5A patent/CN114510027A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107277451A (en) * | 2017-07-14 | 2017-10-20 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of remote guide scene investigation failure |
| CN107610269A (en) * | 2017-09-12 | 2018-01-19 | 国网上海市电力公司 | A kind of power network big data intelligent inspection system and its intelligent polling method based on AR |
| WO2019174009A1 (en) * | 2018-03-15 | 2019-09-19 | 深圳市中联创新自控系统有限公司 | Machine room management method and dynamic environment system |
| CN109360290A (en) * | 2018-10-31 | 2019-02-19 | 深圳供电局有限公司 | Power Line Inspection System |
| CN109685248A (en) * | 2018-11-20 | 2019-04-26 | 上海大学 | A kind of underground coal mine security staff polling path and the efficient overall planning method of task |
| CN110400388A (en) * | 2019-07-02 | 2019-11-01 | 上海德衡数据科技有限公司 | A kind of intelligent inspection system based on augmented reality |
| CN111119887A (en) * | 2020-02-19 | 2020-05-08 | 太原理工大学 | Remote intervention AR inspection system of mine fully-mechanized mining working face under holographic technology |
| CN112434880A (en) * | 2020-12-10 | 2021-03-02 | 清研灵智信息咨询(北京)有限公司 | Patrol route planning and patrol personnel management system based on deep learning |
| CN113768236A (en) * | 2021-09-28 | 2021-12-10 | 中煤科工开采研究院有限公司 | Wearable intelligent individual soldier device used in coal mine |
Non-Patent Citations (1)
| Title |
|---|
| 李大勇等: "基于AR技术的变电站智能巡检系统设计与实现", 《微型电脑应用》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115311752A (en) * | 2022-06-14 | 2022-11-08 | 中铁建网络信息科技有限公司 | Routing inspection task scheduling method based on safety helmet with AR function and safety helmet |
| CN115100285A (en) * | 2022-08-25 | 2022-09-23 | 深圳市信润富联数字科技有限公司 | Wind power sensor installation method, device, equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114510027A (en) | Underground coal mine equipment inspection method and device based on AR technology | |
| CN112910094B (en) | Remote automatic transformer substation inspection system and method based on ubiquitous power Internet of things | |
| US20210133149A1 (en) | Intelligent and automated review of industrial asset integrity data | |
| KR101701278B1 (en) | Remote inspection and maintenance system of ship and offshore plant | |
| CN109753081B (en) | Roadway inspection unmanned aerial vehicle system based on machine vision and navigation method | |
| CN109882244A (en) | Intelligent map building system of underground inspection robot | |
| CN103235562A (en) | Patrol-robot-based comprehensive parameter detection system and method for substations | |
| CN112798619B (en) | Rapid detection system and detection method for tunnel defects | |
| JP2007178240A (en) | Separate distance measuring device and self-advancing measuring equipment | |
| CN111178424A (en) | Petrochemical production site safety compliance real-time detection system and method | |
| CN111037574A (en) | Interactive robot system for communication station inspection and control method thereof | |
| CN108227617A (en) | A kind of tyre crane remote detection and assessment system | |
| CN115937213A (en) | Visual defect identification system of automatic inspection robot for mining monorail crane track | |
| CN111507951A (en) | Cable line tunnel engineering inspection device | |
| CN109443542B (en) | On-line monitoring device and monitoring method for forced draught fan based on infrared thermal imaging technology | |
| CN116720242A (en) | Digital twin panoramic monitoring system for high-voltage cable tunnel | |
| Liu et al. | An approach for auto bridge inspection based on climbing robot | |
| CN113050671A (en) | Unmanned aerial vehicle system for detecting natural gas leakage and detection method | |
| CN113898537A (en) | Remote intelligent inspection system for wind power plant | |
| CN211335993U (en) | 360-degree fault image detection system for metro vehicle | |
| CN210511074U (en) | Full-intelligent oil-gas pipeline inspection system based on unmanned aerial vehicle remote sensing and deep learning | |
| EP1548908B1 (en) | Method and device for inspecting linear infrastructures | |
| CN113043285A (en) | Liftable mining inspection robot | |
| CN113768236A (en) | Wearable intelligent individual soldier device used in coal mine | |
| KR20210073207A (en) | How to manage virtual monitoring by tracking device motion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220517 |
|
| RJ01 | Rejection of invention patent application after publication |