CN114554163B - Coal mine underground operation monitoring system - Google Patents
Coal mine underground operation monitoring system Download PDFInfo
- Publication number
- CN114554163B CN114554163B CN202210436348.XA CN202210436348A CN114554163B CN 114554163 B CN114554163 B CN 114554163B CN 202210436348 A CN202210436348 A CN 202210436348A CN 114554163 B CN114554163 B CN 114554163B
- Authority
- CN
- China
- Prior art keywords
- data
- real time
- real
- image
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000003245 coal Substances 0.000 title claims abstract description 29
- 238000012544 monitoring process Methods 0.000 title claims abstract description 14
- 238000005065 mining Methods 0.000 claims abstract description 24
- 231100000279 safety data Toxicity 0.000 claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 6
- 238000012545 processing Methods 0.000 claims description 8
- 230000007613 environmental effect Effects 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 6
- 239000011521 glass Substances 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 2
- 230000002194 synthesizing effect Effects 0.000 claims description 2
- 238000004364 calculation method Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 3
- 238000005553 drilling Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- E—FIXED CONSTRUCTIONS
- E21—EARTH OR ROCK DRILLING; MINING
- E21F—SAFETY DEVICES, TRANSPORT, FILLING-UP, RESCUE, VENTILATION, OR DRAINING IN OR OF MINES OR TUNNELS
- E21F17/00—Methods or devices for use in mines or tunnels, not covered elsewhere
- E21F17/18—Special adaptations of signalling or alarm devices
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F21—LIGHTING
- F21V—FUNCTIONAL FEATURES OR DETAILS OF LIGHTING DEVICES OR SYSTEMS THEREOF; STRUCTURAL COMBINATIONS OF LIGHTING DEVICES WITH OTHER ARTICLES, NOT OTHERWISE PROVIDED FOR
- F21V33/00—Structural combinations of lighting devices with other articles, not otherwise provided for
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B2027/0178—Eyeglass type
Landscapes
- Engineering & Computer Science (AREA)
- Mining & Mineral Resources (AREA)
- Physics & Mathematics (AREA)
- General Life Sciences & Earth Sciences (AREA)
- Geochemistry & Mineralogy (AREA)
- Geology (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Processing Or Creating Images (AREA)
- Alarm Systems (AREA)
Abstract
A coal mine underground operation monitoring system comprises: the NPU unit is located underground, calculates operation data used for indicating an operator to operate operation equipment according to an underground operation real-time image collected by the mining camera, and processes the data of an operation area collected by the operation environment sensor to obtain environment safety data; and the AR equipment is provided with a camera, real images are obtained by the AR equipment through the camera in real time, operation data and environment safety data are obtained from the NPU unit in real time, virtual images are generated in real time according to the operation data, the virtual images and the real images are synthesized into AR images with the operation data, the environment safety data and warning marks in real time, and the AR images are displayed in real time. The invention can avoid accidents in time.
Description
Technical Field
The invention belongs to the technical field of underground coal mines, and particularly relates to an underground coal mine operation monitoring system.
Background
Coal is used as the first large energy in China, the industrial scale of the coal industry is large, the distribution area is wide, the danger coefficient is high, the demand for improving the safe production level is raised to be a major national subject, and the construction requirement of intelligent mines is very urgent.
At present, when carrying out colliery borehole operation, transmit the real-time video of operation to aboveground equipment through the intelligent camera in the pit usually, the personnel in the pit are kept watch on to the operation in the pit according to the video, and such mode has obtained certain guarantee in the aspect of the security, when having at least the accident, can in time arrange the resource search and rescue in the pit, still exist not enoughly: the video data in the well is transmitted to the well, and even if the video data passes through the 5G network under construction, a large delay, even a few seconds, still can cause accidents to happen.
Disclosure of Invention
Based on this, to the technical problem, provide a colliery borehole operation monitoring system.
In order to solve the technical problems, the invention adopts the following technical scheme:
a coal mine underground operation monitoring system comprises:
the NPU unit is located underground, and the NPU unit calculates operation data used for indicating an operator to operate operation equipment according to a real-time image of underground operation acquired by the mining camera, and processes data of an operation area acquired by an operation environment sensor to obtain environment safety data; and
the AR equipment is provided with a camera, real images are obtained by the AR equipment through the camera in real time, operation data and environment safety data are obtained from the NPU unit in real time, virtual images are generated in real time according to the operation data, the virtual images and the real images are synthesized into AR images with the operation data, the environment safety data and warning marks in real time, and the AR images are displayed in real time.
Compared with the prior art, the method and the device do not need to transmit videos to the well, and the operation data and the environment safety data which are obtained by calculation and processing of the underground NPU unit can be quickly transmitted to the AR equipment, so that the real-time performance of the AR image displayed by the AR equipment is high, underground operators can timely and intuitively know the existence of the surrounding safety parameter data and risks through the AR image, accidents are avoided, and meanwhile, the current operation progress and state can be timely known through the AR image, so that accurate bases are provided for the next operation.
Drawings
The invention is described in detail below with reference to the following figures and embodiments:
FIG. 1 is a schematic structural view of the present invention;
FIG. 2 is a schematic diagram of an AR apparatus of the present invention;
FIG. 3 is a diagram illustrating an AR image according to the present invention;
FIG. 4 is a diagram illustrating an AR image according to a second embodiment of the present invention.
Detailed Description
As shown in fig. 1, an embodiment of the present specification provides a coal mine underground operation monitoring system, which includes a mining camera 110, an operation environment sensor 120, an edge computing device 130, a cloud platform 140, and an AR device 150.
Work environment sensors 120 are used to collect data for the work area.
The mining camera 110 is provided with an NPU (neural-network processing units) unit, and the NPU unit calculates operation data for instructing an operator to operate the operation device according to a real-time image of the downhole operation acquired by the mining camera 110, and processes data of an operation area acquired by the operation environment sensor 120 to obtain environment safety data.
Taking the drilling operation as an example, the operation data includes the length of the drill bit exposed outside the coal wall and the length of the drill bit drilled into the coal wall, and the NPU unit calculates the length of the drill bit exposed outside the coal wall and the length of the drill bit drilled into the coal wall through a pre-trained target detection and tracking model.
For drilling operation, the working environment sensor 120 includes a mining stress sensor and a gas concentration sensor, the data acquired by the mining stress sensor and the gas concentration sensor are analog signals, the analog signals need to be processed by an NPU unit, the analog signals are converted into digital signals, and the digital signals are converted into environment safety data which can be seen by operators, namely, impact energy parameters and gas concentration.
The calculation power of the NPU unit in the mining camera 110 is generally about 1-2T, and the timeliness of data transmission is high when the mining camera 110 is used as an AI calculation power device closest to an operator.
It should be noted that when the computing power of the mining camera 110 is insufficient, centralized operation may be performed by the NPU unit in the downhole edge computing device 130, and then the operation result may be transmitted to the mining camera 110. The calculation power of the NPU unit in the edge calculation device 130 is generally about 10 times that of the mining camera 110, the edge calculation device 130 is a device having the NPU unit in the downhole local area network, such as a router, a server and the like in the downhole local area network, and the mining camera 110, the working environment sensor 120 and the edge calculation device 130 are all connected to the downhole local area network in a wired or wireless manner.
Furthermore, the NPU unit in the cloud platform 140 may also provide an auxiliary computing power, the cloud platform 140 is connected to the local area network downhole, the computing power of the NPU unit in the cloud platform 140 is typically tens of times or even hundreds of times that of the edge computing device 130, and a common and basic big data operation scene, such as the calculation of the whole real environment image, may be processed in a centralized manner, and this part of calculation is used as the basis of the operation, and the updating frequency is relatively low.
The NPU units in the mining camera 110 and the edge computing device 130 are used for computing local images before the eyes of the operator, and the local images are relatively high in updating frequency and high in requirement on timeliness due to the implementation of the operation, and specifically, the NPU unit in the mining camera 110 or the NPU unit in the edge computing device 130 needs to be set in advance according to actual test conditions, so that the safety and timeliness are higher.
An image within 10 seconds can be taken from the mining camera 110, the time (data generation time) for obtaining the operation data and the environmental safety data through calculation and processing by the NPU unit in the mining camera 110 and the time for transmitting the data to the AR device 150 are tested, the two times are added to obtain the total time, similarly, the total time corresponding to the edge calculation device 130 can also be obtained, the two total times are compared, if the total time of the NPU unit in the mining camera 110 is adopted is shorter, the NPU unit in the mining camera 110 is selected, otherwise, the NPU unit in the edge calculation device 130 is used.
Certainly, although the NPU unit is placed in the mining camera 110, the edge computing device 130 and the cloud platform 140, with the rapid development of science and technology, the cost of the NPU unit is likely to be further reduced in the future, and the NPU unit can be directly integrated into the AR device 150, so that the AR device 150 has basic computing power, can obtain operation data through self-computing and process the environment safety data, and finds the mine and seeks help with the camera 110 and the edge computing device 130 under the condition of insufficient computing power.
As shown in fig. 2, the AR device 150 is worn by an operator, is connected to a downhole local area network, and has a camera 151, the AR device 150 obtains a real image in real time through the camera 151, obtains operation data and environment safety data from an NPU unit in real time, generates a virtual image in real time according to the operation data, synthesizes the virtual image and the real image into an AR image with the operation data, the environment safety data and warning labels in real time, and displays the AR image in real time.
In the present embodiment, the AR device 150 is a miner's lamp integrated with AR glasses 152, which includes, in addition to the camera 151 and the AR glasses 152, an illumination unit 153, an audio unit 154, a communication unit 155, and a processing unit 156, the processing unit 156 being configured to:
real images are acquired in real time through the camera 151, and job data and environmental security data are acquired in real time from the NPU unit through the communication unit 155.
And generating a virtual image in real time according to the operation data, and synthesizing the virtual image and the real image into an AR image with the operation data, the environment safety data and the warning label in real time.
And comparing the environmental safety data with a preset threshold in real time, if the comparison result represents that the environmental safety data exceed the standard, sending out a sound alarm through the audio unit 154, and sending alarm information to the aboveground control equipment through the underground local area network through the communication unit 155.
The AR image is displayed in real time through the AR glasses 152, and the warning annotation in the AR image is shown in fig. 3 and 4.
In fig. 3, the impact energy parameter and the gas concentration are not over-marked, so that "safety" is marked at the back of the impact energy parameter and the gas concentration, and "overall safety" is marked at the top, and the drill bit can advance by 20 meters, where 20 meters is a preset distance, and the current drill bit position + the preset distance = the future drill bit position, the processing unit 156 can predict the impact energy parameter and the gas concentration to obtain predicted values of the impact energy parameter and the gas concentration at the future drill bit position, and if the predicted values are within a safety threshold, the drill bit can advance by 20 meters.
In FIG. 4, the impact energy parameter is out of standard and the gas concentration is not out of standard, so the "yellow warning" is marked behind the impact energy parameter, the "safety" is marked behind the gas concentration, and the "yellow warning" is marked at the top, the risk is higher! "
For the drilling operation, the real image is an image of the drill bit exposed out of the coal wall, the virtual image is an image of the drill bit drilled into the coal wall, the virtual image is an extension of the real image, the position (coordinates) of the drill bit in the real image can be determined through an image processing algorithm, and then the virtual image can be generated according to the known length of the drill bit drilled into the coal wall and the shape of the drill bit.
However, those skilled in the art should recognize that the above-described embodiments are illustrative only, and not limiting, and that changes and modifications can be made to the above-described embodiments without departing from the true spirit and scope of the invention, which is defined by the following claims.
Claims (7)
1. A coal mine underground operation monitoring system is characterized by comprising:
the NPU unit is located underground, and the NPU unit calculates operation data used for indicating an operator to operate operation equipment according to a real-time image of underground operation acquired by the mining camera, and processes data of an operation area acquired by an operation environment sensor to obtain environment safety data; and
the AR equipment is provided with a camera, real images are obtained by the AR equipment through the camera in real time, operation data and environment safety data are obtained from the NPU unit in real time, virtual images are generated in real time according to the operation data, the virtual images and the real images are synthesized into AR images with the operation data, the environment safety data and warning marks in real time, and the AR images are displayed in real time;
the borehole operation is the operation of driling, the operation data include the length that the drill bit exposes outside the coal wall and the length that the drill bit bored into the coal wall, real image is the image that the drill bit exposes outside the coal wall, virtual image is the image that the drill bit bored into the coal wall, environmental safety data include impact energy parameter and gas concentration, the warning mark includes the warning mark according to impact energy parameter and gas concentration prediction.
2. The coal mine underground operation monitoring system of claim 1, wherein the NPU unit is provided in the AR device.
3. The coal mine underground operation monitoring system of claim 1, wherein the NPU unit is disposed in the mining camera.
4. The coal mine underground operation monitoring system of claim 3, further comprising an edge computing device and a cloud platform, wherein the mining camera, the edge computing device and the AR device are all arranged in an underground local area network, the cloud platform is connected with the underground local area network, and NPU units are arranged in the edge computing device and the cloud platform.
5. The coal mine underground operation monitoring system of claim 4, wherein the operating environment sensors comprise mine stress sensors and gas concentration sensors.
6. The coal mine underground operation monitoring system of claim 3, wherein the NPU calculates the length of the drill exposed out of the coal wall and the length of the drill drilled into the coal wall through a pre-trained target detection and tracking model.
7. The coal mine downhole operation monitoring system of claim 6, wherein the AR device is an AR glasses integrated miner's lamp comprising a lighting unit, a camera, an audio unit, a communication unit, and a processing unit configured to:
real images are obtained in real time through a camera, and the operation data and the environment safety data are obtained in real time through the communication unit;
generating a virtual image in real time according to the operation data, and synthesizing the virtual image and the real image into an AR image with operation data, environment safety data and warning label in real time;
the environmental safety data are compared with a preset threshold value in real time, if the comparison result represents that the environmental safety data exceed the standard, a sound alarm is sent out through the audio unit, and alarm information is sent to the aboveground control equipment through the underground local area network through the communication unit;
and displaying the AR image in real time through AR glasses.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210436348.XA CN114554163B (en) | 2022-04-25 | 2022-04-25 | Coal mine underground operation monitoring system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210436348.XA CN114554163B (en) | 2022-04-25 | 2022-04-25 | Coal mine underground operation monitoring system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114554163A CN114554163A (en) | 2022-05-27 |
| CN114554163B true CN114554163B (en) | 2022-08-19 |
Family
ID=81667613
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210436348.XA Active CN114554163B (en) | 2022-04-25 | 2022-04-25 | Coal mine underground operation monitoring system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114554163B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014066983A1 (en) * | 2012-10-31 | 2014-05-08 | Centre For Excellence In Mining Innovation | Mining helmet |
| KR20180032138A (en) * | 2016-09-21 | 2018-03-29 | 주식회사 지오제니컨설턴트 | Method for Surveying and Monitoring Mine Site by using Virtual Reality and Augmented Reality |
| WO2020049999A1 (en) * | 2018-09-03 | 2020-03-12 | 三菱自動車工業株式会社 | Manufacturing assistance system, method, and program |
| CN113746936A (en) * | 2021-09-15 | 2021-12-03 | 太原理工大学 | VR and AR distributed cooperation fully-mechanized coal mining face intelligent monitoring system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20130067450A (en) * | 2011-12-14 | 2013-06-24 | 한국전자통신연구원 | Field applied safety helmet apparatus for ar-based of maintenance and method for providing its application service |
| CN106150554B (en) * | 2016-07-29 | 2018-04-10 | 邹平县供电公司 | A kind of electrical communication based on stratum construction detects early warning system |
| CN106089304B (en) * | 2016-07-29 | 2018-10-19 | 邹平县供电公司 | One kind controlling early warning communication system based on stratum construction multifunctional electric |
| US20210366256A1 (en) * | 2018-10-22 | 2021-11-25 | Motive Drilling Technologies, Inc. | Systems and methods for oilfield drilling operations using computer vision |
| CN110984955A (en) * | 2019-11-25 | 2020-04-10 | 江苏安能钻掘设备科技有限公司 | Remote monitoring system for drilling machine |
| US11192192B2 (en) * | 2020-02-10 | 2021-12-07 | The Boeing Company | Method and apparatus for drilling a workpiece |
-
2022
- 2022-04-25 CN CN202210436348.XA patent/CN114554163B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014066983A1 (en) * | 2012-10-31 | 2014-05-08 | Centre For Excellence In Mining Innovation | Mining helmet |
| KR20180032138A (en) * | 2016-09-21 | 2018-03-29 | 주식회사 지오제니컨설턴트 | Method for Surveying and Monitoring Mine Site by using Virtual Reality and Augmented Reality |
| WO2020049999A1 (en) * | 2018-09-03 | 2020-03-12 | 三菱自動車工業株式会社 | Manufacturing assistance system, method, and program |
| CN113746936A (en) * | 2021-09-15 | 2021-12-03 | 太原理工大学 | VR and AR distributed cooperation fully-mechanized coal mining face intelligent monitoring system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114554163A (en) | 2022-05-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111341068B (en) | Construction method of drilling site dangerous area early warning system based on deep learning | |
| CN110121053B (en) | Video monitoring method for drilling site risk grading early warning | |
| CN208900095U (en) | A kind of downhole data acquisition terminal and processing system | |
| CN113153431A (en) | Coal and rock dynamic disaster monitoring and early warning visualization system and method based on 5G communication | |
| CN104742939B (en) | Temperature information collection and early warning method and system for high speed railway | |
| CN115640707A (en) | Tunneling monitoring system and method, electronic equipment and storage medium | |
| CN108230624A (en) | Mountain landslide supervision system based on CC2530 | |
| CN203132582U (en) | Construction-safety multi-sensor automation monitoring system for close subway | |
| CN114320470B (en) | Electric safety monitoring system and method for coal mine shaft | |
| CN114554163B (en) | Coal mine underground operation monitoring system | |
| CN114510027A (en) | Underground coal mine equipment inspection method and device based on AR technology | |
| CN110985125A (en) | Deep well soft coal rock burst disaster monitoring and early warning system and early warning method thereof | |
| AU2021100619A4 (en) | Mine safety monitoring system | |
| CN116582930A (en) | Underground personnel accurate positioning software system based on regional positioning | |
| Liu et al. | Research progress and prospects of intelligent technology in underground mining of hard rock mines | |
| CN114611944B (en) | Mine goaf risk prevention and control management method, controller and system | |
| CN115798137B (en) | Three-dimensional visual big data monitoring and early warning system based on Internet of things | |
| CN110806859A (en) | Modular drilling data monitoring and design system based on machine learning | |
| CN212642810U (en) | Fault fracture zone tunnel construction early warning system | |
| CN106154256B (en) | The automatic real-time positioning system of getting working face drilling depth based on radar detection | |
| CN103161508A (en) | Underground positioning system based on wireless data transmission network | |
| CN113565570A (en) | Index setting method for early warning of water damage | |
| CN113556384A (en) | Outburst prevention early warning analysis system suitable for mining heading machine | |
| CN113653534A (en) | Mine water disaster early warning system and method | |
| CN220526358U (en) | AR (augmented reality) glasses-based coal mine intelligent operation guidance and monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |