CN115251511A - Intelligent fully-mechanized mining working face inspection system based on configurable mine safety helmet - Google Patents
Intelligent fully-mechanized mining working face inspection system based on configurable mine safety helmet Download PDFInfo
- Publication number
- CN115251511A CN115251511A CN202210673789.1A CN202210673789A CN115251511A CN 115251511 A CN115251511 A CN 115251511A CN 202210673789 A CN202210673789 A CN 202210673789A CN 115251511 A CN115251511 A CN 115251511A
- Authority
- CN
- China
- Prior art keywords
- inspection
- safety helmet
- personnel
- configurable
- centralized control
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A42—HEADWEAR
- A42B—HATS; HEAD COVERINGS
- A42B3/00—Helmets; Helmet covers ; Other protective head coverings
- A42B3/04—Parts, details or accessories of helmets
- A42B3/0406—Accessories for helmets
-
- A—HUMAN NECESSITIES
- A42—HEADWEAR
- A42B—HATS; HEAD COVERINGS
- A42B3/00—Helmets; Helmet covers ; Other protective head coverings
- A42B3/04—Parts, details or accessories of helmets
- A42B3/0406—Accessories for helmets
- A42B3/042—Optical devices
-
- A—HUMAN NECESSITIES
- A42—HEADWEAR
- A42B—HATS; HEAD COVERINGS
- A42B3/00—Helmets; Helmet covers ; Other protective head coverings
- A42B3/04—Parts, details or accessories of helmets
- A42B3/0406—Accessories for helmets
- A42B3/0433—Detecting, signalling or lighting devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/003—Navigation within 3D models or images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Human Computer Interaction (AREA)
- Emergency Alarm Devices (AREA)
Abstract
The invention belongs to the field of underground inspection intellectualization, and particularly relates to an intelligent comprehensive mechanized coal mining face inspection system based on a configurable mine safety helmet. The system comprises an inspection end, a centralized control center and a ground dispatching center. The inspection end comprises a configurable mine safety helmet worn by inspection workers of the underground working face. The centralized control center comprises a configurable mine safety helmet, an explosion-proof computer and a Kinect camera. The ground dispatching center comprises a head display VR device, a Kinect camera and a server, workers in the ground dispatching center monitor a working site through the head display VR device, observe the running state of the fully mechanized mining equipment in a virtual scene, position the underground personnel and the physiological state of the workers in other two places, track the human body through the Kinect camera when a virtual conference is carried out, and create corresponding virtual figures.
Description
Technical Field
The invention belongs to the field of underground inspection intellectualization, and particularly relates to an intelligent comprehensive mining working surface inspection system based on a configurable mine safety helmet.
Background
The 'internet + mining industry' characterized by high networking, big data, cooperative work and distributed service of the industry is a necessary way for the industry to realize safe, efficient, green and sustainable development and a new direction with future certainty for the industrial development, and on the basis, the intellectualization of underground equipment and the visualization of a fully mechanized mining face improve the safety of underground work and become a necessary trend.
At present, the working environment is still very complicated in the pit, but the work of patrolling and examining in the pit is still necessary, consequently strengthens the security of patrolling and examining end work, and high efficiency, intellectuality have very big meaning. The equipment worn by the current underground inspection personnel during working comprises work clothes, work shoes, safety helmets and miner lamps, and the safety helmets used as protective tools have upgrading space on functions, so the invention aims at the expansion of the functionality of the safety helmets and aims at the problems faced by the current inspection work, such as remote communication on the ground and underground, work positioning of the inspection personnel, monitoring of the underground working environment and monitoring of the operation condition of fully mechanized mining equipment.
The patent publication No. CN112914188A discloses a mining safety cap integrated with an LED miner lamp, which comprises a mining safety cap body, an LED explosion-proof lamp group, an explosion-proof battery pack, an explosion-proof conducting wire and an explosion-proof charging port, wherein the mining safety cap body comprises a cap shell, a cap brim, an anti-seismic layer, a cap band and buckles, the cap band is positioned on two sides of the cap shell, the lower parts of the cap band are connected through the buckles, the LED explosion-proof lamp group and the explosion-proof switch are integrated in the cap brim, the LED explosion-proof lamp group comprises the LED miner lamp and a lamp shade, a rechargeable lithium battery is arranged in the explosion-proof battery pack and positioned on two sides of the cap shell, and the LED explosion-proof lamp group, the explosion-proof battery pack and the explosion-proof charging port are connected through the explosion-proof conducting wire.
The publication No. CN112690529A discloses a mine safety helmet which comprises a gas concentration sensor, an oxygen concentration sensor, a positioning module, a controller, an alarm, a wireless communication module and a voice broadcasting device, wherein the gas concentration sensor, the oxygen concentration sensor and the positioning module are used for detecting the gas concentration, the oxygen concentration and position coordinates of the current position, the wireless communication module is used for realizing ad hoc networking with the mine safety helmet carried by other personnel in a mine, interaction of the gas concentration and the oxygen concentration of each position is carried out, when the gas concentration or the oxygen concentration of the current position is too high, the alarm is carried out, and an evacuation route is formulated by combining the gas concentration, the oxygen concentration and the position coordinates of the current position of other positions, so that workers carrying the mine safety helmet are guided to evacuate safely.
In the above scheme, the function of safety helmet itself has been strengthened, has improved dangerous early warning ability, has improved colliery workman work efficiency, but does not strengthen the intelligence and patrols and examines the aspect, has the disappearance in the aspect of intellectuality.
The publication number CN113331525A "is used for outstanding safety inspection device of coal and gas", includes the safety helmet, sets up the camera that is used for gathering colliery environmental information in the pit on the safety helmet and sets up the position control device who is used for adjusting the camera position on the safety helmet. The underground patrol personnel wear the safety helmet and acquire underground image information in real time through the camera.
Among the above-mentioned scheme, it obtains data through adding the camera on patrolling and examining personnel's safety helmet to have proposed, has improved and has patrolled and examined efficiency, utilizes the safety helmet as the carrier of camera, can liberate patrolling and examining personnel's both hands, is convenient for patrol and examine personnel real-time manually operation, nevertheless does not have the advantage that AR virtual reality fuses, and the people needs manually operation when the operation, has the risk, exists not enoughly in the aspect of intellectuality.
The disclosure No. CN111374388A discloses an AR safety helmet, which comprises a safety helmet body and AR goggles connected with the safety helmet body, wherein the AR goggles comprise a goggle body, a camera shooting module and an optical waveguide display module which are arranged on the goggle body, an electrochromic film is arranged in front of the goggle body, the camera shooting module is arranged in the middle of the goggle body, and the optical waveguide display module is connected with a control module; the camera shooting and photographing module collects photo or video information; the optical waveguide display module drives signal transmission and displays the acquired photo or video information; the electrochromic film control switch is arranged on one side of the goggle body, and the color of the electrochromic film is adjusted through the switch.
The scheme is mainly used for a non-underground construction site, and a virtual interaction function is integrated, but the function is single, so that the method is not enough in the aspects of exploring more intelligent use of AR equipment, such as virtual interaction and cooperative work, does not have explosion-proof requirements, and cannot be used for underground work.
Disclosure of Invention
The invention provides an intelligent comprehensive mechanized mining face inspection system based on a configurable mine safety helmet, aiming at solving the problems.
The invention adopts the following technical scheme: an intelligent fully-mechanized coal mining face inspection system based on a configurable mine safety helmet comprises an inspection end, a centralized control center and a ground scheduling center.
The inspection end comprises a configurable mine safety helmet worn by inspection workers of the underground working face.
The centralized control center comprises a configurable mine safety helmet, an explosion-proof computer and a Kinect camera, wherein the Kinect camera is arranged obliquely behind workers in the centralized control center and is used for tracking the human bodies of the workers to generate three virtual persons which are respectively a coal mining machine monitoring virtual person, a hydraulic support monitoring virtual person and a scraper conveyor monitoring virtual person; the virtual human respectively manages the monitored coal mining machine, the monitored hydraulic support and the monitored scraper conveyor through the neuron computer management of the bionic human brain; the virtual human is generated through three-dimensional modeling and is bound with a centralized control center worker, and if the system finds abnormality when the equipment runs, a monitoring worker controls any one virtual human through switching management permissions; and the centralized control center staff uses an explosion-proof computer to sort and gather the information collected by the fully mechanized coal mining face and connect the dispatching center and the patrol inspection end staff.
The ground dispatching center comprises a head display VR device, a Kinect camera and a server, ground dispatching center workers monitor a working site through the head display VR device, the running state of the fully mechanized mining equipment, the underground personnel positioning condition and the physiological conditions of workers located in other two places are observed in a virtual scene, human body tracking is carried out through the Kinect camera when a virtual conference is carried out, and a corresponding virtual person is created.
The configurable mine safety helmet comprises a safety helmet main body, wherein AR glasses, a three-dimensional laser radar, a single chip microcomputer controller, a gas concentration sensor, a forehead sensor, an alarm and a power supply module are integrated on the safety helmet main body, and the AR glasses are in signal connection with a wireless base station arranged underground to perform data transmission; the three-dimensional laser radar is placed at the top of the safety helmet and used for collecting surrounding three-dimensional information; the single chip microcomputer controller is connected with the gas concentration sensor and the alarm, when the data measured by the gas concentration sensor exceeds a safety value, the data are transmitted back to the centralized control center through the AR glasses, the alarm sounds to remind inspection personnel, and the centralized control center staff generate an evacuation route in the AR glasses; the forehead sensor detects the blood oxygen concentration, the body temperature and the heart rate of a worker, and the measured data are connected into the AR glasses for monitoring and analysis; the power module provides energy for other structures.
The safety helmet is characterized in that a non-invasive brain-computer interface is integrated on the safety helmet main body, collects electroencephalogram signals of an inspector, detects whether the spirit of the inspector is concentrated or not by detecting, recording, amplifying and transmitting nerve impulses issued by motor neurons and matching with a forehead sensor arranged on the forehead part of the safety helmet, and connects measured data into AR glasses.
The method comprises the steps that a depth camera is arranged on AR glasses, the depth camera collects depth image data and is fused with image data collected by a three-dimensional laser radar, calibration of the depth camera and splicing and optimization of point clouds are carried out at a ground dispatching center to obtain a point cloud model, and meanwhile, a three-dimensional scene is transmitted to the AR glasses and a centralized control center through a wireless network.
The AR glasses are integrated with inertial sensors, the inertial sensors perform positioning in real time, positioning data information is transmitted to a centralized control center to establish a positioning point diagram, points on the positioning point diagram represent the current positions of inspection personnel, and therefore a positioning system is established, and when the centralized control center needs to contact the inspection personnel, communication calling is performed by clicking the points; when the patrol personnel is required to determine a certain position in the well, the positioning system quickly determines the nearby patrol personnel and informs the patrol personnel to go to a designated place; when an accident happens underground, the position of the inspection personnel is confirmed through the positioning point diagram, the evacuation route is quickly established, and the evacuation route is sent to the AR equipment of the inspection personnel.
The non-invasive brain-computer interface is arranged on the inner side of the safety helmet, and the circuit is arranged in the interlayer.
The single chip microcomputer controller, the gas concentration sensor and the alarm are all located in the safety helmet interlayer, and the forehead sensor is located on the inner side of the safety helmet and close to the forehead.
An application method of an intelligent fully-mechanized coal mining face inspection system based on a configurable mine safety helmet.
S1: the remote virtual conference function is divided into three places, namely a ground dispatching center gateway centralized control center and positions of inspection personnel when a conference is carried out; in a video conference, an inspector collects depth image data through a depth camera on AR glasses under complex working conditions, the depth image data and the image data collected by a three-dimensional laser radar are fused, the image data are arranged by a gateway centralized control center and are transmitted to a ground dispatching center server for processing, the depth camera is calibrated at a server side, point cloud is spliced and optimized to obtain a point cloud model, and meanwhile, the server transmits a three-dimensional scene to the inspection end and the gateway centralized control center through a wireless network; at the inspection end, two inspection personnel cooperate with each other, wherein one inspection personnel and the other inspection personnel scan a scene into a virtual scene simultaneously by wearing a configurable mine safety cap, a virtual model is generated by processing, and the scanned inspection personnel are positioned in a three-dimensional scene for real-time explanation; a configurable mine safety helmet is worn by workers in a crossheading centralized control center, skeleton point data of the workers are collected through a Kinect camera, a virtual human model is built, the workers on the ground wear head display VR equipment at the same time, a corresponding virtual human model is built in a virtual scene through the use of face, posture and gesture tracking technologies, the virtual human model achieves simple interaction of movement and expression change, the skeleton point data of the workers are collected through the Kinect camera, and a 3D space is shared through VR simulation.
S2: intelligent monitoring and control functions: data acquired by sensors of underground fully-mechanized mining equipment and intelligent fully-mechanized mining working face patrol inspection systems based on configurable mine safety helmets are transmitted to a server, digital twin bodies of all parts in the underground are built, information is updated in real time, the digital twin fully-mechanized mining working face located at the cloud end simulates and previews the running conditions of the entity fully-mechanized mining equipment in advance, the optimal working condition is judged, and predictive cutting is achieved.
S3: the positioning function is that each inspection worker is positioned through an inertial sensor of AR glasses, data information is transmitted to a crossheading centralized control center to establish a positioning point diagram, points on the positioning point represent the current positions of the inspection workers, and therefore a positioning system is established, and when the crossheading centralized control center needs to contact the inspection workers, communication calling is carried out through clicking the points; when the inspection personnel is required to determine a certain position in the well, the positioning system is used for quickly determining the nearby inspection personnel and informing the inspection personnel to go to a specified place; when an accident happens underground, the positions of the inspection personnel are confirmed through the positioning point diagram, the evacuation route is quickly established and sent to the AR equipment of the inspection personnel, so that the inspection personnel can be quickly evacuated, and the safety of the personnel is ensured.
Compared with the prior art, the invention has the following beneficial effects:
1. the depth sensor of the AR glasses is matched with the three-dimensional laser radar for use, so that the change condition of the coal bed and the real-time pose information of the fully mechanized mining equipment can be accurately scanned, convenience is provided for constructing an underground virtual scene and realizing real-time monitoring, and data support is provided for realizing the automatic cutting of the fully mechanized mining equipment in the follow-up process.
2. The invention is connected with the positioning point diagram system, which is beneficial to scheduling the underground production work, improves the working efficiency and saves the labor cost; and the positions of field personnel can be accurately mastered, and in case of emergency, rescue personnel can judge the position conditions of related personnel according to positioning, and take timely rescue measures to improve the efficiency of rescue work.
3. The invention realizes the function of simulating the virtual meeting on the spot by adding the AR glasses, can simulate the actual situation on the spot, realizes multi-dimensional and multi-level information interaction with other workers, and improves the underground working efficiency.
4. The brain-computer interface and the forehead sensor are added in the invention, so that the physiological condition of the polling personnel can be better detected, various data information of the personnel is provided, and the physical condition of the polling personnel in the underground severe environment is monitored in real time. The inspection personnel can inspect the inspection with higher quality, and the people-oriented idea is embodied.
5. The method has a good effect when measuring the relative pose relation among equipment such as a plurality of hydraulic supports, provides a new angle for constructing and monitoring an underground three-dimensional model through data information acquired by a depth camera of the AR glasses and the three-dimensional laser radar, and makes up important information missing from a contact sensor arranged in an intelligent mining physical system.
6. The invention provides a more convenient underground fully-mechanized mining equipment control method, and by means of a three-dimensional model established by collecting information, workers can control the operation of fully-mechanized mining equipment in a virtual scene, intervene in complex working conditions possibly met and realize control from virtual to real.
7. The invention provides a method for controlling fully mechanized mining equipment by using virtual figures, which uses three virtual persons to monitor a coal mining machine, a hydraulic support and a scraper conveyor respectively, and the virtual persons are managed by a neuron computer of a bionic human brain, so that the number of workers can be effectively reduced, the underground unmanned development is promoted, and the intelligent mining of coal mines is realized.
Drawings
FIG. 1 is a block diagram of a human-oriented configurable mine safety helmet incorporating a plurality of intelligent terminals;
FIG. 2 is a main functional diagram of a human-oriented configurable mine safety helmet incorporating multiple intelligent terminals provided by the present invention;
FIG. 3 is a structural design drawing of a human-oriented configurable mine safety helmet incorporating multiple intelligent terminals in accordance with the present invention;
FIG. 4 is a side view of a human-oriented configurable mine safety helmet incorporating a variety of intelligent terminals provided by the present invention;
fig. 5 is a top view of a human-oriented configurable mine safety helmet incorporating multiple intelligent terminals.
Detailed Description
The invention aims to provide an intelligent fully-mechanized mining working face inspection system based on a configurable mine safety helmet, which is used for realizing the inspection of underground workers, reducing the operation flow of the workers and promoting the intelligent development of the fully-mechanized mining working face through the coordination and coordination among the systems, controlling the operation of a coal mining machine and a hydraulic support through AR glasses which are integrated with various intelligent terminals and are configured on the configurable mine safety helmet, acquiring data through a depth sensor of the AR glasses, acquiring depth image data, and transmitting the acquired data to a crossheading centralized control center for the workers to analyze and establish a three-dimensional model; positioning the inspection personnel through an IMU (inertial sensor) of the AR glasses to generate an underground positioning point diagram; through brain machine interface, detect the mental concentrated state of patrolling and examining personnel, avoid some that probably take place in the pit because the dangerous condition that the personnel operating condition of patrolling and examining is not concentrated from this to cooperate forehead sensor to detect the blood oxygen concentration of patrolling and examining personnel, body temperature and rhythm of the heart, detect the health status of patrolling and examining personnel: scanning the pose information of the hydraulic support and the coal mining machine in the routing inspection process by using a three-dimensional laser radar, obtaining coordinate data of the hydraulic support and the coal mining machine by using the radar as an original point, continuously scanning pulse laser to obtain data of all target points on the fully mechanized mining equipment, and obtaining an accurate three-dimensional image after imaging processing is carried out on the data; and monitoring the gas concentration in real time through a sensor, and monitoring the underground surrounding environment.
The system can realize a remote virtual conference function besides cooperatively detecting various data, when an inspector faces underground complex working conditions and cannot judge by himself, the complex working conditions are scanned by a configurable mine safety helmet, the complex working conditions are transmitted to a computer of a crossheading centralized control center, the complex working conditions are transmitted to a ground dispatching center server after being sorted, a relevant model is built, and workers at three places wear relevant equipment to enter a virtual scene to discuss and analyze the working conditions.
An intelligent comprehensive mining working face inspection system based on a configurable mine safety helmet is composed of an inspection end, a ground scheduling center and a crossheading centralized control center, and is based on the configurable mine safety helmet.
The method comprises the following steps that a worker positioned in a ground dispatching center monitors a working site by wearing HTC (high-temperature live video) head display VR (virtual reality) equipment, observes the running condition of fully mechanized mining equipment, the underground personnel positioning condition and the physiological conditions of workers positioned in other two places in a virtual scene, tracks a human body through a Kinect camera when a virtual conference is carried out, and creates a corresponding virtual character; the ground dispatching center plays a master control role in the system, and plays a role in monitoring the site and making a decision when underground production operation is carried out.
The mine safety helmet is characterized in that workers in the crossheading centralized control center wear configurable mine safety helmets to perform monitoring work, meanwhile, a Kinect camera is used for tracking a human body to generate three virtual persons, and the virtual persons manage the equipment to be monitored and controlled of each part respectively through a neuron computer of a bionic human brain. The virtual human is generated by three-dimensional modeling and bound with monitoring personnel, and if the system finds abnormality during the operation of the equipment, a worker simply controls any virtual image by switching management authorities. And the working personnel of the crossheading centralized control center use an explosion-proof computer to sort and gather the information collected by the fully mechanized coal mining face, so that the functions of coordinating the working face, connecting the dispatching center and the working personnel of the inspection end are achieved.
The underground working face inspection worker wears the configurable mine safety helmet to conduct inspection work, the functions of data acquisition, positioning, environment monitoring and data transmission are achieved through the safety helmet in the working process, and remote virtual conferences with a ground scheduling center and a gateway centralized control center are conducted when complex working conditions which are difficult to solve occur underground.
The three parts jointly form an intelligent fully-mechanized coal mining face system based on the configurable mine safety helmet, and are coordinated and matched.
The invention provides a configurable mine safety helmet integrating multiple intelligent terminals, which comprises a safety helmet main body 1, AR glasses 2, a non-invasive computer interface 3, a three-dimensional laser radar 4, a single chip microcomputer controller 5, a gas concentration sensor 6, a forehead sensor 7, an alarm 8 and a power supply module 9.
The safety helmet main body 1 is of a double-layer structure, and the middle part of the safety helmet main body is a buffer layer, so that the functions of sealing, explosion prevention and shock absorption are achieved.
The AR glasses 2 are connected and fixed with the safety helmet main body 1 through hooks in four directions, namely front, back, left and right, by adopting HoloLens2, and are connected with each base station for data transmission; in order to meet the underground explosion-proof requirement, the main body of the AR glasses 2 is wrapped by engineering plastics, and the lens parts are made of transparent materials, so that the normal work of the AR glasses 2 is guaranteed. And set up wireless base station in the pit, solve the data transmission problem to can be used for as the communication tool between patrolling and examining personnel and the ground crossheading centralized control center, can carry out data acquisition, carry out long-range video conference, monitor control to the fully mechanized mining equipment. Because the dust is more in the pit, in order to prevent that the dust from sheltering from the glasses lens, guarantee that the personnel of patrolling and examining can have a better field of vision, set up AR equipment cleaning point in each cavern in the pit.
The three parts in the system jointly form an intelligent fully-mechanized mining face system based on the configurable mine safety helmet, and the system has the following three specific functions:
1. the remote virtual meeting function is divided into three places, namely a ground dispatching center, a crossheading centralized control center and the positions of patrol personnel when meeting is carried out. In a video conference, an inspector collects depth image data of a complex working condition through a depth camera on AR glasses, the depth image data is fused with image data collected by a three-dimensional laser radar, the image data is sorted by a gateway centralized control center and is transmitted to a ground dispatching center server for processing, the depth camera is calibrated at a server side, point clouds are spliced and optimized to obtain a point cloud model, and meanwhile, a three-dimensional scene is transmitted to the inspection end and the gateway centralized control center through a wireless network by the server. At the inspection end, two inspection personnel can cooperate, wherein one inspection personnel and the other inspection personnel scan a scene into a virtual scene simultaneously by wearing a configurable mine safety cap, a virtual model is generated by processing, and the scanned inspection personnel are positioned in a three-dimensional scene for real-time explanation; the mine safety helmet capable of being configured is worn by workers in the crossheading centralized control center, the bone point data of the workers are collected through the Kinect equipment, a virtual person model is built, the workers on the ground wear HTC (high-speed computer) visual head display VR (virtual reality) equipment at the same time, a corresponding virtual person model is built in a virtual scene through the face, posture and gesture tracking technology, the virtual person can realize simple interaction of action, expression change and the like, the bone point data of the workers are collected through the Kinect equipment, a presence feeling is brought to people through a 3D (three-dimensional) shared space simulated by VR, the conference efficiency can be improved through rich body language, and convenience is brought to mine work.
2. Intelligent monitoring, control function: the sensors of the underground fully-mechanized mining equipment and the data acquired by the equipment in the system are transmitted to the ground server, digital twin bodies of all parts in the underground are built, information is updated in real time, and the running condition of the entity fully-mechanized mining equipment can be simulated and previewed in advance by the digital twin mining working face located at the cloud end, so that the optimal working condition is judged, and cutting prediction is realized.
AR equipment on the configurable mine safety helmet worn by the inspection personnel can read pose information fused with vision and the multiple sensors from the server, the inspection personnel are connected to the digital twin fully mechanized mining face, the inspection personnel can see real-time data changes of the fully mechanized mining equipment in AR glasses, and AR visual monitoring of the working face is completed; when the working personnel in the gateway centralized control center wear the configurable mine safety helmet, simultaneously access the digital twin platform with the patrol personnel, share data, and can control the digital fully-mechanized mining equipment in a virtual scene through the self-contained UGUI (user generated interface), natural gestures, voice and staring AR interaction functions of AR equipment when facing the working condition needing to be processed, so that the rocker arm and the hydraulic support of the coal mining machine are uniformly regulated and controlled; meanwhile, a plurality of virtual images can be generated in a virtual scene, and the virtual human respectively manages the equipment of each part needing to be monitored and controlled through the neuron computer management of the bionic human brain. The virtual human is generated by three-dimensional modeling and bound with monitoring personnel, and if the system finds abnormality during the operation of the equipment, a worker simply controls any virtual image by switching management authorities.
Each patrol inspector carries out non-contact pose measurement on equipment in the area where the production site intelligent patrol inspector belongs through a visible light camera and a depth sensor which are arranged in AR glasses, and acquires sensing information; and secondly, the information of the fully mechanized mining equipment can be observed and judged through human eyes, and a control instruction is sent to the fully mechanized mining equipment through various interaction modes.
Meanwhile, point cloud data acquired through the AR glasses and the three-dimensional laser radar are processed by the centralized control center, uploaded to the dispatching center server, processed by the server to form a three-dimensional model, and then transmitted back to the AR equipment terminal, the equipment type and model are identified through the AR equipment, virtual data are superposed on real equipment, virtual and real data are fused, accuracy of data acquisition is improved, and the method has good help for accurately judging position and posture information of fully-mechanized equipment.
3. The positioning function is that each inspection person is positioned through an IMU (inertial measurement Unit) of the AR glasses, data information is transmitted to the crossheading centralized control center to establish a positioning point diagram, points on the point diagram represent the current positions of the inspection persons, and therefore a positioning system is established, and when the crossheading centralized control center needs to contact the inspection persons, communication calling can be carried out through clicking the points; when the inspection personnel is required to determine a certain position in the well, the nearby inspection personnel can be quickly determined through the positioning system and informed to go to a specified place; when an accident happens underground, the positions of the inspectors are confirmed through the positioning point diagram, the evacuation route is quickly established and sent to the AR equipment of the inspectors, so that the inspectors can be quickly evacuated, and the safety of the inspectors is ensured.
The realization of the virtual conference function is realized, wherein the gesture recognition of the workers at the gateway centralized control center and the ground is completed through a Kinect camera, then the skeleton gestures and the whole body nodes of the workers are predicted by utilizing an algorithm, and the angle of the skeleton joint is calculated by utilizing a reverse motion algorithm. The gesture recognition is based on a customized camera of a head display, the conference software adopts zoom software, and the live broadcast scheme is based on OBS.
Non-invasive brain-computer interface 3 sets up in the safety helmet inboard, the intermediate layer is arranged in to the circuit, when patrolling and examining personnel wear the safety helmet, can collect patrolling and examining personnel's brain electrical signal, through surveying, the record, enlarge, the nerve impulse that transmission motor neuron provided, the forehead sensor that cooperation safety helmet forehead head position set up, whether detect the spirit of patrolling and examining the people at present and concentrate, detect staff's blood oxygen concentration, body temperature and rhythm of the heart, detect patrolling and examining personnel's health, whether guarantee to patrol and examine personnel's spirit is not concentrated with this, whether health status is good, help and protect personnel's work safety in the pit.
The three-dimensional LiDAR 4 is selected from RS-LiDAR-16 and is placed at the top of a safety helmet, an interface is reserved on the safety helmet, the safety helmet can be used for replacing equipment at any time, and the equipment generally records the time when each pulse comes and goes to an object or a position as a specific point representing longitude, latitude and height on a three-dimensional axis. The scanner uses this information along with a three-dimensional visualization program to generate a three-dimensional image. The more pulses that are emitted, the denser and more detailed the image. The point cloud picture obtained by scanning the three-dimensional radar is transmitted to the dispatching center, the three-dimensional reconstruction is carried out on the working condition through the processing of the dispatching center server, and the three-dimensional radar is high in precision and rich in hierarchical details. The method has important significance for confirming the pose of the fully mechanized mining equipment in the pit.
The single chip microcomputer controller 5, the gas concentration sensor 6 and the alarm 8 are all positioned in the interlayer of the safety helmet, and the forehead sensor 7 is positioned at the inner side of the safety helmet and close to the forehead; the single chip microcomputer controller 4 is connected with the gas concentration sensor 5 and the alarm 7, when data measured by the sensors exceed a safety value, the data are transmitted back to the crossheading centralized control center through the AR glasses 2, an alarm is sounded to remind an inspector, and an evacuation route is generated in the AR glasses 2 of the inspector by a crossheading centralized control center worker through a positioning point diagram to help the inspector to keep away from a dangerous area. The forehead sensor 6 and the non-invasive brain-computer interface 3 access the co-measured data to the AR glasses 2 for monitoring analysis.
The power module 9 is positioned at the rear side of the safety helmet and provides energy support for all parts.
The electronic devices except the AR glasses 2, the brain-computer interface 3 and the three-dimensional laser radar 4 are all arranged in the middle interlayer of the safety helmet, and the circuit is wrapped by a buffer, so that the sealing performance and the safety performance are improved.
The present invention is described herein in relation to various components, the above description of the specific embodiments is only for assisting understanding of the method of the present invention and the core idea thereof, the described implementation method is only a part of the embodiments of the present invention, and is only illustrative, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the protection scope of the present invention.
Claims (8)
1. The utility model provides an intelligence is combined and is adopted working face system of patrolling and examining based on configurable mine safety helmet which characterized in that: comprises an inspection end, a centralized control center and a ground dispatching center,
the inspection end comprises a configurable mine safety helmet worn by inspection workers on the underground working surface;
the centralized control center comprises a configurable mine safety helmet, an explosion-proof computer and a Kinect camera, wherein the Kinect camera is arranged obliquely behind workers in the centralized control center and used for tracking the human bodies of the workers to generate three virtual persons which are respectively a coal mining machine monitoring virtual person, a hydraulic support monitoring virtual person and a scraper conveyor monitoring virtual person; the virtual human respectively manages the monitored coal mining machine, the monitored hydraulic support and the monitored scraper conveyor through the neuron computer management of the bionic human brain; the virtual people are generated through three-dimensional modeling and bound with a centralized control center worker, and if the system finds abnormality when the equipment runs, a monitoring worker controls any one virtual person through switching management permissions; the centralized control center staff uses an explosion-proof computer to sort and gather the information collected by the fully mechanized mining face and connect the dispatching center and the patrol end staff;
the ground dispatching center comprises a head display VR device, a Kinect camera and a server, workers in the ground dispatching center monitor a working site through the head display VR device, observe the running state of the fully mechanized mining equipment in a virtual scene, position the underground personnel and the physiological state of the workers in other two places, track the human body through the Kinect camera when a virtual conference is carried out, and create corresponding virtual figures.
2. The intelligent fully mechanized coal mining face inspection system based on configurable mine safety caps of claim 1, wherein: the configurable mine safety helmet comprises a safety helmet main body (1), AR glasses (2), a three-dimensional laser radar (4), a single-chip microcomputer controller (5), a gas concentration sensor (6), a forehead sensor (7), an alarm (8) and a power module (9) are integrated on the safety helmet main body (1),
the AR glasses (2) are in signal connection with a wireless base station arranged underground for data transmission;
the three-dimensional laser radar (4) is placed at the top of the safety helmet and used for collecting surrounding three-dimensional information;
the single chip microcomputer controller (4) is connected with the gas concentration sensor (5) and the alarm (7), when the data measured by the gas concentration sensor (5) exceed a safety value, the data are transmitted back to the centralized control center through the AR glasses (2), the alarm (7) sounds to remind inspection personnel, and the centralized control center staff generate an evacuation route in the AR glasses (2);
the forehead sensor (6) detects the blood oxygen concentration, the body temperature and the heart rate of a worker, and the measured data are connected into the AR glasses (2) for monitoring and analysis;
the power module (9) provides energy for other structures.
3. The intelligent fully mechanized coal mining face inspection system based on the configurable mine safety helmet as recited in claim 2, wherein: the safety helmet is characterized in that a non-invasive brain-computer interface (3) is integrated on the safety helmet main body (1), the non-invasive brain-computer interface (3) collects electroencephalogram signals of inspection personnel, and detects whether the spirit of the current inspection personnel is concentrated or not by detecting, recording, amplifying and transmitting nerve impulses issued by motor neurons and matching with a forehead sensor arranged on the forehead of the safety helmet, and measured data are connected into AR glasses (2).
4. The intelligent fully mechanized mining face inspection system based on the configurable mine safety helmet of claim 3, wherein: the depth camera on the AR glasses (2) collects depth image data, the depth image data is fused with image data collected by the three-dimensional laser radar (4), the depth camera is calibrated and the point cloud is spliced and optimized at the ground dispatching center to obtain a point cloud model, and meanwhile, the three-dimensional scene is transmitted to the AR glasses (2) and the centralized control center through a wireless network.
5. The intelligent comprehensive mining working face inspection system based on the configurable mine safety helmet as claimed in claim 4, wherein the AR glasses (2) are integrated with inertial sensors, the inertial sensors perform real-time positioning, positioning data information is transmitted to the centralized control center to establish a positioning point diagram, points on the positioning point diagram represent the current positions of the inspection personnel, and therefore the positioning system is established, and when the centralized control center needs to contact the inspection personnel, communication calling is performed through clicking the points; when the inspection personnel is required to determine a certain position in the well, the positioning system is used for quickly determining the nearby inspection personnel and informing the inspection personnel to go to a specified place; when an accident happens underground, the position of the inspection personnel is confirmed through the positioning point diagram, the evacuation route is quickly established, and the evacuation route is sent to the AR equipment of the inspection personnel.
6. The intelligent fully mechanized coal mining face inspection system based on the configurable mine safety helmet as recited in claim 3, wherein: the non-invasive brain-computer interface (3) is arranged on the inner side of the safety helmet, and the circuit is arranged in the interlayer.
7. The intelligent fully mechanized mining face inspection system based on the configurable mine safety helmet of claim 2, wherein: the single chip microcomputer controller (5), the gas concentration sensor (6) and the alarm (8) are all located in an interlayer of the safety helmet, and the forehead sensor (7) is located on the inner side of the safety helmet and close to the forehead.
8. The use method of the intelligent fully mechanized mining face inspection system based on the configurable mine safety helmet as claimed in claim 5, characterized in that:
s1: the remote virtual conference function is divided into three places, namely a ground dispatching center crossheading centralized control center and positions of patrol personnel when a conference is carried out; in a video conference, an inspector collects depth image data of a complex working condition through a depth camera on AR glasses, the depth image data is fused with image data collected by a three-dimensional laser radar, the image data is sorted by a gateway centralized control center and is transmitted to a ground dispatching center server for processing, the depth camera is calibrated at a server side, point clouds are spliced and optimized to obtain a point cloud model, and meanwhile, a three-dimensional scene is transmitted to the inspection end and the gateway centralized control center through a wireless network by the server; at an inspection end, two inspection personnel cooperate with each other, wherein one inspection personnel and the other inspection personnel scan a scene into a virtual scene simultaneously by wearing a configurable mine safety cap, a virtual model is generated through processing, and the scanned inspection personnel are positioned in a three-dimensional scene for real-time explanation; the working personnel in the gateway centralized control center wear configurable mine safety helmets at the same time, the Kinect camera collects the bone point data of the working personnel, a virtual person model is built, the ground working personnel wear head display VR equipment at the same time, a corresponding virtual person model is built in a virtual scene by using face, posture and gesture tracking technology, the virtual person realizes simple interaction of action and expression change, the Kinect camera collects the bone point data of the working personnel, and the 3D space is shared through VR simulation;
s2: intelligent monitoring and control functions: the data acquired by sensors of underground fully mechanized mining equipment and intelligent fully mechanized mining working face inspection systems based on configurable mine safety caps are transmitted to a server, digital twin bodies of all parts in the underground are built, information is updated in real time, the digital twin fully mechanized mining working face located at the cloud end simulates and previews the running condition of the entity fully mechanized mining equipment in advance, the optimal working condition is judged, and cutting prediction is realized;
s3: the positioning function is that each inspection worker is positioned through an inertial sensor of AR glasses, data information is transmitted to a crossheading centralized control center to establish a positioning point diagram, points on the positioning point represent the current positions of the inspection workers, and therefore a positioning system is established, and when the crossheading centralized control center needs to contact the inspection workers, communication calling is carried out through clicking the points; when the inspection personnel is required to determine a certain position in the well, the positioning system is used for quickly determining the nearby inspection personnel and informing the inspection personnel to go to a specified place; when an accident happens underground, the positions of the inspection personnel are confirmed through the positioning point diagram, the evacuation route is quickly established and sent to the AR equipment of the inspection personnel, so that the inspection personnel can be quickly evacuated, and the safety of the personnel is ensured.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210673789.1A CN115251511A (en) | 2022-06-15 | 2022-06-15 | Intelligent fully-mechanized mining working face inspection system based on configurable mine safety helmet |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210673789.1A CN115251511A (en) | 2022-06-15 | 2022-06-15 | Intelligent fully-mechanized mining working face inspection system based on configurable mine safety helmet |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115251511A true CN115251511A (en) | 2022-11-01 |
Family
ID=83761455
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210673789.1A Pending CN115251511A (en) | 2022-06-15 | 2022-06-15 | Intelligent fully-mechanized mining working face inspection system based on configurable mine safety helmet |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115251511A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116293544A (en) * | 2023-05-10 | 2023-06-23 | 江苏恒械机械有限公司 | Portable lighting device for mine tunnel |
-
2022
- 2022-06-15 CN CN202210673789.1A patent/CN115251511A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116293544A (en) * | 2023-05-10 | 2023-06-23 | 江苏恒械机械有限公司 | Portable lighting device for mine tunnel |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110492607A (en) | A kind of intelligent substation condition monitoring system based on ubiquitous electric power Internet of Things | |
| CN111383132B (en) | Intelligent building site management system and management method suitable for epidemic situation | |
| CN107368083A (en) | A kind of crusing robot and crusing robot system | |
| CN108388194A (en) | Railway machine room intelligent robot inspection system and its method | |
| CN106527701B (en) | Intelligent wearable device applied to operation and maintenance of wind turbine generator | |
| CN208900095U (en) | A kind of downhole data acquisition terminal and processing system | |
| CN109139115A (en) | A kind of downhole data acquisition terminal, processing system and processing method | |
| CN208044380U (en) | A kind of railway machine room intelligent robot inspection system | |
| CN111920129A (en) | Intelligent safety helmet system | |
| CN116911823A (en) | AR (augmented reality) glasses-based substation inspection interaction method and system | |
| CN114863489B (en) | Virtual reality-based movable intelligent auxiliary inspection method and system for construction site | |
| CN213128247U (en) | Intelligent safety helmet system | |
| CN208607497U (en) | Patrolling railway intelligence wearable device based on high accuracy positioning board | |
| CN115251511A (en) | Intelligent fully-mechanized mining working face inspection system based on configurable mine safety helmet | |
| CN115562349A (en) | Inspection method and device for cooperative operation of unmanned aerial vehicle and ground inspection robot | |
| CN210639588U (en) | Intelligent safety protection wearing equipment for firefighter emergency rescue | |
| CN114298530A (en) | Man-made core HCPS man-machine co-fusion intelligent production system for fully mechanized coal mining face | |
| CN103230119A (en) | Multifunctional safety helmet and method for achieving visual intercom by using same | |
| CN116898172B (en) | Helmet device for detecting complex operation scene and application method | |
| CN111931748B (en) | Worker fatigue detection method suitable for storage battery production workshop | |
| CN219620716U (en) | Crown block hoisting operation visual monitoring system based on 5G communication technology | |
| CN211608330U (en) | Wearable device | |
| CN115690687A (en) | Safe wearing standard detection system based on deep learning technology | |
| CN212635749U (en) | Infrared body temperature screening robot system | |
| CN212628156U (en) | Belt conveyor post robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |