CN117181406B - Mine underground autonomous operation rock crusher and remote control method - Google Patents
Mine underground autonomous operation rock crusher and remote control method Download PDFInfo
- Publication number
- CN117181406B CN117181406B CN202311206975.5A CN202311206975A CN117181406B CN 117181406 B CN117181406 B CN 117181406B CN 202311206975 A CN202311206975 A CN 202311206975A CN 117181406 B CN117181406 B CN 117181406B
- Authority
- CN
- China
- Prior art keywords
- stone
- manipulator
- action
- camera
- drill rod
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 239000011435 rock Substances 0.000 title claims abstract description 18
- 239000004575 stone Substances 0.000 claims abstract description 90
- 230000006870 function Effects 0.000 claims abstract description 27
- 238000011065 in-situ storage Methods 0.000 claims abstract description 7
- 230000009471 action Effects 0.000 claims description 57
- 230000007246 mechanism Effects 0.000 claims description 27
- 239000011159 matrix material Substances 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 10
- 238000005516 engineering process Methods 0.000 claims description 9
- 238000011156 evaluation Methods 0.000 claims description 9
- 230000033001 locomotion Effects 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 9
- 230000036544 posture Effects 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 6
- 230000003068 static effect Effects 0.000 claims description 6
- 230000003190 augmentative effect Effects 0.000 claims description 5
- 230000002787 reinforcement Effects 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 3
- 230000007613 environmental effect Effects 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 3
- 239000003086 colorant Substances 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 abstract description 6
- 230000003116 impacting effect Effects 0.000 abstract description 3
- 230000006872 improvement Effects 0.000 description 8
- 238000001514 detection method Methods 0.000 description 7
- 239000010720 hydraulic oil Substances 0.000 description 4
- 238000005422 blasting Methods 0.000 description 2
- 239000007788 liquid Substances 0.000 description 2
- 239000003921 oil Substances 0.000 description 2
- 230000008054 signal transmission Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000009323 psychological health Effects 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
- 230000008685 targeting Effects 0.000 description 1
- 239000010878 waste rock Substances 0.000 description 1
Landscapes
- Manipulator (AREA)
Abstract
The invention discloses a mine underground autonomous operation rock crusher and a remote control method. The crusher comprises an underground mechanical arm, a drill rod arranged at the tail end of the mechanical arm and capable of impacting stones, a remote operation table and a display which are arranged on the ground surface, a first controller arranged underground, a second controller arranged on the ground surface and an in-situ operation table arranged underground. The invention can reduce the transmission quantity of actual data, reduce the delay of image transmission, improve the real-time performance and the operation efficiency of the displayed image, and simultaneously has the functions of automatic identification of large stones, automatic AI track planning and the like.
Description
Technical Field
The invention relates to a crusher and also relates to a remote control method of the crusher.
Background
In mines, rock and some waste rock produced after underground blasting need to be transported to the surface by underground trackless, drop shafts, rail transport systems, lifting systems. However, the large rock mass produced in blasting ore-falling can seriously affect the transportation, lifting and ore-drawing efficiency and even cause the blockage of the drop shaft. Therefore, it is often necessary to screen the mined rock with a grid before it is dumped into the drop shaft by forklift or truck. Rock with qualified granularity directly falls into the drop shaft from the grid screen, and large blocks on the screen are crushed into qualified granularity by a hammer crusher and then fall into the drop shaft.
Hammer crushers need to be manually operated underground, but the underground operation environment is poor, the noise is large when the stone crusher works, the physical and psychological health of miners is seriously affected, and a large number of operators are needed. Therefore, mine enterprises invest huge resources to introduce a remote control crusher, operation is implemented on the ground surface, and one person can control a plurality of devices. However, the actual effect is not obvious due to the influence of the detection signal transmission and the video signal transmission delay, and sometimes even the tact time is influenced.
In addition, the existing crusher has only a crushing function, cannot effectively crush stones in complex scenes of stacking and shielding the stones, cannot identify large stones, reasonably plans an operation path, and has low automation degree and low working efficiency.
Disclosure of Invention
The invention provides a mine underground autonomous operation rock crusher and a remote control method, which aims at: (1) The problem of poor signal delay and shooting angle during remote control is solved; and (2) solving the problem of low crushing efficiency.
The technical scheme of the invention is as follows:
the rock breaker comprises a mechanical arm installed underground, a drill rod installed at the tail end of the mechanical arm and capable of impacting the rock, a remote operation table and a display which are arranged on the ground surface, a first controller installed underground, a second controller installed on the ground surface and an in-situ operation table installed underground;
the remote operation table and the display are connected with the second controller; the in-situ console is connected with the first controller;
The first controller and the second controller are in communication connection through a network;
the first controller is used for controlling the action of a manipulator formed by a mechanical arm and an actuating mechanism arranged at the tail end of the mechanical arm, and the actuating mechanism comprises a drill rod and a plurality of clamps which are arranged beside the drill rod and can rotate relative to the drill rod; the clamp is used for clamping the stone block by matching with the drill rod and is also used for realizing stone moving operation, namely pushing the stone block sideways to move the stone block;
the crusher further comprises a camera and a three-dimensional laser scanner which are arranged beside the underground mechanical arm, the camera and the three-dimensional laser scanner rotate in real time along with the position of the tail end of the mechanical arm, and the shooting visual angles of the camera and the three-dimensional laser scanner are consistent.
As a further improvement of the mine downhole autonomous working rock breaker: the mechanical arm comprises a large arm which is arranged on the slewing base in a rotating connection mode and a small arm which is arranged at the tail end of the large arm in a rotating connection mode;
The actuating mechanism comprises a hammer head which is arranged at the tail end of the small arm in a rotating connection mode, and the drill rod is arranged at the front end of the hammer head;
the action of the mechanical arm and the actuating mechanism is realized by means of driving of a motor and a hydraulic cylinder.
As a further improvement of the mine downhole autonomous working rock breaker: the pipeline for providing hydraulic oil for the mechanical arm and the executing mechanism is provided with a pressure detection device and a pressure protection device, and the pressure detection device and the pressure protection device are used for collecting hydraulic pressure and stopping oil supply when the pressure is too high;
A hydraulic pressure station for providing hydraulic oil to arm and actuating mechanism is last to be equipped with temperature-detecting device and liquid level detection device.
As a further improvement of the mine downhole autonomous working rock breaker: the mechanical arm and the actuating mechanism are provided with angle sensors for detecting the rotation angle of each rotation joint, namely the rotation joint, and obtaining the three-dimensional coordinate of the tail end of the actuating mechanism according to the rotation angle in real time, so that the actuating mechanism is prevented from exceeding a preset action range.
The invention also discloses a remote control method based on the mine underground autonomous operation rock crusher, which comprises the following steps: after the images and the three-dimensional model are acquired by using the camera and the three-dimensional laser scanner and before the images and the three-dimensional model are displayed by the display, the background and the foreground are removed by the following modes:
For a target image shot by a camera, performing differential processing on the target image and an image shot in advance when the target image is not piled, removing a background part, identifying and removing a part of a manipulator with specified colors in the target image through a chroma key (chroma key) technology, and recovering a part shielded by the manipulator in the current target image according to the target image in other adjacent time before and after the manipulator moves;
For a target three-dimensional model obtained by a three-dimensional laser scanner, removing a background part by removing a part in a preset space range in the model, obtaining a model part corresponding to the manipulator in the target three-dimensional model according to a region occupied by the manipulator in a target image, which is identified by using a chroma key (chroma key) technology, removing the part from the target three-dimensional model, and recovering a part shielded by the manipulator in the current target three-dimensional model according to the target three-dimensional model in other adjacent time before and after the manipulator moves.
As a further improvement of the remote control method: and the second controller fuses an image obtained by removing the background and the foreground of the target image obtained by the camera with a manipulator three-dimensional model obtained by carrying out three-dimensional modeling again according to data detected by a sensor arranged on the manipulator, and the obtained Augmented Reality (AR) fused image is used as a site display image observed by a remote operator.
As a further improvement of the remote control method: defining stone blocks with the size larger than or equal to the mesh openings of the grid sieve as large stone blocks, and defining stone blocks with the size smaller than the mesh openings of the grid sieve as small stone blocks;
The second controller carries out the discernment of big stone based on the image that the camera took to suggestion remote operator breaks big stone, and the discernment mode is:
After removing the background and the foreground, sequentially processing the images shot by the camera according to time sequence, extracting feature descriptors from the images, tracking the moving condition of the extracted feature descriptors according to time sequence, dividing the images into a plurality of areas according to the moving condition, wherein the dividing principle is that the moving condition of the feature descriptors in each area is the same, and judging stone corresponding to the area larger than a preset value as the large stone.
As a further improvement of the remote control method: the action track is automatically planned for the manipulator to be further automatically executed by:
Step A1, defining a stacker state matrix D t, which is obtained by:
The method comprises the steps of obtaining a stacking model V t expressed by voxel at t moment through a three-dimensional laser scanner, and simultaneously creating a characteristic tracking matrix M t at t moment according to the identification process of a large stone, wherein the values of elements in M t represent the movement conditions in the positions in an image corresponding to the elements: the elements corresponding to the areas in the same movement condition use the same integer value which is more than or equal to 1, and 0 represents that the corresponding areas are static; querying values of positions facing to one side of the camera in V t from M t, matching the queried values and merging the values into V t to obtain a stacking state matrix D t; t is a time sequence number;
Step A2, defining an action group A, wherein the action group comprises a stone moving action, a clamping and grabbing action and a drill rod crushing action; the clamp comprises an upper knuckle and a lower knuckle arranged at the tail end of the upper knuckle; the stone moving action is represented by the starting and stopping postures of the lower knuckle, the clamping and gripping actions are represented by the starting and stopping postures of the upper knuckle and the lower knuckle, and the crushing action is represented by the starting and stopping postures of the drill rod;
Step A3, defining a manipulator state, wherein the manipulator state q t at the time t is represented by the pose of an upper knuckle, the pose of a lower knuckle and the pose of a drill rod;
Step A4, defining a feedback function r (t) =n (t) -n (t-1), where r (t) is a feedback function value at time t, and n (t) is the number of elements in the stacking state matrix D t with a non-stationary state in a moving condition at time t;
step A5, defining a strategy function, wherein the strategy function f is provided with:
at＝f(et,st|Dgoal)；
Wherein e t obeys the Gaussian distribution of the specified variance, D goal is a target stacker state matrix with all moving situations being static, s t is a state composed of q t and D t, a t is an action to be executed selected by a constructed function f according to Gaussian distribution parameters e t and aiming at D goal according to the current state s t, the action is represented by the starting and stopping postures of an upper knuckle, a lower knuckle and a drill rod, if the manipulator state q t+αat+∈t at the time t+1 calculated according to a t is not out of limit, the manipulator state q t+1＝qt+αat+∈t at the time t+1 is specified, wherein alpha is a tuning coefficient, and E t is an environmental noise value obeying the specified Gaussian distribution;
step A6, obtaining a strategy function f by adopting a deep reinforcement learning (SAC) algorithm: constructing a model comprising 5 neural networks, wherein 4 are value evaluation networks and 1 is action strategy network, and then repeatedly executing the following steps until the model converges: storing training data in an experience pool, sampling a batch of data from the experience pool, updating a value evaluation network, and updating an action strategy network by using the output of the value evaluation network; after the model converges, a strategy function f is obtained according to the action strategy network, and the function can select the optimal action under a given state.
As a further improvement of the remote control method: the training data is obtained in the following way: the action of the manipulator is controlled in a manual operation mode, and D t、at'、st、qt and r (t) at various moments in the operation process are recorded as training data through an angle sensor, a camera and a three-dimensional laser scanner, wherein a t' is the action adopted in the manual operation.
As a further improvement of the remote control method: the steps of the operation by the remote operation table are as follows:
Step B1, waiting for ore to pour onto a grid;
Step B2, operating the mechanical arm and the actuating mechanism at the tail end to shift the stone blocks so as to enable the small stone blocks to penetrate through the grid screen as much as possible;
Step B3, judging whether the stone stockpile is empty or not according to the image obtained by the camera, returning to the step B1 if the stone stockpile is empty, and otherwise, executing the step B4;
Step B4, carrying out the identification operation of the large stone blocks, and determining the positions of the large stone blocks according to the identification result;
step B5, operating the mechanical arm and the actuating mechanism at the tail end to impact crush and clamp crush the large stone blocks;
And B6, judging whether the stone stockpile is empty according to the image obtained by the camera, returning to the step B1 if the stone stockpile is empty, and otherwise, executing the step B2.
Compared with the prior art, the invention has the following positive effects:
(1) According to the invention, a camera and a three-dimensional laser scanner are simultaneously adopted in the pit to acquire a real-time image and a model in the pit, on one hand, a stable stone image with smaller data volume is obtained through the removal of the foreground and the background, on the other hand, a three-dimensional display model of a mechanical arm is quickly built on the earth surface based on the data of an angle sensor, and finally, the two images are fused through an Augmented Reality (AR) technology. Because most of the weather time stone is motionless, operators are more concerned about the pose of the mechanical arm, and through an Augmented Reality (AR) fusion technology, the real-time performance of the position of the mechanical arm in a picture is ensured to meet the operation requirement, the transmission quantity of actual data is greatly reduced, the delay of image transmission is reduced, the real-time performance of the displayed image is finally improved, and the operation efficiency is improved.
(2) According to the invention, the clamp is matched with the drill rod, so that not only can the stone be impacted, but also the stone can be moved and crushed, and the stone can pass through the grid screen more quickly.
(3) The invention also provides a large stone identification function, which can assist operators and systems to automatically acquire the coordinate positions of the large stone, thereby further improving the crushing efficiency;
(4) The invention also realizes the automatic generation of the optimal operation path based on a deep reinforcement learning (SAC) algorithm, and realizes the artificial intelligent autonomous operation.
Drawings
FIG. 1 is a schematic view of the structure of the crusher of the present invention;
FIG. 2 is a schematic view of the structure of the crusher according to the present invention;
FIG. 3 is a flow chart of the invention for identifying a large block of stone.
Detailed Description
The technical scheme of the invention is described in detail below with reference to the accompanying drawings:
Referring to fig. 1 and 2, a rock breaker for autonomous working downhole in mines for breaking up stones placed on a screen 7 comprises a mechanical arm mounted downhole, a drill rod 6 mounted at the end of the mechanical arm for impacting the stones, a remote console 2 and a display 1 provided on the ground surface, a first controller 11 mounted downhole, a second controller 12 mounted on the ground surface, and an in situ console 10 mounted downhole.
The remote operation table 2 and the display 1 are connected with the second controller 12; the in-situ console 10 is connected to a first controller 11.
The first controller 11 and the second controller 12 are in communication connection through a network.
The first controller 11 is used for controlling the action of a manipulator composed of a mechanical arm and an actuating mechanism arranged at the tail end of the mechanical arm, and the actuating mechanism comprises a drill rod 6 and a plurality of clamps 5 which are arranged beside the drill rod 6 and can rotate relative to the drill rod 6. The clamp 5 comprises an upper knuckle and a lower knuckle arranged at the tail end of the upper knuckle, and is used for clamping a stone block in cooperation with the drill rod 6 and also used for realizing stone moving operation, namely pushing the stone block sideways to move the stone block.
The crusher further comprises a camera 3 and a three-dimensional laser scanner 4 which are arranged beside the underground mechanical arm, the camera 3 and the three-dimensional laser scanner 4 rotate in real time along with the position of the tail end of the mechanical arm, and the shooting visual angles of the camera 3 and the three-dimensional laser scanner 4 are consistent.
Specifically, the mechanical arm comprises a large arm 9 which is installed on the slewing base in a rotating connection mode, and a small arm 8 which is installed at the tail end of the large arm 9 in a rotating connection mode.
The actuating mechanism comprises a hammer head which is arranged at the tail end of the small arm 8 in a rotating connection mode, and the drill rod 6 is arranged at the front end of the hammer head.
The clamps 5 are arranged in groups around the drill rod 6.
The action of the mechanical arm and the actuating mechanism is realized by means of driving of a motor and a hydraulic cylinder. In general, the rotary joints of the mechanical arm are driven by motors, and the drill rod 6, the telescopic rod for controlling the knuckle motion and the like are driven by hydraulic cylinders.
Furthermore, a pressure detection device and a pressure protection device are arranged on a pipeline for providing hydraulic oil for the mechanical arm and the actuating mechanism, and the pressure detection device and the pressure protection device are used for collecting hydraulic pressure and stopping oil supply when the pressure is too high. A hydraulic pressure station for providing hydraulic oil to arm and actuating mechanism is last to be equipped with temperature-detecting device and liquid level detection device.
Furthermore, the mechanical arm and the executing mechanism are provided with angle sensors for detecting the rotation angle of each rotation joint, namely the rotation joint, and obtaining the three-dimensional coordinate of the tail end of the executing mechanism according to the rotation angle in real time, so that the executing mechanism is prevented from exceeding a preset action range.
The forward motion solution and the inverse motion solution of the manipulator are realized through matrix operation. Positive kinematic solutions are those that calculate the position and pose of the robot tip by knowing the joint angle and the link length (distance between the revolute joints). The inverse kinematics solution is the inverse process of the positive kinematics, namely, the position and the gesture of the end actuating mechanism in a coordinate system are known, the angles of all joints of the robot are solved, and then driving components such as a motor and the like are controlled to move according to the angles. The matrix calculation process is the prior art and will not be described in detail.
The embodiment also provides a remote control method based on the crusher.
First, the control system processes an image in real time: after the image and the three-dimensional model are acquired by using the camera 3 and the three-dimensional laser scanner 4, the background and the foreground are removed in the following manner before being displayed by the display 1.
For the target image shot by the camera 3, the background part is removed by performing differential processing on the target image and the image shot in advance when the target image is not piled up, then the part of the manipulator with the specified color in the target image is identified and removed by a chroma key (chroma key) technology, and then the part blocked by the manipulator in the current target image is restored according to the target image in other adjacent time before and after the manipulator moves.
For the target three-dimensional model obtained by the three-dimensional laser scanner 4, removing the background part by removing the part in the preset space range in the model, obtaining the corresponding model part of the manipulator in the target three-dimensional model according to the area occupied by the manipulator in the target image identified by using a chroma key (chroma key) technology, removing the part from the target three-dimensional model, and recovering the part shielded by the manipulator in the current target three-dimensional model according to the target three-dimensional model in other adjacent time before and after the manipulator moves.
Further, the second controller 12 fuses an image obtained by removing the background and the foreground of the target image acquired by the camera 3 with a manipulator three-dimensional model obtained by re-performing three-dimensional modeling based on data detected by a sensor mounted on the manipulator, and uses the fused image as an on-site display image observed by a remote operator. Because most of the weather time stone is motionless, operators are more concerned about the pose of the mechanical arm, and through an Augmented Reality (AR) fusion technology, the real-time performance of the position of the mechanical arm in a picture is ensured to meet the operation requirement, the transmission quantity of actual data is greatly reduced, the delay of image transmission is reduced, the real-time performance of the displayed image is finally improved, and the operation efficiency is improved.
The stone having a size equal to or larger than the mesh of the mesh 7 is defined as a large stone, and the stone having a size smaller than the mesh of the mesh 7 is defined as a small stone.
The second controller 12 performs the recognition of the large stone block based on the image photographed by the camera 3 to prompt the remote operator to crush the large stone block in the following manner:
As shown in fig. 3, after removing the background and the foreground, the image shot by the camera 3 is sequentially processed according to time sequence, during processing, the feature descriptors of the image are extracted first, then the moving situation of the extracted feature descriptors along with time is tracked according to time sequence, the image is divided into a plurality of areas according to the moving situation, the dividing principle is that the moving situation of the feature descriptors in each area is the same, and the stone corresponding to the area larger than the preset value is determined as the big stone.
The crusher has two modes of intelligent automatic operation based on AI and automatic operation based on fixed programs besides local manual operation and remote manual operation.
The intelligent automatic operation manipulator track planning process based on AI is as follows:
Step A1, defining a stacker state matrix D t, which is obtained by:
The three-dimensional laser scanner 4 is used for obtaining a stacking model V t expressed by voxelization at the time t, and simultaneously, a characteristic tracking matrix M t at the time t is created according to the identification process of a large stone, wherein the values of elements in M t represent the movement conditions in the positions in the image corresponding to the elements: the elements corresponding to the areas in the same movement condition use the same integer value which is more than or equal to 1, and 0 represents that the corresponding areas are static; querying values of positions facing to the camera 3 in V t from M t, matching the queried values and merging the values into V t to obtain a stacking state matrix D t; t is the time sequence number.
Step A2, defining an action group A, wherein the action group comprises a stone moving action, a clamping and grabbing action and a breaking action of a drill rod 6; the stone-moving action is represented by the start-stop position of the lower knuckle, the gripping and gripping action is represented by the start-stop positions of the upper knuckle and the lower knuckle, and the breaking action is represented by the start-stop position of the drill rod 6.
And A3, defining a manipulator state, wherein the manipulator state q t at the time t is represented by the pose of the upper knuckle, the pose of the lower knuckle and the pose of the drill rod 6.
Step A4, defining a feedback function r (t) =n (t) -n (t-1), where r (t) is a feedback function value at time t, and n (t) is the number of elements in the t-time stacking state matrix D t that are in a non-stationary state in a moving condition.
Step A5, defining a strategy function, wherein the strategy function f is provided with:
at＝f(et,st|Dgoal)；
Wherein e t obeys the gaussian distribution of the specified variance, D goal is a target stacker state matrix with all moving conditions being static, s t is a state composed of q t and D t, a t is an action to be performed by a constructed function f according to a gaussian distribution parameter e t, targeting D goal, and selected according to the current state s t, the action being represented by the upper knuckle, the lower knuckle and the start-stop pose of the drill rod 6, and if the manipulator state q t+αat+∈t at time t+1 calculated according to a t is not overrun, the manipulator state q t+1＝qt+αat+∈t at time t+1 is specified, wherein α is a tuning coefficient, and e t is an environmental noise value obeying the specified gaussian distribution.
Step A6, obtaining a strategy function f by adopting a deep reinforcement learning (SAC) algorithm: constructing a model comprising 5 neural networks, wherein 4 are value evaluation networks and 1 is action strategy network, and then repeatedly executing the following steps until the model converges: storing training data in an experience pool, sampling a batch of data from the experience pool, updating a value evaluation network, and updating an action strategy network by using the output of the value evaluation network; after the model converges, a strategy function f is obtained according to the action strategy network, and the function can select the optimal action under a given state.
Further, the training data is obtained by the following steps: the action of the manipulator is controlled in a manual operation mode, and D t、at'、st、qt and r (t) at various moments in the operation process are recorded as training data through the angle sensor, the camera 3 and the three-dimensional laser scanner 4, wherein a t' is the action taken in the manual operation.
The automatic operation process based on the fixed program is as follows:
and B1, waiting for ore to pour onto the grid 7.
And step B2, operating the mechanical arm and the actuating mechanism at the tail end to shift the stone blocks so as to enable the small stone blocks to penetrate through the grid screen 7 as much as possible.
And B3, judging whether the stone stockpile is empty according to the image obtained by the camera 3, returning to the step B1 if the stone stockpile is empty, and otherwise, executing the step B4.
And B4, carrying out the identification operation of the large stone blocks, and determining the positions of the large stone blocks according to the identification result.
And B5, operating the mechanical arm and the actuating mechanism at the tail end to impact crush and clamp crush the massive stone blocks.
And B6, judging whether the stone stockpile is empty according to the image obtained by the camera 3, returning to the step B1 if the stone stockpile is empty, and otherwise, executing the step B2.
Claims (6)
1. The utility model provides a remote control method based on mine is independently operated rock breaker in pit, mine is independently operated rock breaker in pit is used for broken stone of placing on grid (7), and it includes installs the arm in the pit, installs at arm end can strike drill rod (6) and remote operation platform (2) and display (1) of earth's surface to the stone, its characterized in that: the mine underground autonomous operation rock crusher further comprises a first controller (11) installed underground, a second controller (12) installed on the surface of the earth and an in-situ operation table (10) installed underground;
The remote operation table (2) and the display (1) are connected with the second controller (12); the in-situ console (10) is connected with the first controller (11);
the first controller (11) and the second controller (12) are in communication connection through a network; the first controller (11) is used for controlling the action of a manipulator formed by a mechanical arm and an actuating mechanism arranged at the tail end of the mechanical arm, the actuating mechanism comprises a drill rod (6) and a plurality of clamps (5) which are arranged beside the drill rod (6) and can rotate relative to the drill rod (6); the clamp (5) is used for clamping the stone block by matching with the drill rod (6) and is also used for realizing stone moving operation, namely pushing the stone block sideways to move the stone block;
The crusher further comprises a camera (3) and a three-dimensional laser scanner (4) which are arranged beside the underground mechanical arm, wherein the camera (3) and the three-dimensional laser scanner (4) rotate in real time along with the position of the tail end of the mechanical arm, and the shooting visual angles of the camera (3) and the three-dimensional laser scanner (4) are consistent; the remote control method comprises the following steps:
After the images and the three-dimensional model are acquired by using the camera (3) and the three-dimensional laser scanner (4), and before the images and the three-dimensional model are displayed by the display (1), the background and the foreground are removed by the following modes: for a target image shot by a camera (3), performing differential processing on the target image and an image shot in advance when the target image is not piled, removing a background part, identifying and removing a part of a manipulator with specified colors in the target image through a chroma key (chroma key) technology, and recovering a part shielded by the manipulator in the current target image according to the target image in other adjacent time before and after the manipulator moves;
For a target three-dimensional model obtained by a three-dimensional laser scanner (4), removing a background part by removing a part in a preset space range in the model, obtaining a model part corresponding to the manipulator in the target three-dimensional model according to a region occupied by the manipulator in a target image, which is identified by using a chroma key (chroma key) technology, removing the part from the target three-dimensional model, and recovering a part shielded by the manipulator in the current target three-dimensional model according to the target three-dimensional model in other adjacent time before and after the manipulator moves.
2. The remote control method as claimed in claim 1, wherein: the second controller (12) uses an Augmented Reality (AR) fusion image obtained by fusing an image obtained by removing the background and the foreground of the target image obtained by the camera (3) with a manipulator three-dimensional model obtained by carrying out three-dimensional modeling again according to data detected by a sensor arranged on the manipulator as a site display image observed by a remote operator.
3. The remote control method as claimed in claim 1, wherein: defining stone blocks with the size larger than or equal to the mesh openings of the grid screen (7) as large stone blocks, and defining stone blocks with the size smaller than the mesh openings of the grid screen (7) as small stone blocks;
The second controller (12) carries out the discernment of big stone based on the image that camera (3) took to suggestion remote operator carries out the breakage to big stone, and the discernment mode is:
After removing the background and the foreground, sequentially processing the images shot by the camera (3) according to time sequence, extracting feature descriptors from the images, tracking the moving condition of the extracted feature descriptors according to time sequence, dividing the images into a plurality of areas according to the moving condition, wherein the dividing principle is that the moving condition of the feature descriptors in each area is the same, and judging stone corresponding to the area larger than a preset value as the large stone.
4. A remote control method as claimed in claim 3, characterized in that: the action track is automatically planned for the manipulator to be further automatically executed by:
Step A1, defining a stacker state matrix D t, which is obtained by:
The method comprises the steps of obtaining a stacking model V t expressed by voxel at t moment through a three-dimensional laser scanner, and simultaneously creating a characteristic tracking matrix M t at t moment according to the identification process of a large stone, wherein the values of elements in M t represent the movement conditions in the positions in an image corresponding to the elements: the elements corresponding to the areas in the same movement condition use the same integer value which is more than or equal to 1, and 0 represents that the corresponding areas are static; querying values of positions facing to one side of the camera in V t from M t, matching the queried values and merging the values into V t to obtain a stacking state matrix D t; t is a time sequence number;
Step A2, defining an action group A, wherein the action group comprises a stone moving action, a clamping and grabbing action and a drill rod crushing action; the clamp comprises an upper knuckle and a lower knuckle arranged at the tail end of the upper knuckle; the stone moving action is represented by the starting and stopping postures of the lower knuckle, the clamping and gripping actions are represented by the starting and stopping postures of the upper knuckle and the lower knuckle, and the crushing action is represented by the starting and stopping postures of the drill rod;
Step A3, defining a manipulator state, wherein the manipulator state q t at the time t is represented by the pose of an upper knuckle, the pose of a lower knuckle and the pose of a drill rod;
Step A4, defining a feedback function r (t) =n (t) -n (t-1), where r (t) is a feedback function value at time t, and n (t) is the number of elements in the stacking state matrix D t with a non-stationary state in a moving condition at time t;
step A5, defining a strategy function, wherein the strategy function f is provided with:
at＝f(et,st|Dgoal)；
Wherein e t obeys the Gaussian distribution of the specified variance, D goal is a target stacker state matrix with all moving situations being static, s t is a state composed of q t and D t, a t is an action to be executed selected by a constructed function f according to Gaussian distribution parameters e t and aiming at D goal according to the current state s t, the action is represented by the starting and stopping postures of an upper knuckle, a lower knuckle and a drill rod, if the manipulator state q t+αat+∈t at the time t+1 calculated according to a t is not out of limit, the manipulator state q t+1＝qt+αat+∈t at the time t+1 is specified, wherein alpha is a tuning coefficient, and E t is an environmental noise value obeying the specified Gaussian distribution;
step A6, obtaining a strategy function f by adopting a deep reinforcement learning (SAC) algorithm: constructing a model comprising 5 neural networks, wherein 4 are value evaluation networks and 1 is action strategy network, and then repeatedly executing the following steps until the model converges: storing training data in an experience pool, sampling a batch of data from the experience pool, updating a value evaluation network, and updating an action strategy network by using the output of the value evaluation network; after the model converges, a strategy function f is obtained according to the action strategy network, and the function can select the optimal action under a given state.
5. The remote control method as claimed in claim 4, wherein: the training data is obtained in the following way: the action of the manipulator is controlled in a manual operation mode, and D t、at'、st、qt and r (t) at various moments in the operation process are recorded as training data through an angle sensor, a camera (3) and a three-dimensional laser scanner (4), wherein a t' is the action adopted in the manual operation.
6. A remote control method as claimed in claim 3, characterized in that: the steps of the operation by the remote operation table (2) are as follows:
step B1, waiting for ore to pour onto a grid screen (7);
step B2, operating the mechanical arm and the actuating mechanism at the tail end to shift the stone blocks so as to enable the small stone blocks to penetrate through the grid screen (7) as much as possible;
Step B3, judging whether the stone stockpile is empty or not according to the image obtained by the camera (3), returning to the step B1 if the stone stockpile is empty, otherwise, executing the step B4;
Step B4, carrying out the identification operation of the large stone blocks, and determining the positions of the large stone blocks according to the identification result;
step B5, operating the mechanical arm and the actuating mechanism at the tail end to impact crush and clamp crush the large stone blocks;
and B6, judging whether the stone stockpile is empty or not according to the image obtained by the camera (3), returning to the step B1 if the stone stockpile is empty, and otherwise, executing the step B2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311206975.5A CN117181406B (en) | 2023-09-18 | 2023-09-18 | Mine underground autonomous operation rock crusher and remote control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311206975.5A CN117181406B (en) | 2023-09-18 | 2023-09-18 | Mine underground autonomous operation rock crusher and remote control method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117181406A CN117181406A (en) | 2023-12-08 |
| CN117181406B true CN117181406B (en) | 2024-05-17 |
Family
ID=88983231
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311206975.5A Active CN117181406B (en) | 2023-09-18 | 2023-09-18 | Mine underground autonomous operation rock crusher and remote control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117181406B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH1018299A (en) * | 1996-07-05 | 1998-01-20 | Daiho Constr Co Ltd | Excavating equipment in caisson |
| JP2019174287A (en) * | 2018-03-28 | 2019-10-10 | 太平洋セメント株式会社 | Object recognition device, method, program, and object removal system |
| CN110499802A (en) * | 2019-07-17 | 2019-11-26 | 爱克斯维智能科技(苏州)有限公司 | A kind of image-recognizing method and equipment for excavator |
| CN115674192A (en) * | 2022-10-09 | 2023-02-03 | 中国矿业大学 | Coal mine underground mechanical arm grabbing control method based on visual positioning |
| CN116237135A (en) * | 2023-03-06 | 2023-06-09 | 中钢集团马鞍山矿山研究总院股份有限公司 | AI-based secondary crushing and plug cleaning robot |
-
2023
- 2023-09-18 CN CN202311206975.5A patent/CN117181406B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH1018299A (en) * | 1996-07-05 | 1998-01-20 | Daiho Constr Co Ltd | Excavating equipment in caisson |
| JP2019174287A (en) * | 2018-03-28 | 2019-10-10 | 太平洋セメント株式会社 | Object recognition device, method, program, and object removal system |
| CN110499802A (en) * | 2019-07-17 | 2019-11-26 | 爱克斯维智能科技(苏州)有限公司 | A kind of image-recognizing method and equipment for excavator |
| CN115674192A (en) * | 2022-10-09 | 2023-02-03 | 中国矿业大学 | Coal mine underground mechanical arm grabbing control method based on visual positioning |
| CN116237135A (en) * | 2023-03-06 | 2023-06-09 | 中钢集团马鞍山矿山研究总院股份有限公司 | AI-based secondary crushing and plug cleaning robot |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117181406A (en) | 2023-12-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2018425665B2 (en) | Method, apparatus and system for monitoring a condition associated with operating heavy equipment such as a mining shovel or excavator | |
| US11401684B2 (en) | Perception-based alignment system and method for a loading machine | |
| JPWO2007138987A1 (en) | Work robot | |
| Eraliev et al. | Sensing, perception, decision, planning and action of autonomous excavators | |
| CN113982690B (en) | Control method, device, system and tunneling, anchoring and protecting integrated machine based on anchoring operation | |
| Shariati et al. | Towards autonomous mining via intelligent excavators | |
| JP7064664B2 (en) | Target recognition device, method and program, and target removal system | |
| CN117181406B (en) | Mine underground autonomous operation rock crusher and remote control method | |
| Lampinen et al. | Autonomous robotic rock breaking using a real‐time 3D visual perception system | |
| CN113846715B (en) | Land leveler control method and device and land leveler | |
| CN113805479B (en) | Automatic operation control method and control system for punching and grabbing machine | |
| JPH09195321A (en) | Travelling automatic crushing plant and automatic hydraulic shovel | |
| JP7117273B2 (en) | Fragmented piece detection method, fragmented fragmentation method, fragmented fragment detection device and fragmented fragmentation device | |
| Winstanley et al. | Field and service applications-Dragline automation-A dedade of development-Shared Autonomy for Improving Mining Equipment Productivity | |
| Zhao et al. | AES: Autonomous excavator system for real-world and hazardous environments | |
| US20240134388A1 (en) | Apparatus and method for positioning equipment relative to a drill hole | |
| CN110722536A (en) | Electric-driving movable type tunnel excavation dredging industrial robot device and method | |
| Sarata et al. | Field test of autonomous loading operation by wheel loader | |
| CN115506471A (en) | Dredging robot and dredging method | |
| CN110980061A (en) | Novel intelligence major possession refuse treatment system | |
| CN113677870A (en) | Method and device for enabling an automatic operation of a material extraction installation at a working face of a material extraction site | |
| Corke et al. | Robotics for the mining industry | |
| CN112275437A (en) | Mining stone crusher operating system | |
| CN111411654A (en) | Automatic loading method for underground scraper | |
| Singh | State of the Art in Automation of Earthmoving, 2002 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |