CN115727851A - Pose detection system and method for underground coal mine tunneling equipment - Google Patents
Pose detection system and method for underground coal mine tunneling equipment Download PDFInfo
- Publication number
- CN115727851A CN115727851A CN202211438343.7A CN202211438343A CN115727851A CN 115727851 A CN115727851 A CN 115727851A CN 202211438343 A CN202211438343 A CN 202211438343A CN 115727851 A CN115727851 A CN 115727851A
- Authority
- CN
- China
- Prior art keywords
- laser
- image
- tunneling equipment
- point
- pose
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses a position and posture detection system and method for underground coal mine tunneling equipment, and relates to the technical field of position and posture detection. The system comprises: the system comprises a three-laser direction indicator module, a data acquisition module and a pose resolving module. The three-laser direction indicator module is positioned in a rear roadway of the tunneling equipment to be tested and used for emitting combined laser comprising three mutually parallel laser lines to the tunneling equipment to be tested, wherein the three laser lines are arranged in a triangular shape; the data acquisition module is positioned on the machine body of the tunneling equipment to be detected, is opposite to the three laser direction indicator modules and is used for acquiring the characteristic image of the combined laser; the characteristic image is a three-laser-spot image or a three-laser-spot three-laser-line image; and the pose resolving module is connected with the data acquisition module and is used for calculating pose information of the tunneling equipment to be detected according to the characteristic image. The invention can overcome the problem that the tunnel characteristics are not obvious or even the characteristics are lost under the working condition environment with high dust and low illumination, and improves the stability and the anti-interference capability of the pose detection system.
Description
Technical Field
The invention relates to the technical field of pose detection, in particular to a system and a method for detecting pose of underground coal mine tunneling equipment.
Background
The tunneling is an important production link of coal mine production, and the rapid tunneling of a coal mine tunnel is a key technical measure for ensuring high and stable yield of a mine in a coal mine. With the development of fully mechanized mining technology and the demand of less humanization and no humanization, the position and posture detection of the tunneling equipment enables the roadway tunneling to become a common and key technology for intelligent application and efficient intensive production of coal mines.
The machine vision pose measurement technology is to establish a pose resolving model by utilizing space geometric projection constraint conditions so as to realize measurement of a target pose, has the advantages of non-contact measurement, high precision, good stability and the like, and is primarily applied to underground coal mines. However, in the process of underground coal mine roadway tunneling, under the working condition environment with high dust and low illumination, roadway characteristics are not obvious, and even characteristics are lost.
Disclosure of Invention
The invention aims to provide a pose detection system and a pose detection method for underground coal mine excavation equipment, which are used for overcoming the problems that roadway characteristics are not obvious or even characteristics are lost under working condition environments with high dust and low illumination, and improving the stability and the anti-interference capability of the pose detection system.
In order to achieve the purpose, the invention provides the following scheme:
a position and posture detection system for underground coal mine tunneling equipment comprises:
the three-laser direction indicator module is positioned in a rear roadway of the tunneling equipment to be tested and used for emitting combined laser to the tunneling equipment to be tested; the combined laser comprises three laser lines which are parallel to each other and arranged in a triangular shape;
the data acquisition module is positioned on the machine body of the tunneling equipment to be tested, is opposite to the three laser direction indicator modules and is used for acquiring the characteristic image of the combined laser; the characteristic image is a three-laser-spot image or a three-laser-spot three-laser-line image;
the pose resolving module is connected with the data acquisition module and used for calculating pose information of the tunneling equipment to be detected according to the characteristic image; the pose information includes: the heading device comprises a pitching angle, a yaw angle and a roll angle of a machine body of the heading device to be detected relative to a roadway, and horizontal, vertical and front-back offset distances of the machine body of the heading device to be detected from a central line of the roadway.
Optionally, the three laser pointer module comprises:
the laser direction indicator support is positioned in a rear roadway of the tunneling equipment to be tested;
the three laser direction indicators are respectively positioned on the laser direction indicator support and are used for emitting combined laser to the tunneling equipment to be tested; the three laser direction indicators are parallel to each other in pairs and are distributed in a triangular shape.
Optionally, the data acquisition module comprises:
the camera explosion-proof shell is fixed on the machine body of the tunneling equipment to be detected;
and the industrial camera is fixed in the camera explosion-proof shell, and the lens faces the three laser direction indicator modules and is used for acquiring the characteristic image of the combined laser.
Optionally, the pose resolving module includes:
the image characteristic judging unit is connected with the data acquisition module and used for acquiring a characteristic image of the combined laser and judging whether the characteristic image is a three-laser-point image or a three-laser-point three-laser-line image;
the laser point feature extraction unit is connected with the image feature judgment unit and used for extracting three laser points in the feature image and determining the position coordinates of each laser point when the feature image is a three-laser-point image;
the laser line characteristic extraction unit is connected with the image characteristic judgment unit and used for extracting three laser lines in the characteristic image and determining the position coordinates of each laser line and the position coordinates of the corresponding laser point when the characteristic image is a three-laser-point three-laser-line image; the number of the laser points is three, and the three laser points respectively correspond to the starting points of the three laser lines;
the three-point model calculation unit is connected with the laser point feature extraction unit and used for calculating the position coordinates of the tunneling equipment to be tested according to the position coordinates of the laser points;
the three-point three-line model calculation unit is connected with the laser line feature extraction unit and used for calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser line and each laser point;
and the pose resolving unit is respectively connected with the three-point model calculating unit and the three-point three-line model calculating unit and is used for calculating the pose information of the tunneling equipment to be detected according to the position coordinates of the tunneling equipment to be detected.
Optionally, the system further comprises:
and the pose visualization module is connected with the pose resolving module and is used for displaying the pose information.
The invention also provides a position and posture detection method of the underground coal mine tunneling equipment, which is applied to the system and comprises the following steps:
acquiring a characteristic image of the combined laser, and judging whether the characteristic image is a three-laser-spot image or a three-laser-spot three-laser-line image; the characteristic image is acquired by a data acquisition module through combined laser emitted by the three laser direction indicator modules;
when the characteristic image is a three-laser-spot image, extracting three laser spots in the characteristic image and determining the position coordinates of each laser spot; calculating the position coordinates of the tunneling equipment to be tested according to the position coordinates of the laser points;
when the characteristic image is a three-laser-spot three-laser-line image, extracting three laser lines in the characteristic image, and determining the position coordinates of each laser line and the position coordinates of the corresponding laser spot; the number of the laser points is three, and the three laser points respectively correspond to the starting points of the three laser lines; calculating the position coordinates of the tunneling equipment to be tested according to the position coordinates of each laser line and each laser point;
and calculating the pose information of the tunneling equipment to be detected according to the position coordinates of the tunneling equipment to be detected.
Optionally, the extracting three laser points in the feature image specifically includes:
carrying out distortion correction, gaussian filtering, morphological gradient conversion, RGB channel separation and gray level conversion on the characteristic image to obtain a first binary image;
performing closed operation on the first binary image, and determining each area in the first binary image;
selecting an area meeting first set conditions in the first binary image to perform ellipse fitting, and removing stray light to obtain a plurality of elliptical light spots; the first setting condition is as follows: the number of the area boundary points is within a first set range;
judging whether the number of the elliptical light spots is equal to three; if so, taking the three elliptical light spots as three laser points in the characteristic image; if not, returning to the step of acquiring the characteristic image of the combined laser and judging whether the characteristic image is a three-laser-spot image or a three-laser-spot three-laser-line image.
Optionally, the extracting three laser lines in the feature image specifically includes:
carrying out distortion correction, gaussian filtering and gray level conversion on the characteristic image to obtain a second binary image;
performing closed operation on the second binary image, and determining each region in the second binary image;
determining the minimum circumscribed rectangle of the area meeting a second set condition in the second binary image, and performing rectangle combination and contour point straight line fitting on the minimum circumscribed rectangle meeting a third set condition to obtain three laser lines in the characteristic image; the second setting condition is as follows: the number of the area edge points is within a second set range; the third setting condition is as follows: the aspect ratio range of the minimum circumscribed rectangle is within a third set range, and the area of the minimum circumscribed rectangle is within a fourth set range.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
according to the invention, the three laser direction indicator modules are arranged to emit the three combined lasers which are parallel to each other and arranged in a triangular shape to the tunneling equipment to be detected, the data acquisition module is used for acquiring the characteristic images of the combined lasers in real time, and the characteristic images with obvious characteristics can be acquired under the working condition environment with high dust and low illumination, so that the pose resolving module is used for calculating the pose information of the tunneling equipment to be detected according to the acquired three-laser-point images or three-laser-point three-laser-line images, and the stability and the anti-interference capability of the pose detection system are improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a structural diagram of a position and posture detection system of underground coal mine tunneling equipment provided by the invention;
FIG. 2 is a schematic model diagram of a three laser pointer module;
FIG. 3 is a schematic diagram of a feature image being a three laser point image;
FIG. 4 is a schematic diagram of a feature image being a three laser spot three laser line image;
FIG. 5 is a flow chart of the position and posture detection method for underground coal mine tunneling equipment provided by the invention;
FIG. 6 is a pose detection flow diagram provided by the present invention;
FIG. 7 is a perspective schematic view of a door-shaped three-wire structure;
FIG. 8 is a schematic diagram showing the positional relationship among three points a, b and c in a three-point three-wire model;
FIG. 9 is a flow chart of the three-point model image processing provided by the present invention;
FIG. 10 is a schematic diagram of a three-laser-point visual pose measurement method;
FIG. 11 is a schematic projection diagram of a three-point positioning model.
Description of the symbols:
the system comprises a 1-laser direction indicator support, a 2-laser direction indicator, a 3-industrial personal computer, a 4-explosion-proof control box, a 5-industrial camera and a 6-camera explosion-proof shell.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
The invention aims to provide a pose detection system and method for underground coal mine tunneling equipment, which are used for overcoming the problems that roadway characteristics are not obvious or even characteristics are lost under working condition environments with high dust and low illumination, and improving the stability and the anti-interference capability of the pose detection system.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a structural diagram of a position and posture detection system for underground coal mine tunneling equipment, as shown in fig. 1, the position and posture detection system for underground coal mine tunneling equipment provided by the invention comprises: the system comprises a three-laser direction indicator module, a data acquisition module and a pose resolving module.
The three-laser direction indicator module is positioned in a rear roadway of the tunneling equipment to be tested and used for emitting combined laser to the tunneling equipment to be tested; the combined laser comprises three mutually parallel laser lines which are arranged in a triangular shape (as shown by three mutually parallel straight lines in fig. 1). The data acquisition module is positioned on the machine body of the tunneling equipment to be tested, is opposite to the three laser direction indicator modules and is used for acquiring characteristic images of the combined laser; the characteristic image is a three-laser-point image or a three-laser-point three-laser-line image. The pose resolving module is connected with the data acquisition module and used for calculating pose information of the tunneling equipment to be detected according to the characteristic image; the pose information includes: the heading equipment to be tested is characterized by comprising a pitching angle, a yaw angle and a roll angle of a machine body of the heading equipment to be tested relative to a roadway, and horizontal, vertical and front-back offset distances of the machine body of the heading equipment to be tested from a central line of the roadway.
Further, the system further comprises: a pose visualization module; the pose visualization module is connected with the pose resolving module and used for displaying the pose information. The pose visualization module can be arranged on the tunneling equipment body or outside the tunneling equipment body, and can dynamically display the pose information of the tunneling equipment in real time in various forms (such as data, graphs, oscillograms and the like), so that the tunneling, remote and intelligent control and deviation correction of the tunneling equipment are facilitated.
In this embodiment, the three laser pointer modules include: a laser direction indicator support 1 and three laser direction indicators 2. The laser direction indicator support 1 is positioned in a rear roadway of the tunneling equipment to be tested; the three laser direction indicators 2 are respectively positioned on the laser direction indicator support 1 and are used for emitting combined laser to the tunneling equipment to be tested; the three laser direction indicators 2 are parallel to each other two by two, and the three laser direction indicators 2 are distributed in a triangular shape. Fig. 2 is a schematic model diagram of a three-laser-pointer module. As shown in fig. 2, the three laser direction indicators 2 are respectively located at a point a, a point b and a point c on the laser direction indicator support 1, and the connection lines between every two of the three laser direction indicators 2 form a triangle. The laser direction indicator 2 is preferably a mine explosion-proof laser direction indicator, and the imaging characteristics of the laser direction indicator are used as the input of a data acquisition module and are used for providing a data source for pose calculation.
The data acquisition module comprises: a camera explosion proof case 6 and an industrial camera 5. The camera explosion-proof shell 6 is fixed on a machine body of the tunneling equipment to be tested; the industrial camera 5 is fixed in the camera explosion-proof shell 6, and the lens faces the three laser direction indicator modules and is used for collecting characteristic images of the combined laser. The industrial camera 5 is preferably an explosion-proof industrial camera. Specifically, the camera explosion-proof shell 6 is installed at the rear part of the tunneling equipment body, the industrial camera 5 is fixed in the camera explosion-proof shell 6, a lens of the industrial camera faces the rear part of the roadway, and the industrial camera is used for acquiring image information of the three-laser direction finder in real time in the operation process of the tunneling equipment. Further, the camera explosion-proof shell 6 is fixed on the heading machine platform through bolts, the camera opening faces backwards, and the industrial camera 5 is installed inside the camera explosion-proof shell 6. In practical application, the data acquisition module further comprises a communication line.
The pose resolving module takes an industrial computer as a computing core, and is preferably an industrial personal computer 3. The industrial personal computer 3 is arranged at the position of the body of the tunneling equipment to be detected, and can process and resolve the feature images acquired by the explosion-proof industrial camera in the data acquisition module in real time, so that data are provided for the pose visualization module. In order to protect the industrial personal computer 3 from being damaged by external environment (such as falling rocks in a roadway), the system also comprises an explosion-proof control box 4; the explosion-proof control box 4 is fixed at the position of the machine body of the tunneling equipment to be detected, and the industrial personal computer 3 is installed in the explosion-proof control box 4.
Further, the pose solution module includes: the system comprises an image feature judgment unit, a laser point feature extraction unit, a laser line feature extraction unit, a three-point model calculation unit, a three-point three-line model calculation unit and a pose calculation unit.
The image characteristic judging unit is connected with the data acquisition module and used for acquiring the characteristic image of the combined laser and judging whether the characteristic image is a three-laser-point image or a three-laser-point three-laser-line image. In addition, the imaging characteristics can be predicted according to the dust condition in the tunnel, when no dust exists or the dust concentration is low, the characteristic image is a three-laser-spot image, and when the dust concentration is high, the characteristic image is a three-laser-spot three-laser-line image.
And the laser point feature extraction unit is connected with the image feature judgment unit and used for extracting three laser points in the feature image and determining the position coordinates of each laser point when the feature image is a three-laser-point image. And the three-point model calculating unit is connected with the laser point feature extracting unit and used for calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of the laser points.
The laser line feature extraction unit is connected with the image feature judgment unit and is used for extracting three laser lines in the feature image and determining the position coordinates of each laser line and the position coordinates of the corresponding laser point when the feature image is a three-laser-point three-laser-line image; the number of the laser points is three, and the three laser points respectively correspond to the starting points of the three laser lines. And the three-point three-line model calculating unit is connected with the laser line characteristic extracting unit and used for calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser line and each laser point.
The pose calculating unit is respectively connected with the three-point model calculating unit and the three-point three-line model calculating unit and is used for calculating pose information of the tunneling equipment to be detected according to the position coordinates of the tunneling equipment to be detected.
In specific application, as shown in fig. 1, the laser direction indicator support 1 is installed at the top end of a roadway of a heading face and fixed with an anchor cable at the top of a coal roadway by adopting three nuts, and is reliable in installation and convenient to detach. The laser direction indicator support 2 is used for mounting the laser direction indicator 2. The laser direction indicator 2 is arranged in an inverted triangle, the front end face of the laser direction indicator 2 is kept in the same plane, the plane is vertical to the ground, and the laser direction indicator 2 is parallel after being arranged, so that the error between the side length of a triangle formed at the beginning of laser point emission and the side length of a triangle formed on a certain plane in the air after a certain distance of laser irradiation is not more than 5 cm.
Installing three mining explosion-proof laser direction indicators and supports thereof at the top of a coal roadway behind tunneling equipment, and acquiring imaging information of the three mining explosion-proof laser direction indicators in real time through an explosion-proof industrial camera fixed on a machine body of the tunneling equipment; when the dust is less, the imaging characteristics of the three mining explosion-proof laser direction indicators are three points, namely the characteristic images are three laser point images, as shown in fig. 3, so that the pose is resolved by using the three laser points; when the dust concentration is high, the imaging characteristics of the three mining explosion-proof laser direction indicators are three lines formed by emission of three laser points, namely, the characteristic image is a three-laser-point three-laser-line image as shown in fig. 4, so that the pose is resolved by using the three lines of the three points; and finally, obtaining a pitch angle, a yaw angle and a roll angle of the tunneling equipment in the tunneling process and horizontal, vertical and front-back offset distances of the machine body relative to a center line of a roadway.
The invention also provides a pose detection method of the underground coal mine tunneling equipment, the method is applied to the system, the figure 5 is a flow chart of the pose detection method of the underground coal mine tunneling equipment provided by the invention, and the figure 6 is a flow chart of the pose detection method provided by the invention. As shown in fig. 5 and 6, the method includes:
step S1: acquiring a characteristic image of the combined laser, and judging whether the characteristic image is a three-laser-point image or a three-laser-point three-laser-line image; the characteristic image is acquired by the data acquisition module through combined laser emitted by the three laser direction indicator modules.
Step S2: when the characteristic image is a three-laser-spot image, extracting three laser spots in the characteristic image, and determining the position coordinates of each laser spot; and calculating the position coordinates of the tunneling equipment to be tested according to the position coordinates of the laser points.
And step S3: when the characteristic image is a three-laser-spot three-laser-line image, extracting three laser lines in the characteristic image, and determining the position coordinates of each laser line and the position coordinates of the corresponding laser spot; the number of the laser points is three, and the three laser points respectively correspond to the starting points of the three laser lines; and calculating the position coordinates of the tunneling equipment to be tested according to the position coordinates of each laser line and each laser point.
And step S4: and calculating the pose information of the tunneling equipment to be detected according to the position coordinates of the tunneling equipment to be detected. The laser direction indicator has three-dimensional coordinate information relative to the roadway, the laser direction indicator has three-dimensional coordinate information relative to the industrial camera, and the industrial camera has three-dimensional coordinate information relative to the tunneling equipment body. Therefore, the pitch angle, the yaw angle and the roll angle of the tunneling equipment body relative to the roadway and the horizontal, vertical and front-back offset distances from the center line of the roadway can be calculated according to the position coordinates of the tunneling equipment to be detected.
The method comprises the following specific implementation steps:
the method comprises the following steps: arranging a tunneling machine (namely tunneling equipment to be detected) in a tunneling roadway, and installing a three-laser direction indicator at the rear part of a tunneling machine body, wherein the three-laser direction indicator is installed at the top of the roadway and is not far away from an industrial camera installed at the rear part of the tunneling machine; an explosion-proof camera shell is arranged at the rear part of the development machine body, and an industrial camera is placed inside the explosion-proof camera shell.
Step two: acquiring image information by using an explosion-proof industrial camera, calibrating the camera, and determining calibration parameters; the calibration parameters comprise internal parameters of the camera (namely an industrial camera), and the internal parameters of the camera comprise the focal length of the camera, the coordinates of the main pixel point and the image distortion coefficient.
Step three: in the tunneling process, image information (namely characteristic images) collected by an industrial camera in real time is processed through an industrial computer arranged in the pose resolving module on the tunneling machine body, and the pose of the tunneling equipment is resolved in real time by using three laser spot models or three laser spots and three laser line models.
Step four: in the tunneling process, the pose calculating module converts the machine body information into the tunnel coordinate system through the coordinates of the feature points in the tunnel coordinate system, the coordinates of the feature points in the camera coordinate system and the camera coordinates in the machine body coordinate system to obtain the pose of the tunneling equipment. Specifically, when the feature image is a three-laser-spot image, the feature points are three laser spots; and when the characteristic image is a three-laser-point three-laser-line image, the characteristic point is a point obtained by two laser points and a perspective principle.
Step five: in the tunneling process, the pose visualization module is used for displaying the pose information of the tunneling equipment of the pose resolving module in real time, so that the tunneling equipment can be controlled and corrected remotely and intelligently.
The above steps are discussed in detail below.
In step S3, the extracting three laser lines in the feature image specifically includes:
step S3.1: and carrying out distortion correction, gaussian filtering and gray level conversion on the characteristic image to obtain a second binary image.
Step S3.2: and performing closed operation on the second binary image, and determining each region in the second binary image.
Step S3.3: determining the minimum circumscribed rectangle of the region meeting a second set condition in the second binary image, and performing rectangle combination and contour point straight line fitting on the minimum circumscribed rectangle meeting a third set condition to obtain three laser lines in the characteristic image; the second setting condition is as follows: the number of the area edge points is within a second set range; the third setting condition is as follows: the height-width ratio range of the minimum circumscribed rectangle is within a third set range, and the area of the minimum circumscribed rectangle is within a fourth set range.
As a specific implementation, when the characteristic image is a three-laser-spot three-laser-line image, the adopted image processing method is as follows:
(1) And carrying out distortion correction on the original image by using the focal length, the main pixel point coordinates and the image distortion coefficient, and outputting the corrected image.
(2) And performing Gaussian filtering on the corrected image to eliminate Gaussian noise.
(3) And performing gray level conversion on the Gaussian filtered image, and outputting a binarized image (namely a second binary image). Wherein, the binary threshold value is an empirical value.
(4) Detecting all contours in the binary image, wherein the detection method comprises the following steps: and (3) performing closed operation on the binarized image, then obtaining the edge of each region, finding out the minimum circumscribed rectangle of the region with the number of the edge points of the region in the (a, b) interval, and judging the linear contour meeting the condition according to the height-width ratio range (c, d) of the minimum circumscribed rectangle and the area range [ e, f ] of the minimum circumscribed rectangle. The aspect ratio range (c, d) and the area range [ e, f ] are empirical values. Recording the coordinates of each contour point; screening the area outline meeting the requirements for the first time, and solving the problem of more small areas on the line; performing rectangle combination for the second time, and combining the areas with similar external rectangle angles; and finally, screening a large area after polymerization meeting the linear proportion for the last time, and providing a basis for linear fitting.
(5) And processing the contour points into a straight line through least square fitting. The other two straight lines are fitted in sequence in the manner described above. And respectively acquiring the slopes and the intercepts of the three straight lines, substituting the slopes and the intercepts into a three-point three-line model, and calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser line and each laser point.
The calculating the position coordinates of the tunneling equipment to be tested according to the position coordinates of each laser line and each laser point specifically comprises: determining at least one door-shaped three-line structure according to the position coordinates of each laser line and each laser point; the gate-shaped three-line structure is formed by connecting lines of any two laser points and two corresponding laser lines, and the connecting lines of the three laser points and the three laser lines jointly form three gate-shaped three-line structures; and solving the portal three-line structure based on the perspective principle of the portal three-line structure to obtain the position coordinate of the tunneling equipment to be detected.
In addition, when the dust concentration is high, the imaging characteristic is three points and three lines (namely the characteristic image is three laser points and three laser line images), and if an object is shielded to cause one point and one line to be shielded, the pose can be resolved by using two points and two lines based on the perspective principle of a door-shaped three-line structure.
In the specific application, firstly, on the basis of a two-point two-line heading machine pose calculation model, a laser point and a line generated by the laser point are extracted, the slope and the intercept of a straight line obtained by image processing are substituted into the model to obtain the position coordinate of the heading equipment, and then the pose of the machine body in a roadway is obtained through coordinate conversion.
Fig. 7 is a perspective schematic diagram of a door-shaped three-line structure, as shown in fig. 7, according to the three-line perspective characteristic of the door-shaped structure, when two parallel straight lines are not parallel to the camera image plane, a unique solution exists in pose calculation between the camera coordinate system and the laser measurement coordinate system. The invention provides a three-point three-line vision measurement model, which is characterized in that a new laser is added, namely the original two laser beams are changed into three laser beams, and the laser characteristics are abstracted from the original two light spots and two lines to form three light spots and three lines.
The three-point three-line model provided by the invention is based on a three-line perspective geometric structure model of a door-shaped structure, and is different in that the three-line perspective geometric structure model of the door-shaped three-line structure is calculated three times in the invention, namely three different dotted line combination modes of ac, ab and bc respectively, and the position relation of three points a, b and c is shown in figure 8.
x g ＝t 1 *x ac +t 2 *x ab +t 3 *x bc (1)
y g ＝t 1 *y ac +t 2 *y ab +t 3 *y bc (2)
z g ＝t 1 *z ac +t 2 *z ab +t 3 *z bc (3)
1＝t 1 +t 2 +t 3 (4)
In the formula: x is the number of g ，y g ，z g Into a groupT is a weighted value of one of the three combination modes according to the positioned coordinate (namely the three-dimensional point coordinate of the tunneling equipment to be detected), wherein t is 1 Weighted value, t, for the combination of ac 2 Weight value of ab three combinations, t 3 A weighted value for the bc combination; x is the number of ac ，y ac ，z ac Position coordinates, x, of the excavating equipment calculated for ac two points ab ，y ab ，z ab Position coordinates, x, of the excavating equipment calculated for points ab bc ，y bc ，z bc And (4) calculating the position coordinates of the tunneling equipment for the two points bc. The way of calculating the coordinates based on the door-shaped three-line structure is not described herein.
In step S2, the extracting three laser points in the feature image specifically includes:
step S2.1: and carrying out distortion correction, gaussian filtering, morphological gradient conversion, RGB channel separation and gray level conversion on the characteristic image to obtain a first binary image.
Step S2.2: and performing closed operation on the first binary image, and determining each area in the first binary image.
Step S2.3: selecting an area meeting a first set condition in the first binary image to perform ellipse fitting, and removing stray light to obtain a plurality of elliptical light spots; the first setting condition is as follows: the number of the area boundary points is within a first setting range.
Step S2.4: judging whether the number of the elliptical light spots is equal to three; if so, taking the three elliptical light spots as three laser points in the characteristic image; and if not, returning to the step of acquiring the characteristic image of the combined laser and judging whether the characteristic image is a three-laser-spot image or a three-laser-spot three-laser-line image.
As a specific implementation, when the feature image is a three-laser-spot image, the image processing method adopted with reference to fig. 9 includes:
(1) And carrying out distortion correction on the image by using the internal parameters (namely the focal length, the coordinates of the main pixel points and the image distortion coefficient) of the industrial camera obtained by calibration, and outputting the corrected image.
(2) And performing Gaussian filtering on the corrected image, and performing morphological gradient transformation after filtering.
(3) And separating R, G and B channels of the converted image to obtain an R channel image, solving the maximum gray scale of the image, and determining the range of the binarization threshold value according to the maximum gray scale.
(4) And if the number of the area boundaries is more than 3, judging whether the number of the boundary points is more than a and less than b, if so, calculating the minimum rectangle of the area, and deleting some veiling glare according to the aspect ratio of the minimum rectangle. The number ranges (a, b) of the boundary points are empirical values.
(5) Fitting the residual area by utilizing ellipse fitting, acquiring the central coordinates of the light spots, removing part of stray light according to the distance range [ g, h ] between the elliptical light spots, comparing with the image at the previous moment, and removing part of stray light.
(6) Judging the number of the points, if the number of the points is not equal to 3, acquiring the images again, otherwise substituting the images into a three-point positioning model of the heading machine, and calculating the position coordinates of the heading equipment to be detected according to the position coordinates of each laser point.
The calculating of the position coordinates of the tunneling equipment to be tested according to the position coordinates of the laser points specifically comprises: determining a rotation matrix of an industrial camera positioned on a machine body of the tunneling equipment to be tested and a direction measurement vector of each laser point in a camera coordinate system according to the position coordinate of each laser point; and determining the position coordinate of the tunneling equipment to be tested according to the rotation matrix and the direction measurement vector.
Fig. 10 is a schematic diagram of a three-laser-point visual pose measurement method. In a specific application, the three-point positioning model gives the laser spot characteristic f as shown in FIG. 10 i Position of =1,2,3 in reference coordinate system G P i And the direction measurement vector of the feature point in the camera coordinate system C b i The goal is to estimate the rotation matrix and position of the camera G p C . Where { C } represents the camera coordinate system, with position p C In an orientation of
{ G } denotes a reference coordinate system.
From the geometric relationship, obtain
In formula (5): G p i representing the coordinates of the three laser points in a reference coordinate system, G p C is the position of the camera or cameras, and, C b i vectors are measured for the direction of the feature points in the camera coordinate system. d i And expressing Euclidean distances from the camera to the feature points in the reference coordinate system, subtracting every two Euclidean distances, and projecting to a normal vector of the corresponding surface.
The camera coordinates are specifically calculated as follows:
FIG. 11 is a schematic projection diagram of a three-point positioning model. As shown in fig. 11, for 3 known spatial points P 1 、P 2 And P 3 The length of 3 side lengths of the triangle is A, B and C. The imaging points in the imaging plane of the camera are respectively P c1 、P c2 And P c3 Center point O of optical axis of camera and space point P 1 、P 2 And P 3 The unit vector of the composition is k 1 、k 2 And k 3 。
Will k 2 And k 3 The included angle between the two is set as alpha, k 1 And k 3 The included angle between the two is beta, k is 1 And k 2 The included angle between the two is gamma, then:
note O point and nullIntermediate point P 1 、P 2 And P 3 Each having a distance of m 1 、m 2 And m 3 . Based on the cosine theorem of triangles, equation (8) can be obtained.
To simplify the calculation, the following equivalent substitution is made
In the formula (9), p represents m 1 And m 3 Q represents m 2 And m 3 The proportional relationship between them.
Obtainable formulae (10) and (11)
In the formula (11), a j Denotes the coefficients of the polynomial, and j denotes the order of the polynomial.
Wherein, the first and the second end of the pipe are connected with each other,
from the above, p, q, m can be obtained 1 、m 2 And m 3 The value of (c). The position coordinate of each space point in the camera is P ci ＝m i k i I =1,2,3. The image coordinates of the 2D points are known, 3 moreThe chord angle is known, the coordinates of the 3D points are known, and only x and y are unknown. A is given by formula (11) j Are known. Therefore we can find the values of p and q. The system of equations of the fourth degree has four theoretical solutions. Let four poses solve as (x) m ，y m ) M =1,2,3,4, the minimum two-norm is found.
As shown in equation (12), the minimum is the pose of the camera (i.e., the roadheader body) relative to the target coordinate system.
The position of the heading machine under the roadway coordinate system can be obtained by positioning the heading machine body with three laser points and then obtaining the relation between the machine body coordinate system and the roadway coordinate system according to the known relation between the laser direction indicator coordinate system and the roadway coordinate system.
The invention provides a position and posture detection system and a position and posture detection method of underground coal mine tunneling equipment based on a three-laser direction indicator, aiming at solving the problem that dust is large in the tunneling or overhauling process and the problem that the dust is featureless, and compared with the prior art, the position and posture detection system and the position and posture detection method have the following advantages:
(1) The system is complete and stable, and can be primarily applied to actual operation.
(2) Three groups of two lines are used for fusion calculation, so that the calculation result is more stable and accurate.
(3) The invention integrates a three-point and three-point three-line calculation method to realize self-adaptive environment switching. The three-point model calculation method is suitable for being used when the underground coal mine dust concentration is low, and the three-point three-line model calculation method is suitable for being used when the underground coal mine dust concentration is high, and has good applicability.
(4) The machine vision based on the laser direction finder is applied to pose detection of underground coal mine tunneling equipment, has the characteristics of low cost, flexibility in operation, strong anti-interference performance, high measurement precision, good system stability and the like, and has certain market popularization value.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to assist in understanding the core concepts of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the foregoing, the description is not to be taken in a limiting sense.
Claims (8)
1. The position and posture detection system for the underground coal mine tunneling equipment is characterized by comprising:
the three-laser direction indicator module is positioned in a rear roadway of the tunneling equipment to be tested and used for emitting combined laser to the tunneling equipment to be tested; the combined laser comprises three laser lines which are parallel to each other and arranged in a triangular shape;
the data acquisition module is positioned on the machine body of the tunneling equipment to be detected, is opposite to the three laser direction indicator modules and is used for acquiring the characteristic image of the combined laser; the characteristic image is a three-laser-spot image or a three-laser-spot three-laser-line image;
the pose resolving module is connected with the data acquisition module and used for calculating pose information of the tunneling equipment to be detected according to the characteristic image; the pose information includes: the heading equipment to be tested is characterized by comprising a pitching angle, a yaw angle and a roll angle of a machine body of the heading equipment to be tested relative to a roadway, and horizontal, vertical and front-back offset distances of the machine body of the heading equipment to be tested from a central line of the roadway.
2. The coal mine underground tunneling equipment pose detection system according to claim 1, wherein the three laser direction finder module comprises:
the laser direction indicator support is positioned in a rear roadway of the tunneling equipment to be tested;
the three laser direction indicators are respectively positioned on the laser direction indicator bracket and are used for emitting combined laser to the tunneling equipment to be tested; the three laser direction indicators are parallel to each other in pairs and are distributed in a triangular shape.
3. The position and pose detection system for the coal mine underground tunneling equipment according to claim 1, wherein the data acquisition module comprises:
the camera explosion-proof shell is fixed on the machine body of the tunneling equipment to be detected;
and the industrial camera is fixed in the camera explosion-proof shell, and the lens faces the three laser direction indicator modules and is used for acquiring the characteristic image of the combined laser.
4. The coal mine underground tunneling equipment pose detection system according to claim 1, wherein the pose resolving module comprises:
the image characteristic judging unit is connected with the data acquisition module and used for acquiring a characteristic image of the combined laser and judging whether the characteristic image is a three-laser-point image or a three-laser-point three-laser-line image;
the laser point feature extraction unit is connected with the image feature judgment unit and used for extracting three laser points in the feature image and determining the position coordinates of each laser point when the feature image is a three-laser-point image;
the laser line characteristic extraction unit is connected with the image characteristic judgment unit and used for extracting three laser lines in the characteristic image and determining the position coordinates of each laser line and the position coordinates of the corresponding laser point when the characteristic image is a three-laser-point three-laser-line image; the number of the laser points is three, and the three laser points respectively correspond to the starting points of the three laser lines;
the three-point model calculation unit is connected with the laser point feature extraction unit and used for calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of the laser points;
the three-point three-line model calculation unit is connected with the laser line feature extraction unit and used for calculating the position coordinates of the tunneling equipment to be detected according to the position coordinates of each laser line and each laser point;
and the pose resolving unit is respectively connected with the three-point model calculating unit and the three-point three-line model calculating unit and is used for calculating the pose information of the tunneling equipment to be detected according to the position coordinates of the tunneling equipment to be detected.
5. The coal mine underground tunneling equipment pose detection system according to claim 1, further comprising:
and the pose visualization module is connected with the pose resolving module and is used for displaying the pose information.
6. A coal mine underground tunneling equipment pose detection method is applied to the system of claim 4, and comprises the following steps:
acquiring a characteristic image of the combined laser, and judging whether the characteristic image is a three-laser-point image or a three-laser-point three-laser-line image; the characteristic image is acquired by a data acquisition module through combined laser emitted by the three laser direction indicator modules;
when the characteristic image is a three-laser-spot image, extracting three laser spots in the characteristic image and determining the position coordinates of each laser spot; calculating the position coordinates of the tunneling equipment to be tested according to the position coordinates of the laser points;
when the characteristic image is a three-laser-spot three-laser-line image, extracting three laser lines in the characteristic image, and determining the position coordinates of each laser line and the position coordinates of the corresponding laser spot; the number of the laser points is three, and the three laser points respectively correspond to the starting points of the three laser lines; calculating the position coordinates of the tunneling equipment to be tested according to the position coordinates of each laser line and each laser point;
and calculating the pose information of the tunneling equipment to be detected according to the position coordinates of the tunneling equipment to be detected.
7. The method for detecting the pose of the coal mine underground tunneling equipment according to claim 6, wherein the extracting three laser points in the characteristic image specifically comprises the following steps:
carrying out distortion correction, gaussian filtering, morphological gradient conversion, RGB channel separation and gray level conversion on the characteristic image to obtain a first binary image;
performing closed operation on the first binary image, and determining each area in the first binary image;
selecting an area meeting a first set condition in the first binary image to perform ellipse fitting, and removing stray light to obtain a plurality of elliptical light spots; the first setting condition is as follows: the number of the area boundary points is within a first set range;
judging whether the number of the elliptical light spots is equal to three; if so, taking the three elliptical light spots as three laser points in the characteristic image; if not, returning to the step of acquiring the characteristic image of the combined laser and judging whether the characteristic image is a three-laser-spot image or a three-laser-spot three-laser-line image.
8. The method for detecting the pose of the coal mine underground tunneling equipment according to claim 6, wherein the extracting three laser lines in the characteristic image specifically comprises:
carrying out distortion correction, gaussian filtering and gray level conversion on the characteristic image to obtain a second binary image;
performing closed operation on the second binary image, and determining each region in the second binary image;
determining the minimum circumscribed rectangle of the region meeting a second set condition in the second binary image, and performing rectangle combination and contour point straight line fitting on the minimum circumscribed rectangle meeting a third set condition to obtain three laser lines in the characteristic image; the second setting condition is as follows: the number of the area edge points is within a second set range; the third setting condition is as follows: the aspect ratio range of the minimum circumscribed rectangle is within a third set range, and the area of the minimum circumscribed rectangle is within a fourth set range.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211438343.7A CN115727851A (en) | 2022-11-17 | 2022-11-17 | Pose detection system and method for underground coal mine tunneling equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211438343.7A CN115727851A (en) | 2022-11-17 | 2022-11-17 | Pose detection system and method for underground coal mine tunneling equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115727851A true CN115727851A (en) | 2023-03-03 |
Family
ID=85296166
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211438343.7A Pending CN115727851A (en) | 2022-11-17 | 2022-11-17 | Pose detection system and method for underground coal mine tunneling equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115727851A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116735079A (en) * | 2023-08-15 | 2023-09-12 | 山东宇飞传动技术有限公司 | Mining winch balance detection equipment and method |
-
2022
- 2022-11-17 CN CN202211438343.7A patent/CN115727851A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116735079A (en) * | 2023-08-15 | 2023-09-12 | 山东宇飞传动技术有限公司 | Mining winch balance detection equipment and method |
| CN116735079B (en) * | 2023-08-15 | 2023-11-14 | 山东宇飞传动技术有限公司 | Mining winch balance detection equipment and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11673587B2 (en) | Comprehensive inspection vehicle for subway tunnel | |
| US20220276383A1 (en) | System and method for rock mass structure detection and dangerous rock detection | |
| CN109753081B (en) | Roadway inspection unmanned aerial vehicle system based on machine vision and navigation method | |
| JP5319741B2 (en) | Overhead image generation device, map data generation system, overhead image generation method of overhead image generation device, and overhead image generation program | |
| AU2007355942B2 (en) | Arrangement and method for providing a three dimensional map representation of an area | |
| US4238828A (en) | Position detecting apparatus | |
| CN105550692B (en) | The homing vector landing concept of unmanned plane based on marker color and contour detecting | |
| CN104268935A (en) | Feature-based airborne laser point cloud and image data fusion system and method | |
| JP6060682B2 (en) | Road surface image generation system, shadow removal apparatus, method and program | |
| CN111735445B (en) | Monocular vision and IMU (inertial measurement Unit) integrated coal mine tunnel inspection robot system and navigation method | |
| CN104457719B (en) | The attitude measuring of class rectangular shield construction and measuring method | |
| CN109373978B (en) | Surrounding rock displacement monitoring method for roadway surrounding rock similar simulation | |
| CN110455368B (en) | Method and system for measuring volume of mineral carried by mine bucket | |
| WO2019144289A1 (en) | Systems and methods for calibrating an optical system of a movable object | |
| JP7233575B2 (en) | MAP GENERATION SYSTEM, MAP GENERATION METHOD AND MAP GENERATION PROGRAM | |
| CN115727851A (en) | Pose detection system and method for underground coal mine tunneling equipment | |
| JP2010249709A (en) | Cross section measuring device, cross section measuring method, and cross section measuring program | |
| CN115984766A (en) | Rapid monocular vision three-dimensional target detection method for underground coal mine | |
| CN104786227A (en) | Drop type switch replacing control system and method based on high-voltage live working robot | |
| CN109143246A (en) | Underwater pile detection method, system, device and storage medium | |
| CN112254646A (en) | Push bench posture recognition system and method and storage medium | |
| CN104858877B (en) | High-tension line drop switch changes the control method of control system automatically | |
| CN112907674B (en) | Camera calibration method and device for unmanned vehicle | |
| CN110658844B (en) | Ultra-high voltage direct current line channel unmanned aerial vehicle monitoring method and system | |
| CN114964025A (en) | Tunnel inverted arch deformation monitoring and early warning method based on binocular vision and early warning system thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |