AU2021350657A1 - Positioning method and system for fully mechanized mining face - Google Patents
Positioning method and system for fully mechanized mining face Download PDFInfo
- Publication number
- AU2021350657A1 AU2021350657A1 AU2021350657A AU2021350657A AU2021350657A1 AU 2021350657 A1 AU2021350657 A1 AU 2021350657A1 AU 2021350657 A AU2021350657 A AU 2021350657A AU 2021350657 A AU2021350657 A AU 2021350657A AU 2021350657 A1 AU2021350657 A1 AU 2021350657A1
- Authority
- AU
- Australia
- Prior art keywords
- camera device
- coordinate system
- depth camera
- coordinates
- absolute
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
Abstract
Provided by the present invention are a positioning method and positioning system for a fully mechanized mining face, wherein the positioning method comprises: acquiring absolute coordinates of a target image in an absolute coordinate system; if a depth camera device (200) collects a target image of one group of targets (100), then determining camera coordinates of the target image in a coordinate system of the depth camera device (200); determining absolute coordinates of the depth camera device (200) according to the absolute coordinates of the target image and the camera coordinates; and if the depth camera device (200) does not collect the target image, then controlling a tracking camera device (300) to turn on, and according to a conversion matrix between the coordinate system of the depth camera device (200) and the absolute coordinate system and a conversion matrix between a coordinate system of the tracking camera device (300) and the coordinate system of the depth camera device (200) at the time when the tracking camera device (300) is turned on, obtaining absolute coordinates of the tracking camera device (300) relative to the absolute coordinate system. The present invention solves the problem in the prior art in which continuous high-precision three-dimensional positioning coordinates of a target object cannot be obtained when there is no GPS signal.
Description
This application claims the priority of the Chinese patent application filed on September
2 8 th, 2020 before the CNIPA with the application number of 202011042486.7 and the title of
which is incorporated herein in its entirety by reference.
The present invention relates to the field of computer vision and the field of autonomous
navigation, and more particularly, to a positioning method and system for a fully mechanized
mining face.
With the country's strong demand for green mining, high-efficiency production and
safety first in coal mine industry, the automation of fully mechanized mining in underground
coal mine is becoming more and more popular, and how to realize unmanned and intelligent
mining becomes extremely important, wherein the independent positioning of key targets in
the underground coal mine is one of key basic technologies to realize unmanned and
intelligent mining in the underground coal mine. However, a high-precision positioning
technology for the targets in the underground coal mine is relatively backward at present,
which is specifically manifested in the following aspects. (1) Due to the particularity of
environment of the underground coal mine, GPS cannot be used for auxiliary positioning and calibration, so that many mature ground positioning technical solutions based on inertial navigation (most of which are based on the inertial navigation combined with GPS auxiliary calibration) cannot be directly applied to the environment of the underground coal mine.
Therefore, there is no mature high-precision real-time positioning technical solution capable
of lasting for a long time in the underground coal mine at present. (2) A positioning
technology used in the environment of the underground coal mine is mostly a technology
based on wireless positioning at present, and this technology has no time accumulation error
in principle and has high positioning precision in a one-dimensional space, but this
technology cannot achieve high positioning precision in a two-dimensional or even
three-dimensional space. Therefore, this technology is mostly used in a regional positioning
scene with a low precision requirement.
Based on the technical problems above, the Chinese patent discloses an indoor passive
navigation and positioning system and method, and the system comprises a depth camera
device, an inertial navigator, an identifier and an industrial control computer. The identifier is
posted on a wall, and the depth camera device, the inertial navigator and the industrial control
computer are integrally mounted on a robot. Image data and depth data containing the
identifier are collected by the depth camera device, pixel coordinates of certain three angular
points of the identifier are detected and inverted to obtain pixel coordinates of a fourth
angular point, and then camera coordinates of the fourth angular point are obtained.
Meanwhile, absolute coordinates of the fourth angular point may be obtained by a digital
recognition technology. In addition, an attitude angle of the robot at a current time may be
obtained by the inertial navigator. A conversion matrix between a camera coordinate system
and an absolute coordinate system may be obtained through absolute coordinates and camera coordinates of the identifier and the attitude angle of the robot, so as to obtain absolute coordinates of the robot. However, this solution has the following technical problems: (1) this system can obtain self-positioning information of the robot only when the depth camera device can collect data of the identifier, and is very dependent on the detection of the identifier, thus being difficult to obtain continuous positioning information; (2) an inertial navigation device is used in this system to obtain the attitude angle of the robot at the current time, so that a precision of the inertial navigator itself and a time synchronicity between the inertial navigator and the depth camera device may both affect a self-positioning result, and an accumulation error of the inertial navigator may become larger and larger with the passage of time; and (3) it is easy to introduce unnecessary error interference when this system obtains the camera coordinates and the absolute coordinates of the identifier, and the digital recognition technology is easy to make mistakes in detecting world coordinates of the identifier in the image.
A technical problem to be solved by the present invention is a problem in the prior art in
which continuous high-precision three-dimensional positioning coordinates of a target object
cannot be obtained when there is no GPS signal. Therefore, the present invention provides a
positioning method and positioning system for a fully mechanized mining face capable of
achieving continuous high-precision positioning.
To solve the technical problem above, the present invention provides the following
technical solution.
A positioning method for a fully mechanized mining face comprises: providing a depth
camera device and a tracking camera device on an end device of the fully mechanized mining
face, and arranging a plurality of groups of targets in a roadway of the fully mechanized
mining face, wherein adjacent groups of targets have a set interval therebetween, and each
group of targets comprises at least three target images; acquiring absolute coordinates of the
target image in an absolute coordinate system; if the depth camera device collects a target
image of one group of targets, determining camera coordinates of the target image in a depth
camera device coordinate system; and determining absolute coordinates of the depth camera
device according to the absolute coordinates and the camera coordinates of the target image;
and if the depth camera device fails to collect the target image, controlling the tracking
camera device to turn on, and obtaining absolute coordinates of the tracking camera device
relative to the absolute coordinate system according to a conversion matrix between the depth
camera device coordinate system and the absolute coordinate system and a conversion matrix
between a tracking camera device coordinate system and the depth camera device coordinate
system at the time when the tracking camera device is turned on.
In some embodiments of the present invention, the step of if the depth camera device
collects the target image, determining the camera coordinates of the target image in the depth
camera device coordinate system, specifically comprises: the target image being rectangular,
acquiring pixel coordinates of four angular points of the target image relative to the depth
camera device coordinate system, and determining pixel coordinates of a central point of the
target image relative to the depth camera device coordinate system; and obtaining camera
coordinates of the central point of the target image relative to the depth camera device
coordinate system according to an internal reference matrix of the depth camera device.
In some embodiments of the present invention, the step of acquiring the pixel
coordinates of the four angular points of the target image relative to the depth camera device
coordinate system, and determining the pixel coordinates of the central point of the target
image relative to the depth camera device coordinate system is achieved by the following
formula:
(2 Y2 ~ A0 (a 7y3 a 1 bU2 X(; ~ X2 13 I1- Y'3 wherein 2=XCg2*7Xg*22, C1=X3*Y1-XiY3,and
(xo, yo), (x, yI), (x2, y2) and (x3, y3) are the pixel coordinates of the four angular points of
the target image relative to the depth camera device coordinate system; and (x, y) are the
pixel coordinates of the central point of the target image relative to the depth camera device
coordinate system.
In some embodiments of the present invention, the step of obtaining the camera
coordinates of the central point of the target image relative to the depth camera device
coordinate system according to the internal reference matrix of the depth camera device is
achieved by the following formula:
-LiQC Yac
wherein (x, y) are the pixel coordinates of the central point of the target image relative to
the depth camera device coordinate system, (Xdc, Ydc, Zdc) are the camera coordinates of the
central point of the target image relative to the depth camera device coordinate system, and H
is the internal reference matrix of the depth camera device.
In some embodiments of the present invention, the step of determining the absolute
coordinates of the depth camera device according to the absolute coordinates and the camera
coordinates of the target image comprises: determining the conversion matrix between the
depth camera device coordinate system and the absolute coordinate system according to the
absolute coordinates and the camera coordinates of the target image, and determining the
absolute coordinates of the depth camera device according to the conversion matrix.
In some embodiments of the present invention, the step of determining the conversion
matrix between the depth camera device coordinate system and the absolute coordinate
system according to the absolute coordinates and the camera coordinates of the target image
is achieved by the following formula:
Hd2w* dc _
Zdc zw"
wherein (Xdc, Ydc, Zdc) are the camera coordinates of the central point of the target image
relative to the depth camera device coordinate system, (xi, yi, zw) are the absolute
coordinates of the central point of the target image relative to the absolute coordinate system,
and Hd2w1 is the conversion matrix between the depth camera device coordinate system and the
absolute coordinate system.
In some embodiments of the present invention, the step of determining the absolute
coordinates of the depth camera device according to the conversion matrix is achieved by the
following formula:
XdF 0 Yd =Hd 2 0
Zd d 0
wherein (xdw, yd., Zdw) are the absolute coordinates of the depth camera device relative to
the absolute coordinate system, and Hd2w is the conversion matrix between the depth camera
device coordinate system and the absolute coordinate system.
In some embodiments of the present invention, the step of obtaining the absolute
coordinates of the tracking camera device relative to the absolute coordinate system
according to the conversion matrix between the depth camera device coordinate system and
the absolute coordinate system and the conversion matrix between the tracking camera device
coordinate system and the depth camera device coordinate system at the time when the
tracking camera device is turned on comprises:
obtaining a conversion matrix between the tracking camera device coordinate system
and the absolute coordinate system according to the conversion matrix between the depth
camera device coordinate system and the absolute coordinate system and the conversion
matrix between the tracking camera device coordinate system and the depth camera device
coordinate system at the time when the tracking camera device is turned on, and obtaining the
absolute coordinates of the tracking camera device relative to the absolute coordinate system
according to the conversion matrix between the tracking camera device coordinate system
and the absolute coordinate system.
In some embodiments of the present invention, the step of obtaining the conversion
matrix between the tracking camera device coordinate system and the absolute coordinate
system according to the conversion matrix between the depth camera device coordinate system and the absolute coordinate system and the conversion matrix between the tracking camera device coordinate system and the depth camera device coordinate system at the time when the tracking camera device is turned on is achieved by the following formula:
Ht2w=Hd2w-Ht2d,
wherein Ht2wis the conversion matrix between the tracking camera device coordinate
system and the absolute coordinate system, Hd2w is the conversion matrix between the depth
camera device coordinate system and the absolute coordinate system, and Ht2d is the
conversion matrix between the tracking camera device coordinate system and the depth
camera device coordinate system.
In some embodiments of the present invention, the step of obtaining the absolute
coordinates of the tracking camera device relative to the absolute coordinate system
according to the conversion matrix between the tracking camera device coordinate system
and the absolute coordinate system is achieved by the following formula:
Zt1 .' 1a.
wherein (xtw, yt, ztw) are the absolute coordinates of the tracking camera device relative
to the absolute coordinate system, H2w is the conversion matrix between the tracking camera
device coordinate system and the absolute coordinate system, and (xto, yto, zto) are translation
information of coordinates of the tracking camera device at a current time relative to the time
when the tracking camera device is turned on.
The present invention also discloses a positioning system for a fully mechanized mining
face, which comprises: a depth camera device and a tracking camera device fixed on an end
device, and a plurality of groups of targets arranged in a roadway of the fully mechanized mining face, wherein adjacent groups of targets have a set interval therebetween, and each group of targets comprises at least three target images; wherein the positioning method above is performed in the positioning system.
Compared with the prior art, the technical solution of the present invention has the
following technical effects:
In the positioning method and positioning system for the fully mechanized mining face
provided by the present invention, a plurality of groups of targets are arranged along an
advancing direction of the fully mechanized mining face, each group of targets is provided
with at least three target images, and both of a depth camera device and a tracking camera
device are arranged on an end device of the fully mechanized mining face, so that the
absolute coordinates of the depth camera device may be determined by the absolute
coordinates of the target image and the camera coordinates of the target image relative to the
depth camera device if the depth camera device is in a calibration position where the target
image can be collected, thus positioning the fully mechanized mining face; and the
positioning of the fully mechanized mining face is realized by the tracking camera device if
the depth camera device is in a blind area where the target image cannot be collected, thus
obtaining the continuous positioning information in a process of advancing in the roadway of
the fully mechanized mining face, and being convenient for controlling the fully mechanized
mining face.
Further, in the positioning method and positioning system for the fully mechanized
mining face provided by the present invention, only the absolute coordinates of the three
target images are needed in the positioning when the positioning is carried out by the depth
camera device. Compared with the positioning through simultaneous detection by a depth camera and an inertial navigation device in the prior art, the positioning precision of the present invention is higher.
The preferred embodiments in the present invention will be described in detail
hereinafter with reference to the drawings, which will be beneficial for understanding the
objects and advantages of the present invention, wherein:
FIG. 1 is a schematic structural diagram of a specific embodiment of a positioning
system for a fully mechanized mining face of the present invention;
FIG. 2 is a schematic structural diagram of a target image of the present invention;
FIG. 3 is a relationship diagram of an absolute coordinate system, a depth camera device
coordinate system and a tracking camera device coordinate system of the present invention;
and
FIG. 4 is a flow chart of a positioning method for a fully mechanized mining face of the
present invention.
The technical solution of the present invention is clearly and completely described
hereinafter with reference to the drawings. Obviously, the described embodiments are only
some but not all of the embodiments of the present invention. Based on the embodiments in
the present invention, all other embodiments obtained by those of ordinary skills in the art
without going through any creative work should fall within the scope of protection of the
present invention.
In the description of the present invention, it should be noted that, the orientation or
position relationship indicated by the terms "center", "up", "down", "left", "right", "vertical",
"horizontal", "inside". "outside", and the like is based on the orientation or position
relationship shown in the drawings, it is only for the convenience of description of the
present invention and simplification of the description, and it is not to indicate or imply that
the indicated device or element must have a specific orientation, and be constructed and
operated in a specific orientation. Therefore, the terms should not be understood as limiting
the present invention. Moreover, the terms "first", "second" and "third" are used for
descriptive purposes only and cannot be understood as indicating or implying relative
importance.
In the description of the present invention, it should be noted that, the terms
"installation", "connected" and "connection" should be understood in a broad sense unless
otherwise specified and defined. For example, they may be fixed connection, removable
connection or integrated connection; and may be direct connection, or indirect connection
through an intermediate medium, and connection inside two elements. The specific meanings
of the above terms in the present invention can be understood in a specific case by those of
ordinary skills in the art.
In addition, the technical features involved in different implementations of the present
invention described hereinafter may be combined with each other as long as they do not
conflict with each other.
Specific embodiments of a positioning system for a fully mechanized mining face and a
positioning method thereof of the present invention are as follows. As shown in FIG. 1, the
positioning system for the fully mechanized mining face comprises a depth camera device
200 and a tracking camera device 300 fixed on an end device 400 (such as a reversed loader
and the like), and a plurality of groups of targets 100 arranged in a roadway 500 of the fully
mechanized mining face. The plurality of groups of targets 100 are arranged along an
advancing direction of the fully mechanized mining face, adjacent groups of targets 100 have
a set interval therebetween, and each group of targets 100 comprises at least three target
images 100a. The target image 100a is rectangular.
As shown in FIG. 4, the positioning method of the positioning system for the fully
mechanized mining face above comprises the following steps.
In 101, absolute coordinates of the target image 100a in an absolute coordinate system
are acquired.
Since an ID number of each target image 100a in the plurality of groups of targets 100
for calibration is unique in a whole environment, absolute three-dimensional space
coordinates of each target image 100a are collected by a total station, wherein steps of
measuring the absolute space coordinates by the total station sequentially comprise: 1) taking
out the total station, fixing a tripod, and mounting; 2) selecting one point on an experimental
site as an observation station point, and then selecting two points as observation points; 3)
adjusting a vertical circle and a horizontal circle of the total station; 4) focusing, aiming at a
target, and setting three-dimensional coordinates of the observation station point; 5) setting
coordinates of a rear view point, automatically calculating an azimuth angle of a rear view
direction by the total station at the moment, and setting a reading number of a horizontal
circle of the rear view direction as the azimuth angle; 6) setting a prism constant, and setting
an atmospheric correction value or an air temperature value and an air pressure value; 7)
measuring an instrument height and a prism height and inputting them into the total station; 8) aiming at a center of the target, pressing a coordinate measurement key, and making the total station start to measure a distance and calculate and display three-dimensional coordinates of a measurement point; and 9) recording the measured three-dimensional coordinates of the target and the ID number of the corresponding target.
In 102, whether the depth camera device can collect the target image is judged, and if
the depth camera device can collect the target image, step 103 is implemented; and if the
depth camera device cannot collect the target image, step 104 is implemented.
In 103, positioning is carried out by the depth camera device 200.
With the advancement of face mining, the end device 400 moves forward continuously,
the depth camera device 200 enters calibration areas (as shown in a position pl and a position
p3 in FIG. 1), and the depth camera device 200 collects RGB image data and point cloud
depth data containing information of the target 100. The data are processed in a
microcomputer, the ID number of the target image 100a and camera coordinates of four
angular points of the target image 100a in the depth camera device coordinate system are
identified and detected, and absolute coordinates of the depth camera device 200 are
determined according to the absolute coordinates and the camera coordinates of the target
image 100a.
In 104, positioning is carried out by the tracking camera device 300.
If the depth camera device 200 does not collect the target image 100a, which means that
the depth camera device 200 is located in a blind area of calibration (as shown in a position
p2 in FIG 1), the positioning system automatically enters a self-positioning endurance stage,
and the tracking camera device 300 enables a tracking function at the moment. Since
positions of the tracking camera device 300 and the depth camera device are relatively fixed, absolute coordinates of the tracking camera device 300 relative to the absolute coordinate system are obtained, according to a conversion matrix between the depth camera device coordinate system and the absolute coordinate system and a conversion matrix between the tracking camera device coordinate system and the depth camera device coordinate system at the time when the tracking camera device 300 is turned on.
In the positioning system above of the present invention, the plurality of groups of
targets 100 are arranged along an advancing direction of the fully mechanized mining face,
and both of the depth camera device 200 and the tracking camera device 300 are arranged on
the end device 400 of the fully mechanized mining face, so that the absolute coordinates of
the depth camera device 200 may be determined by the absolute coordinates of the target
image 100a and the camera coordinates of the target image 100a relative to the depth camera
device 200, if the depth camera device 200 is in a calibration position where the target image
100a can be collected, thus positioning the fully mechanized mining face; and the fully
mechanized mining face is positioned by the tracking camera device 300 if the depth camera
device 200 is in a blind area where the target image 100a cannot be collected, thus obtaining
the continuous positioning information in a process of advancing the fully mechanized
mining face in the roadway 500, and being convenient for controlling the fully mechanized
mining face.
Specifically, since the target image 100a is rectangular, the step 101 of acquiring the
absolute coordinates of the target image 100a in the absolute coordinate system specifically
comprises acquiring absolute coordinates (x., y, zw) of a central point of the target image
100a.
Specifically, the step 103 of carrying out the positioning by the depth camera device 200
is achieved by the following specific steps.
In 1031, pixel coordinates of the four angular points of the target image 100a relative to
the depth camera device coordinate system are acquired, and pixel coordinates of the central
point of the target image 100a relative to the depth camera device coordinate system are
determined.
As shown in FIG. 2, in a clockwise direction, the pixel coordinates of the four angular
points of the target image 100a are sequentially (xo, yo), (xI, yi), (x2,y2) and (x3,y3), angular
points of opposite angles of the target image 100a may constitute two straight lines, which are
namely (xo, yo) and (x2, y2) as well as (xi, yi) and (x3, y3), an equation of the straight line may
be expressed as ax+by+c=0, and parameters of equations of the two straight lines are
respectively as follows:
S=Y2 - YU 3 = A -71
C = X2 * Yo -xu* Y2, and IC. =-x3' Y - X1 'Y3.
The pixel coordinates (x, y) of the central point of the target image 100a relative to the
depth camera device coordinate system are obtained by the following formula:
(X2Y)= * C2 s - bis •2 __ * 02 - _ aa *
\a@2 * bia - a13 , b@2'a@2 , bia - a13 , b021/.
In 1032, camera coordinates of the central point of the target image 100a relative to the
depth camera device coordinate system are obtained according to the pixel coordinates of the
central point of the target image 100a relative to the depth camera device coordinate system
and an internal reference matrix of the depth camera device 200.
The step 1032 above is achieved by the following formula:
[X ]af
Q0 1
wherein .
H is the internal reference matrix of the depth camera device 200, f represents a focal
length of the depth camera device 200, dx and dy respectively represent sizes of unit pixels
on a u axis and av axis of the depth camera device 200, and u and v represent an optical
center, which is namely an intersection between an optical axis of a camera and a plane of an
image.
(x, y) are the pixel coordinates of the central point of the target image 100a relative to
the depth camera device coordinate system, and (Xdc, ydc, Zdc) are the camera coordinates of
the central point of the target image 100a relative to the depth camera device coordinate
system, wherein the camera coordinates are three-dimensional coordinates.
In 1033, the conversion matrix between the depth camera device coordinate system and
the absolute coordinate system is determined according to the absolute coordinates and the
camera coordinates of the target image 100a.
The step 1033 above is achieved by the following formula:
Ydc _ Hd2w .
Zdc Z.
L1 L_1j
wherein (Xdc, ydc, Zdc) are the camera coordinates of the central point of the target image
100a relative to the depth camera device coordinate system, (xw, yw, Zw) are the absolute
coordinates of the central point of the target image 100a relative to the absolute coordinate system, and as shown in FIG. 3, Hd2w is the conversion matrix between the depth camera device coordinate system and the absolute coordinate system. Hd2w is a 3x4 matrix, so if camera coordinates and absolute coordinates of target centers of the three target images 100a are known, Hd2w may be obtained by solving simultaneous equations.
In 1034, the absolute coordinates of the depth camera device 200 are determined
according to the conversion matrix.
Since coordinates of a camera itself in depth camera device coordinate system are (0, 0,
), the step 1034 above is achieved by the following formula:
X" 0] Yd. =Hd 0
Zd d 0
wherein (xdw, yd., Zdw) are the absolute coordinates of the depth camera device 200
relative to the absolute coordinate system, and Hd2w is the conversion matrix between the
depth camera device coordinate system and the absolute coordinate system.
The absolute coordinates (xdw, yd., Zdw) of the depth camera device may be obtained by
cooperation of the depth camera device and the absolute coordinates of the three target
images. Positioning is carried out through detection simultaneously by a depth camera and an
inertial navigation device in the prior art, and a positioning precision of the inertial
navigation device decreases with time. In contrast, the absolute coordinates of the depth
camera device are calculated by the absolute coordinates of the three target images in the
present invention, so that a positioning precision is higher.
Specifically, the step 104 of carrying out the positioning by the tracking camera device
300 is achieved by the following specific steps.
In 1041, a conversion matrix between the tracking camera device coordinate system and
the absolute coordinate system is obtained, according to the conversion matrix between the
depth camera device coordinate system and the absolute coordinate system and the
conversion matrix between the tracking camera device coordinate system and the depth
camera device coordinate system at the time when the tracking camera device is turned on.
Since the tracking camera device 300 and the depth camera device are both fixedly
connected to the end device 400, and have a fixed positional relationship, the conversion
matrix between the tracking camera device coordinate system and the depth camera device
coordinate system may be obtained as follows:
H2=Rtd 1]4~4
The conversion matrix Hd2w between the depth camera device coordinate system and the
absolute coordinate system is obtained at the time when the tracking camera device is turned
on.
At the moment, the conversion matrix Ht2w between the tracking camera device
coordinate system and the absolute coordinate system is obtained by the following formula:
Ht2w=Hd2w-Ht2d.
In 1042, absolute coordinates of the tracking camera device 300 relative to the absolute
coordinate system are obtained according to the conversion matrix between the tracking
camera device coordinate system and the absolute coordinate system by the following
formula:
r1 - 1., wherein (xi., yt., zt.) are the absolute coordinates of the tracking camera device 300 relative to the absolute coordinate system, H2 is the conversion matrix between the tracking camera device coordinate system and the absolute coordinate system at the time when the tracking camera device is turned on, and (xio, yto, zio) are translation information of coordinates of the tracking camera device at a current time relative to the time when the tracking camera device is turned on.
Obviously, the above embodiments are only examples for clearly illustrating the present
invention, but are not intended to limit the implementations of the present invention. For
those of ordinary skills in the art, other different forms of changes or variations may be made
on the basis of the above description. It is not necessary or possible to exhaust all the
implementations herein. Moreover, the obvious changes or variations derived from this are
still included within the scope of protection of the present invention.
Claims (11)
1. A positioning method for a fully mechanized mining face, comprising:
providing a depth camera device and a tracking camera device on an end device of the
fully mechanized mining face, and arranging a plurality of groups of targets in a roadway of
the fully mechanized mining face, wherein adjacent groups of targets have a set interval
therebetween, and each group of targets comprises at least three target images;
acquiring absolute coordinates of the target image in an absolute coordinate system;
if the depth camera device collects a target image of one group of targets, determining
camera coordinates of the target image in a depth camera device coordinate system; and
determining absolute coordinates of the depth camera device according to the absolute
coordinates and the camera coordinates of the target image; and
if the depth camera device fails to collect the target image, controlling the tracking
camera device to turn on, and obtaining absolute coordinates of the tracking camera device
relative to the absolute coordinate system according to a conversion matrix between the depth
camera device coordinate system and the absolute coordinate system and a conversion matrix
between a tracking camera device coordinate system and the depth camera device coordinate
system at the time when the tracking camera device is turned on.
2. The positioning method for the fully mechanized mining face according to claim 1,
wherein step of if the depth camera device collects the target image, determining the camera
coordinates of the target image in the depth camera device coordinate system specifically
comprises:
the target image being rectangular, acquiring pixel coordinates of four angular points of
the target image relative to the depth camera device coordinate system, and determining pixel coordinates of a central point of the target image relative to the depth camera device coordinate system; and obtaining camera coordinates of the central point of the target image relative to the depth camera device coordinate system according to an internal reference matrix of the depth camera device.
3. The positioning method for the fully mechanized mining face according to claim 2,
wherein step of acquiring the pixel coordinates of the four angular points of the target image
relative to the depth camera device coordinate system, and determining the pixel coordinates
of the central point of the target image relative to the depth camera device coordinate system
is achieved by the following formula:
( =bO2 eci; - bi3 " c02 ais e cUr - aU2 *, ci31 u2 b 13 - 13 " N2 au2 , bls - a13 1 12
ia 02 Y2 ~ A A Y1
wherein 2 =-X2 *YU - XG *Y2, C1i=X3 ' YI -1Xi1 I, and
(xo, yo), (x, yI), (x2, y2) and (x3, y3) are the pixel coordinates of the four angular points of
the target image in a clockwise or counterclockwise direction relative to the depth camera
device coordinate system; and (x, y) are the pixel coordinates of the central point of the target
image relative to the depth camera device coordinate system.
4. The positioning method for the fully mechanized mining face according to claim 3,
wherein step of obtaining the camera coordinates of the central point of the target image
relative to the depth camera device coordinate system according to the internal reference
matrix of the depth camera device is achieved by the following formula:
1 , wherein (x, y) are the pixel coordinates of the central point of the target image relative to the depth camera device coordinate system, (Xdc,dc,Zdc) are the camera coordinates of the central point of the target image relative to the depth camera device coordinate system, and H is the internal reference matrix of the depth camera device.
5. The positioning method for the fully mechanized mining face according to claim 2,
wherein step of determining the absolute coordinates of the depth camera device according to
the absolute coordinates and the camera coordinates of the target image comprises:
determining the conversion matrix between the depth camera device coordinate system
and the absolute coordinate system according to the absolute coordinates and the camera
coordinates of the target image, and determining the absolute coordinates of the depth camera
device according to the conversion matrix.
6. The positioning method for the fully mechanized mining face according to claim 5,
wherein step of determining the conversion matrix between the depth camera device
coordinate system and the absolute coordinate system according to the absolute coordinates
and the camera coordinates of the target image is achieved by the following formula:
H dc _ Hd2w* Zdc Zw"
wherein (Xdc,Ydc, Zdc) are the camera coordinates of the central point of the target image
relative to the depth camera device coordinate system, (xi, yi, zw) are the absolute
coordinates of the central point of the target image relative to the absolute coordinate system,
and Hd2w1 is the conversion matrix between the depth camera device coordinate system and the
absolute coordinate system.
7. The positioning method for the fully mechanized mining face according to claim 5,
wherein step of determining the absolute coordinates of the depth camera device according to
the conversion matrix is achieved by the following formula:
X" 0] Yd. =H • 0
Zd d 0
wherein (xdw, yd., Zdw) are the absolute coordinates of the depth camera device relative to
the absolute coordinate system, and Hd2 is the conversion matrix between the depth camera
device coordinate system and the absolute coordinate system.
8. The positioning method for the fully mechanized mining face according to claim 1,
wherein step of obtaining the absolute coordinates of the tracking camera device relative to
the absolute coordinate system according to the conversion matrix between the depth camera
device coordinate system and the absolute coordinate system and the conversion matrix
between the tracking camera device coordinate system and the depth camera device
coordinate system at the time when the tracking camera device is turned on comprises:
obtaining a conversion matrix between the tracking camera device coordinate system
and the absolute coordinate system according to the conversion matrix between the depth
camera device coordinate system and the absolute coordinate system and the conversion
matrix between the tracking camera device coordinate system and the depth camera device
coordinate system at the time when the tracking camera device is turned on, and obtaining the
absolute coordinates of the tracking camera device relative to the absolute coordinate system
according to the conversion matrix between the tracking camera device coordinate system
and the absolute coordinate system.
9. The positioning method for the fully mechanized mining face according to claim 8,
wherein step of obtaining the conversion matrix between the tracking camera device
coordinate system and the absolute coordinate system according to the conversion matrix
between the depth camera device coordinate system and the absolute coordinate system and
the conversion matrix between the tracking camera device coordinate system and the depth
camera device coordinate system at the time when the tracking camera device is turned on is
achieved by the following formula:
Ht2,=Hd2w-Ht2d,
wherein Ht2, is the conversion matrix between the tracking camera device coordinate
system and the absolute coordinate system, Hd2w is the conversion matrix between the depth
camera device coordinate system and the absolute coordinate system at a current time, and
Ht2d is the conversion matrix between the tracking camera device coordinate system and the
depth camera device coordinate system.
10. The positioning method for the fully mechanized mining face according to claim 8,
wherein step of obtaining the absolute coordinates of the tracking camera device relative to
the absolute coordinate system according to the conversion matrix between the tracking
camera device coordinate system and the absolute coordinate system is achieved by the
following formula:
wherein (xtw, ytw, ztw) are the absolute coordinates of the tracking camera device relative
to the absolute coordinate system, H2. is the conversion matrix between the tracking camera
device coordinate system and the absolute coordinate system, and (xto, yto, zto) are translation information of coordinates of the tracking camera device at a current time relative to the time when the tracking camera device is turned on.
11. A positioning system for a fully mechanized mining face, comprising: a depth camera device and a tracking camera device fixed on an end device, and a plurality of groups of targets arranged in a roadway of the fully mechanized mining face, wherein adjacent groups of targets have a set interval therebetween, and each group of targets comprises at least three target images; wherein the positioning method according to any one of claims 1 to 10 is performed in the positioning system.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011042486.7 | 2020-09-28 | ||
| CN202011042486.7A CN112161567B (en) | 2020-09-28 | 2020-09-28 | Positioning method and system for fully mechanized coal mining face |
| PCT/CN2021/102661 WO2022062521A1 (en) | 2020-09-28 | 2021-06-28 | Positioning method and system for fully mechanized mining face |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| AU2021350657A1 true AU2021350657A1 (en) | 2023-04-13 |
Family
ID=73861975
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| AU2021350657A Pending AU2021350657A1 (en) | 2020-09-28 | 2021-06-28 | Positioning method and system for fully mechanized mining face |
Country Status (3)
| Country | Link |
|---|---|
| CN (1) | CN112161567B (en) |
| AU (1) | AU2021350657A1 (en) |
| WO (1) | WO2022062521A1 (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112161567B (en) * | 2020-09-28 | 2022-05-03 | 北京天玛智控科技股份有限公司 | Positioning method and system for fully mechanized coal mining face |
| CN112985374B (en) * | 2021-02-09 | 2023-04-21 | 北京天玛智控科技股份有限公司 | Positioning method, positioning assembly and positioning system |
| CN114299409B (en) * | 2022-03-07 | 2022-07-01 | 中煤科工开采研究院有限公司 | Underground personnel positioning method and device based on image recognition and depth information |
| CN114993273A (en) * | 2022-05-19 | 2022-09-02 | 中铁隆工程集团有限公司 | Engineering device terminal identification, tracking and positioning method, system and application |
| CN115468510A (en) * | 2022-09-26 | 2022-12-13 | 中车长春轨道客车股份有限公司 | Detection system, detection method and device for cross section profile of rail transit vehicle |
| CN115343299B (en) * | 2022-10-18 | 2023-03-21 | 山东大学 | Lightweight highway tunnel integrated detection system and method |
| CN117665780B (en) * | 2024-01-31 | 2024-04-19 | 武汉大学 | Laser scanner system error calibration target selection method, system and storage medium |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8229166B2 (en) * | 2009-07-07 | 2012-07-24 | Trimble Navigation, Ltd | Image-based tracking |
| CN102878976A (en) * | 2012-09-26 | 2013-01-16 | 三一重型装备有限公司 | Heading machine position and posture detecting system and heading machine |
| CN104236540B (en) * | 2014-06-24 | 2017-06-23 | 上海大学 | indoor passive navigation locating method |
| CN106908000B (en) * | 2017-01-20 | 2019-05-17 | 中国矿业大学 | A kind of measuring system and method for hydraulic support top beam bearing height and posture |
| US11064184B2 (en) * | 2017-08-25 | 2021-07-13 | Aurora Flight Sciences Corporation | Aerial vehicle imaging and targeting system |
| US10895460B2 (en) * | 2017-11-06 | 2021-01-19 | Cybernet Systems Corporation | System and method for generating precise road lane map data |
| JPWO2019188718A1 (en) * | 2018-03-28 | 2020-04-30 | バンドー化学株式会社 | Surface shape monitoring device, wear amount measuring system and surface shape monitoring system |
| CN108694381A (en) * | 2018-05-11 | 2018-10-23 | 星络科技有限公司 | Object positioning method and object trajectory method for tracing |
| CN108742841B (en) * | 2018-05-30 | 2020-11-06 | 上海交通大学 | Tool real-time positioning device of multi-position tracker |
| CN109974693B (en) * | 2019-01-31 | 2020-12-11 | 中国科学院深圳先进技术研究院 | Unmanned aerial vehicle positioning method and device, computer equipment and storage medium |
| CN110700840B (en) * | 2019-10-21 | 2021-10-22 | 北京易联创安科技发展有限公司 | Position and posture measuring method of heading machine based on laser target positioning |
| CN110941239B (en) * | 2019-12-17 | 2023-02-28 | 中国矿业大学 | Deep mine environment monitoring robot system and monitoring method |
| CN111553948B (en) * | 2020-04-27 | 2023-01-17 | 冀中能源峰峰集团有限公司 | Heading machine cutting head positioning method based on double tracers |
| CN111645074A (en) * | 2020-06-01 | 2020-09-11 | 李思源 | Robot grabbing and positioning method |
| CN112161567B (en) * | 2020-09-28 | 2022-05-03 | 北京天玛智控科技股份有限公司 | Positioning method and system for fully mechanized coal mining face |
-
2020
- 2020-09-28 CN CN202011042486.7A patent/CN112161567B/en active Active
-
2021
- 2021-06-28 AU AU2021350657A patent/AU2021350657A1/en active Pending
- 2021-06-28 WO PCT/CN2021/102661 patent/WO2022062521A1/en active Application Filing
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022062521A1 (en) | 2022-03-31 |
| CN112161567B (en) | 2022-05-03 |
| CN112161567A (en) | 2021-01-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2021350657A1 (en) | Positioning method and system for fully mechanized mining face | |
| CN1788188B (en) | Picked-up image display method and device | |
| GB2620877A (en) | On-board positioning device-based roadside millimeter-wave radar calibration method | |
| CN104217439A (en) | Indoor visual positioning system and method | |
| CN105371847A (en) | Indoor live-action navigation method and system | |
| CN108775901B (en) | Real-time SLAM scene map construction system, navigation system and method | |
| KR101880185B1 (en) | Electronic apparatus for estimating pose of moving object and method thereof | |
| KR102006291B1 (en) | Method for estimating pose of moving object of electronic apparatus | |
| CN106370160A (en) | Robot indoor positioning system and method | |
| CN110108269A (en) | AGV localization method based on Fusion | |
| CN105116886A (en) | Robot autonomous walking method | |
| CN112985374B (en) | Positioning method, positioning assembly and positioning system | |
| CN109387194A (en) | A kind of method for positioning mobile robot and positioning system | |
| CN111221020A (en) | Indoor and outdoor positioning method, device and system | |
| CN110488838A (en) | A kind of interior independent navigation robot accurately repeats localization method | |
| KR100564236B1 (en) | Self-localization apparatus and method of mobile robot | |
| CN114485654A (en) | Multi-sensor fusion positioning method and device based on high-precision map | |
| CN115027482A (en) | Fusion positioning method in intelligent driving | |
| CN111402702A (en) | Map construction method, device and system | |
| CN117310627A (en) | Combined calibration method applied to vehicle-road collaborative road side sensing system | |
| CN112857367B (en) | Heading machine pose detection method based on machine vision and inertial navigation | |
| CN112884832B (en) | Intelligent trolley track prediction method based on multi-view vision | |
| CN114966793A (en) | Three-dimensional measurement system, method and GNSS system | |
| CN112798020A (en) | System and method for evaluating positioning accuracy of intelligent automobile | |
| CN114543807B (en) | High-precision evaluation method of SLAM algorithm in extreme scene |