CN109839109A - The absolute position and posture detection method of development machine based on image recognition and Multi-sensor Fusion - Google Patents
The absolute position and posture detection method of development machine based on image recognition and Multi-sensor Fusion Download PDFInfo
- Publication number
- CN109839109A CN109839109A CN201910139486.XA CN201910139486A CN109839109A CN 109839109 A CN109839109 A CN 109839109A CN 201910139486 A CN201910139486 A CN 201910139486A CN 109839109 A CN109839109 A CN 109839109A
- Authority
- CN
- China
- Prior art keywords
- gyroscope
- development machine
- video camera
- working face
- measured
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Gyroscopes (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The present invention proposes the absolute position and posture detection method of development machine based on image recognition and Multi-sensor Fusion, and this method installs laser indicator lamp on machine body of boring machine, and laser indicator lamp projects target pattern on working face；Video camera is installed in the left and right driving arm and fuselage of development machine, three video cameras acquire target pattern simultaneously；Gyroscope is installed respectively at the left and right driving arm of development machine and fuselage center, gyroscope acquires three axis angular rates and rotation amount under local Coordinate System；The displacement that displacement sensor acquisition development machine is displaced to working face is installed in development machine electric cabinet；According to acquisition data, the equivalent constraint equation of pixel coordinate of the same characteristic point in the left and right imaging plane of stereo camera, the three-dimensional coordinate of real-time resolving characteristic point are established, and then realizes positioning of the fuselage in work space of planes.
Description
Technical field
The present invention relates to mine down-hole boom-type roadheader pose detection technique fields, especially a kind of to be based on image recognition
With the absolute position and posture detection method of development machine of Multi-sensor Fusion.
Background technique
With the raising of mining mechanization level, the standard that exploitation has become relevant enterprise is carried out using boom-type roadheader
Production method.Under current nation-building digital mine, the call for promoting dangerous production environment unmanned, in recent years, part
Coal mine starts to draw using vibration and the vibration of temperature sensor detection development machine and temperature parameter using laser sensor and video
Face instructs surface personnel to operate underground development machine, although time of staff into a line operation can be reduced
Number, but the degree of automation is lower.Some researchs resolve multiple using multi-sensor collection displacement, posture information, embedded system
Miscellaneous equation solution pose improves the degree of automation, but due to sensing data there are error and data volume causes greatly accurately
It spends low.Therefore, it designs a set of development machine intelligent positioning detection method suitable for mine environment to be necessary, with computer graphic
As the promotion of identification technology, the biggish manless working face of data volume is more and more to obtain target information using image recognition.In view of
The complex environment of coal mining development machine working face can be increased accurately using the method that image recognition and multisensor blend
Property and stability, meet the requirement of modern Mine digitalization development.
Summary of the invention
Goal of the invention: it is an object of the invention to propose a set of development machine intelligent positioning detection side suitable for mine environment
Method.
Technical solution: to achieve the above object, the present invention proposes following technical scheme:
The absolute position and posture detection method of development machine based on image recognition and Multi-sensor Fusion, comprising steps of
(1) defining the face that development machine to be excavated is working face；Machine body of boring machine towards working face one side center
Place's installation laser indicator lamp, laser indicator lamp project target pattern on working face；Development machine left driving arm towards work
Make that left video camera is installed in the one side in face, right video camera, left and right video camera are installed in one side of the right driving arm towards working face
Target pattern on collecting work face simultaneously；Displacement sensor is installed in development machine electric cabinet, displacement sensor is for acquiring
The displacement that development machine is displaced to working face in unit time；Pacify respectively at the left and right driving arm of development machine and fuselage center
Gyroscope is filled, three gyroscopes export three axis angular rates and rotation amount under local Coordinate System respectively；
(2) some pixel in cruciform pattern is chosen as characteristic point；Characteristic point is calculated separately in left and right video camera
Pixel coordinate in imaging plane；
(3) characteristic point is established in the equivalent constraint equation of left and right camera coordinate system:
AX=0
X=[x x ']
Wherein, A is characterized the homogeneous coordinates matrix a little in left video camera imaging coordinate system, and x is characterized a little in left camera shooting
Pixel coordinate in machine imaging plane, x ' are characterized the pixel coordinate a little in the right imaging plane of video camera, and p is left video camera
Projection matrix, p ' are the projection matrix of right video camera；
(4) it is calculated according to characteristic point in the equivalent constraint equation of left and right camera coordinate system:
Wherein, B indicates that coordinate of the characteristic point in machine body of boring machine coordinate system, θ are left and right video camera photographing lens axis
Between deflection angle；
(5) development machine is calculated separately where three gyroscopes according to the measurement result of displacement sensor and three gyroscopes
Posture equation at position, wherein the posture Equation f at i-th of gyroscope are as follows:
Wherein, Δ y is the displacement that development machine is displaced to working face in the unit time of displacement sensor out, miFor
The measurement result vector of i-th of gyroscope, mi=[m1, m2, m3, m4, m5, m6, m7], m1The angle speed of x-axis rotation is measured for gyroscope
Degree, m2The angular speed of y-axis rotation, m are measured for gyroscope3The angular speed of z-axis rotation, m are measured for gyroscope4It is measured for gyroscope
Component of the rotation amount vector section in x-axis, m5Rotation amount vector section is measured in the component of y-axis, m for gyroscope6For gyroscope survey
Obtain component of the rotation amount vector section in z-axis, m7Rotation amount scalar, τ are measured for gyroscopeiTime for i-th of gyroscope is normal
Number, TsFor the preset gyroscope sampling period.
Further, the left and right video camera is Mine-used I. S video camera.
Further, three gyroscopes are contained in mine explosion-suppression case.
The utility model has the advantages that compared with prior art, present invention has the advantage that
1, the absolute pose of real-time resolving coalcutter is combined using image, displacement and acceleration information, compares traditional sensors
Perception increases accuracy and stability, meets the requirement of modern Mine digitalization development.
2, the camera angles evaluation method based on polynomial fitting is designed, opposite deflection angle is selected to possess video camera most
It is big to be overlapped the visual field, and then three, fuselage reconstruct is made to possess biggish dynamic range.
Specific embodiment
The present invention will be further explained combined with specific embodiments below.
The present embodiment uses following hardware device: three Mine-used I. S video cameras, Cross hair laser indicator light, one
Platform displacement sensor, three electronic gyroscopes, hardware device are provided with wireless communication function, the data of acquisition are periodically passed through nothing
Gauze network passes to background computer；Three Mine-used I. S video cameras be separately mounted on the left and right driving arm of development machine with
And mouth is transported at rear, acquires image with the rate of 30 frame per second；Cross hair laser indicator light is installed on development machine anteriomedial position
It sets；Displacement sensor is mounted in development machine electric cabinet, every 1s transmission primaries displacement signal.Three electronic gyroscopes are sealed in
In mine explosion-suppression case, wherein two are separately mounted on the left and right driving arm of development machine, another is mounted on the several of machine body of boring machine
What center.
Workflow of the invention is as follows:
(1) the note face to be excavated of development machine is working face；In machine body of boring machine towards at the center of the one side of working face
Cross hair laser indicator light is installed, Cross hair laser indicator light projects cruciform pattern on working face；Development machine or so two
The cruciform pattern on video camera while collecting work face on arm；Displacement sensor in development machine electric cabinet acquires development machine
The displacement being displaced to working face；Local Coordinate System is acquired respectively in two gyroscopes for tunneling arm and fuselage center of development machine
Under 3-axis acceleration and attitude quaternion；
(2) after background computer receives acquisition data, following steps are executed:
(2-1) chooses some pixel in cruciform pattern as characteristic point；Characteristic point is calculated separately in left and right camera shooting
Pixel coordinate in machine imaging plane；
(2-2) establishes characteristic point in the equivalent constraint equation of left and right camera coordinate system:
AX=0
X=[x x ']
Wherein, A is characterized the homogeneous coordinates matrix a little in left video camera imaging coordinate system, and x is characterized a little in left camera shooting
Pixel coordinate in machine imaging plane, x ' are characterized the pixel coordinate a little in the right imaging plane of video camera, and p is left video camera
Projection matrix, p ' are the projection matrix of right video camera；
(2-3), in the equivalent constraint equation of left and right camera coordinate system, is calculated according to characteristic point:
And then it calculates:
Wherein, wherein B indicates that coordinate of the characteristic point in machine body of boring machine coordinate system, θ are that left and right video camera shoots mirror
Deflection angle between head axis；
(2-4) calculates the posture Equation f of development machine according to the measurement result of displacement sensor and three gyroscopes:
Wherein, Δ y is the displacement that development machine is displaced to working face in the unit time of displacement sensor out, miFor
The measurement result vector of i-th of gyroscope, mi=[m1, m2, m3, m4, m5, m6, m7], m1The angle speed of x-axis rotation is measured for gyroscope
Degree, m2The angular speed of y-axis rotation, m are measured for gyroscope3The angular speed of z-axis rotation, m are measured for gyroscope4It is measured for gyroscope
Component of the rotation amount vector section in x-axis, m5Rotation amount vector section is measured in the component of y-axis, m for gyroscope6For gyroscope survey
Obtain component of the rotation amount vector section in z-axis, m7Rotation amount scalar, τ are measured for gyroscopeiTime for i-th of gyroscope is normal
Number, TsFor the preset gyroscope sampling period.
The above is only a preferred embodiment of the present invention, it should be pointed out that: for the ordinary skill people of the art
For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also answered
It is considered as protection scope of the present invention.
Claims (3)
1. the absolute position and posture detection method of development machine based on image recognition and Multi-sensor Fusion, which is characterized in that comprising steps of
(1) defining the face that development machine to be excavated is working face；Pacify in machine body of boring machine towards at the center of the one side of working face
Laser indicator lamp is filled, laser indicator lamp projects target pattern on working face；Development machine left driving arm towards working face
One side on left video camera is installed, right video camera is installed in right one side of the driving arm towards working face, left and right video camera is simultaneously
Target pattern on collecting work face；Displacement sensor is installed in development machine electric cabinet, displacement sensor is used for acquisition units
The displacement that development machine is displaced to working face in time；Top is installed respectively at the left and right driving arm of development machine and fuselage center
Spiral shell instrument, three gyroscopes export three axis angular rates and rotation amount under local Coordinate System respectively；
(2) some pixel in target pattern is chosen as characteristic point；It is flat in left and right video camera imaging to calculate separately characteristic point
Pixel coordinate in face；
(3) characteristic point is established in the equivalent constraint equation of left and right camera coordinate system:
AX=0
X=[x x ']
Wherein, A is characterized the homogeneous coordinates matrix a little in left video camera imaging coordinate system, x be characterized a little left video camera at
As the pixel coordinate in plane, x ' is characterized the pixel coordinate a little in the right imaging plane of video camera, and p is the projection of left video camera
Matrix, p ' are the projection matrix of right video camera；
(4) it is calculated according to characteristic point in the equivalent constraint equation of left and right camera coordinate system:
Wherein, B indicates coordinate of the characteristic point in machine body of boring machine coordinate system, and θ is between left and right video camera photographing lens axis
Deflection angle；
(5) development machine is calculated separately three gyroscope positions according to the measurement result of displacement sensor and three gyroscopes
The posture equation at place, wherein the posture Equation f at i-th of gyroscope are as follows:
Wherein, Δ y is the displacement that development machine is displaced to working face in the unit time of displacement sensor out, miIt is i-th
The measurement result vector of gyroscope, mi=[m1, m2, m3, m4, m5, m6, m7], m1The angular speed of x-axis rotation, m are measured for gyroscope2
The angular speed of y-axis rotation, m are measured for gyroscope3The angular speed of z-axis rotation, m are measured for gyroscope4Rotation amount is measured for gyroscope
Component of the vector section in x-axis, m5Rotation amount vector section is measured in the component of y-axis, m for gyroscope6Rotation is measured for gyroscope
Measure component of the vector section in z-axis, m7Rotation amount scalar, τ are measured for gyroscopeiFor the time constant of i-th of gyroscope, TsFor
The preset gyroscope sampling period.
2. the absolute position and posture detection method of the development machine according to claim 1 based on image recognition and Multi-sensor Fusion,
It is characterized in that, the left and right video camera is Mine-used I. S video camera.
3. the absolute position and posture detection method of the development machine according to claim 1 based on image recognition and Multi-sensor Fusion,
It is characterized in that, three gyroscopes are contained in mine explosion-suppression case.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910139486.XA CN109839109B (en) | 2019-02-25 | 2019-02-25 | Development machine absolute pose detection method based on image recognition and multi-sensor fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910139486.XA CN109839109B (en) | 2019-02-25 | 2019-02-25 | Development machine absolute pose detection method based on image recognition and multi-sensor fusion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109839109A true CN109839109A (en) | 2019-06-04 |
| CN109839109B CN109839109B (en) | 2020-11-24 |
Family
ID=66884961
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910139486.XA Active CN109839109B (en) | 2019-02-25 | 2019-02-25 | Development machine absolute pose detection method based on image recognition and multi-sensor fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109839109B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111879314A (en) * | 2020-08-10 | 2020-11-03 | 中国铁建重工集团股份有限公司 | Multi-sensor fusion roadway driving equipment real-time positioning system and method |
| CN112050732A (en) * | 2020-08-21 | 2020-12-08 | 西安科技大学 | Method and system for automatically detecting spatial pose of cantilever type heading machine |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201273195Y (en) * | 2008-10-16 | 2009-07-15 | 中国矿业大学 | Position detection apparatus of fully-mechanized mining face coal mining machine |
| EP2189753A2 (en) * | 2008-11-24 | 2010-05-26 | Politechnika Slaska | Assembly stand for mineral cutting head |
| CN201568056U (en) * | 2009-12-17 | 2010-09-01 | 中国矿业大学 | Orientation laser intelligent receiving target |
| CN104729501A (en) * | 2015-03-19 | 2015-06-24 | 中国矿业大学(北京) | Rotating-sector-laser-based position and pose measurement method of cantilever excavator |
| CN105115499A (en) * | 2015-09-09 | 2015-12-02 | 中船重工（青岛）轨道交通装备有限公司 | Guide system applied to double-shield tunneling machine and positioning method |
| CN105136134A (en) * | 2015-08-10 | 2015-12-09 | 兖州煤业股份有限公司 | Heading machine position and posture detection and adjustment method and system |
| CN107014379A (en) * | 2017-05-25 | 2017-08-04 | 中国矿业大学 | A kind of development machine absolute space apparatus for detecting position and posture and method |
| CN108592951A (en) * | 2018-05-30 | 2018-09-28 | 中国矿业大学 | A kind of coalcutter inertial navigation Initial Alignment Systems and method based on optical flow method |
| CN108731672A (en) * | 2018-05-30 | 2018-11-02 | 中国矿业大学 | Coalcutter attitude detection system and method based on binocular vision and inertial navigation |
| CN109115172A (en) * | 2018-08-23 | 2019-01-01 | 西安科技大学 | Coalcutter fuselage position and posture detection method based on machine vision |
-
2019
- 2019-02-25 CN CN201910139486.XA patent/CN109839109B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201273195Y (en) * | 2008-10-16 | 2009-07-15 | 中国矿业大学 | Position detection apparatus of fully-mechanized mining face coal mining machine |
| EP2189753A2 (en) * | 2008-11-24 | 2010-05-26 | Politechnika Slaska | Assembly stand for mineral cutting head |
| CN201568056U (en) * | 2009-12-17 | 2010-09-01 | 中国矿业大学 | Orientation laser intelligent receiving target |
| CN104729501A (en) * | 2015-03-19 | 2015-06-24 | 中国矿业大学(北京) | Rotating-sector-laser-based position and pose measurement method of cantilever excavator |
| CN105136134A (en) * | 2015-08-10 | 2015-12-09 | 兖州煤业股份有限公司 | Heading machine position and posture detection and adjustment method and system |
| CN105115499A (en) * | 2015-09-09 | 2015-12-02 | 中船重工（青岛）轨道交通装备有限公司 | Guide system applied to double-shield tunneling machine and positioning method |
| CN107014379A (en) * | 2017-05-25 | 2017-08-04 | 中国矿业大学 | A kind of development machine absolute space apparatus for detecting position and posture and method |
| WO2018214730A1 (en) * | 2017-05-25 | 2018-11-29 | 中国矿业大学 | Device and method for detecting absolute spatial orientation of roadheader |
| CN108592951A (en) * | 2018-05-30 | 2018-09-28 | 中国矿业大学 | A kind of coalcutter inertial navigation Initial Alignment Systems and method based on optical flow method |
| CN108731672A (en) * | 2018-05-30 | 2018-11-02 | 中国矿业大学 | Coalcutter attitude detection system and method based on binocular vision and inertial navigation |
| CN109115172A (en) * | 2018-08-23 | 2019-01-01 | 西安科技大学 | Coalcutter fuselage position and posture detection method based on machine vision |
Non-Patent Citations (4)
| Title |
|---|
| JORGE LOBO 等: "Relative Pose Calibration Between Visual and Inertial Sensors", 《THE INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH》 * |
| 刘兴堂 等: "《精确制导武器与精确制导控制技术》", 31 August 2009 * |
| 田原: "悬臂式掘进机导航技术现状及其发展方向", 《工矿自动化》 * |
| 黄东: "基于视觉/惯导的掘进机实时位姿测量方法研究", 《激光技术》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111879314A (en) * | 2020-08-10 | 2020-11-03 | 中国铁建重工集团股份有限公司 | Multi-sensor fusion roadway driving equipment real-time positioning system and method |
| CN112050732A (en) * | 2020-08-21 | 2020-12-08 | 西安科技大学 | Method and system for automatically detecting spatial pose of cantilever type heading machine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109839109B (en) | 2020-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113140040B (en) | Multi-sensor fusion underground coal mine space positioning and mapping method and device | |
| CN108731672B (en) | Coal mining machine attitude detection system and method based on binocular vision and inertial navigation | |
| CN110009739A (en) | The extraction and coding method of the motion feature of the digital retina of mobile camera | |
| CN103591955B (en) | Integrated navigation system | |
| CN107990876A (en) | The quick scanning means in underground mine goaf and method based on unmanned vehicle | |
| CN107289910B (en) | Optical flow positioning system based on TOF | |
| CN109974715B (en) | Tunneling machine autonomous navigation system and method combining strapdown inertial navigation and light spot identification | |
| CN103983262A (en) | Determination method and apparatus for advancing route based on optical communication | |
| CN106989773B (en) | A kind of attitude transducer and posture renewal method | |
| CN110864692A (en) | Pose determination method of heading machine | |
| CN106225779B (en) | Development machine positioning system and localization method based on three laser labelling point images | |
| CN112476433A (en) | Mobile robot positioning method based on array boundary identification | |
| CN108106613B (en) | Positioning method and system based on visual assistance | |
| CN109631938A (en) | Development machine autonomous positioning orientation system and method | |
| CN112697131B (en) | Underground mobile equipment positioning method and system based on vision and inertial navigation system | |
| KR20110064712A (en) | Motion capture system using integrated sensor system | |
| CN109839109A (en) | The absolute position and posture detection method of development machine based on image recognition and Multi-sensor Fusion | |
| CN207095572U (en) | A kind of hardware platform for being used for helmet attitude measurement in flight system | |
| CN110162098A (en) | A kind of mining unmanned plane | |
| CN108444468A (en) | The bearing compass of vision and inertial navigation information is regarded under a kind of fusion | |
| CN112067058A (en) | Automatic monitoring equipment for detecting karst channel and use method | |
| CN111199576A (en) | Outdoor large-range human body posture reconstruction method based on mobile platform | |
| CN109974696A (en) | The indoor occupant autonomic positioning method merged based on SLAM with gait IMU | |
| CN110411443A (en) | A kind of rocker arm of coal mining machine inertia/visual combination determines appearance device and method | |
| CN210268669U (en) | Inertial navigation positioning system for underground mining mobile equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |