CN112161571B - Low-data-volume binocular vision coal mining machine positioning and pose detection system and method - Google Patents

Low-data-volume binocular vision coal mining machine positioning and pose detection system and method Download PDF

Info

Publication number
CN112161571B
CN112161571B CN202011070727.9A CN202011070727A CN112161571B CN 112161571 B CN112161571 B CN 112161571B CN 202011070727 A CN202011070727 A CN 202011070727A CN 112161571 B CN112161571 B CN 112161571B
Authority
CN
China
Prior art keywords
positioning
binocular camera
coal mining
mining machine
camera module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011070727.9A
Other languages
Chinese (zh)
Other versions
CN112161571A (en
Inventor
曾庆良
徐文乾
高魁东
孙利青
万丽荣
张鑫
张晓迪
王亮
姜考
刘志海
王成龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University of Science and Technology
Original Assignee
Shandong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University of Science and Technology filed Critical Shandong University of Science and Technology
Priority to CN202011070727.9A priority Critical patent/CN112161571B/en
Publication of CN112161571A publication Critical patent/CN112161571A/en
Application granted granted Critical
Publication of CN112161571B publication Critical patent/CN112161571B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/26—Measuring arrangements characterised by the use of optical techniques for measuring angles or tapers; for testing the alignment of axes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Geometry (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention discloses a low-data-volume binocular vision coal mining machine positioning and pose detection system and method, which comprises an LED mark unit, a vision positioning detection unit and an upper computer; the LED mark unit comprises a positioning mark point and a registration mark point, the visual positioning detection unit comprises a plurality of binocular camera modules, and two video cameras and a processor are integrated in the binocular camera modules. In the walking process of the coal mining machine, a camera is used for identifying positioning mark points arranged on the coal mining machine, so that the positioning and pose detection of the coal mining machine are realized; the processor analyzes the position of the coal mining machine in the image and selects to start the next camera for taking a picture or stop taking a picture by itself; and between adjacent cameras, the coordinate system is converted by registering the mark points. The invention effectively reduces the data volume transmitted between the binocular camera module and the upper computer, has low transformation cost, high acquisition speed and high detection precision, provides important guarantee for realizing unmanned mining of mines and has better application prospect.

Description

Low-data-volume binocular vision coal mining machine positioning and pose detection system and method
Technical Field
The invention relates to a low-data-volume binocular vision coal mining machine positioning and pose detection system and method, and belongs to the technical field of coal mining machine positioning detection.
Background
The fully mechanized coal mining face mining is an important link of underground coal mining, the drum shearer is a main device for mining coal on the fully mechanized coal mining face, and in order to accelerate the development of coal mine intellectualization and reduce the number of workers on the working face, equipment is required to replace manpower to detect the position and the posture of the shearer in real time, so that the production can be carried out smoothly.
At present, in the actual production process of a coal mine, the positioning and pose angle detection of a coal mining machine are mainly carried out by using a manual observation method, a large number of workers are used, and the work trend of less humanization of coal mine production is violated.
At present, most of intelligent coal mines adopt a laser SLAM technology, the generated data volume is huge, more useless information is provided, and the complex communication conditions of the fully mechanized coal mining face cause great difficulty in transmitting the data. Therefore, it is necessary to provide a system and a method for positioning a coal mining machine and detecting the pose of the coal mining machine in real time with low data volume.
Disclosure of Invention
The purpose of the invention is as follows: in order to overcome the defects in the prior art, the invention provides the binocular vision coal mining machine positioning and pose detection system and method with low data volume, which effectively reduce the data volume transmitted between the binocular camera module and the upper computer, have low modification cost and high detection precision and provide important guarantee for realizing unmanned mining of mines.
The technical scheme is as follows: in order to achieve the purpose, the invention adopts the technical scheme that:
a binocular vision coal mining machine positioning and pose detection system with low data volume comprises an LED mark unit, a vision positioning detection unit and an upper computer;
the LED marking unit comprises a positioning marking point and a registration marking point, the positioning marking point is arranged on the coal mining machine, and the registration marking point is arranged on the scraper conveyor; the visual positioning detection unit comprises a plurality of binocular camera modules which are arranged along the conveying direction of the scraper conveyor, the lenses of the binocular camera modules face the coal wall to be mined, the visual fields of adjacent binocular camera modules have an overlapping range, and the registration mark points are located in the overlapping range;
two cameras and a processor are integrated in the binocular camera modules, and data transmission connection is carried out between the adjacent binocular camera modules; the processor is used for processing the shot digital image, obtaining a coordinate transformation matrix of the adjacent binocular camera modules through a camera calibration technology, and controlling the start and stop of the binocular camera module and the two adjacent binocular camera modules; and the upper computer calculates to obtain the three-dimensional coordinates of the positioning mark points according to the output image processing result, so that the positioning and pose detection of the coal mining machine are realized.
Furthermore, a positioning support rod is arranged at the bottom of the binocular camera module and fixed on a base of the hydraulic support through the positioning support rod, the height of the positioning support rod is the same as that of a machine body of the coal mining machine, and the fixing effect can enable a lens of the binocular camera module to always keep facing a wall to be mined in the frame moving process.
Furthermore, the optical axes of the two cameras on the binocular camera module are parallel and are both perpendicular to the coal wall, and the relative poses of the two cameras are fixed.
Furthermore, the positioning mark points adopt LED circular light sources with different colors, and the installation positions of the positioning mark points on the coal mining machine comprise a central point of the connecting part of the rocker arm and the roller, a hinge joint of the rocker arm and the machine body, a machine body contour line angular point and a key point.
Furthermore, the registration mark points adopt LED circular light sources with different colors from the positioning mark points and are arranged on the side edge of the middle groove of the scraper conveyor.
A detection method of the binocular vision coal mining machine positioning and pose detection system based on the low data volume comprises the following steps:
s1, when the coal mining machine of the fully mechanized mining face walks on the scraper conveyor for operation, the binocular camera module is started to continuously shoot the coal mining machine, and the shooting speed is once every 0.1 second;
the S2 processor processes the shot digital image and extracts the pixel coordinates of the positioning mark point and the registration mark point;
s3, calculating a coordinate transformation matrix of the adjacent binocular camera module through the obtained pixel coordinates of the registration mark points and the camera intrinsic parameter matrix through matrix transformation, and uploading the coordinate transformation matrix to an upper computer;
s4, obtaining three-dimensional homogeneous coordinates of the positioning mark points in a binocular camera module relative coordinate system through the obtained pixel coordinates of the positioning mark points and the camera intrinsic parameter matrix, and uploading the coordinates to an upper computer;
s5, judging the volume of the coal mining machine in the visual field according to the distribution positions of the positioning mark points in the visual field of the binocular camera module, thereby realizing the start-stop control of the binocular camera module in the step S1: when the volume of the coal mining machine in the visual field exceeds one half, a processor of the binocular camera module sends a signal according to the traveling direction of the coal mining machine, and a next adjacent binocular camera module is started; when the volume of the coal mining machine in the visual field is zero, a processor of the binocular camera module sends a signal, and the binocular camera module is closed; in other cases, the processor does not send out a start-stop instruction;
s6, according to the coordinate transformation matrix obtained in the step S3, a transformation matrix of the binocular camera module relative to the coordinate system and the world coordinate system is obtained through calculation, and the three-dimensional homogeneous coordinates of the positioning mark points in the world coordinate system are obtained through combination with the three-dimensional homogeneous coordinates obtained in the step S4, so that the rocker arm angle, the body attitude angle parameter and the position coordinates of the coal mining machine in the world coordinate system are further obtained.
Further, the image processing method employed by the processor in step S2 includes: firstly, converting a shot image from an RGB space to an HSV space, identifying a specific color, determining the position of a positioning mark point in the image on a coal mining machine through the color, removing image noise by using Gaussian filtering, carrying out binarization processing on the image, carrying out morphological processing on the binarized image by using an expansion and corrosion algorithm, searching and drawing a graph contour, removing a contour with an excessively small contour length and a parent contour, carrying out minimum circumcircle fitting on the remaining contour, and taking the center of a circumcircle as the pixel coordinates of the positioning mark point and the registration mark point.
Further, the method for judging the traveling direction of the coal mining machine in the step S5 includes: and extracting three-dimensional coordinates of the same positioning mark point in a relative coordinate system of the binocular camera module by using two images continuously shot by the same binocular camera module, and judging the walking direction of the coal mining machine by comparing the change of the coordinates twice.
Has the advantages that: compared with the prior art, the low-data-volume binocular vision coal mining machine positioning and pose detection system and method provided by the invention have the following advantages:
1. a small number of binocular camera modules are used for continuous photographing at the same time, and specific coordinate values are only output to an upper computer, so that the output data volume is effectively reduced, and the requirement on the communication condition of the fully mechanized mining face is not high;
2. the problems of poor illumination condition and low visibility of the fully mechanized mining face are solved by using the luminous LED mark points, the attitude information of the coal mining machine is convenient to obtain, and the detection precision is improved;
3. the new equipment is less, the reconstruction cost of the fully mechanized coal mining face is low, and the construction of the face is not affected after the new equipment is additionally arranged.
Drawings
FIG. 1 is a schematic view of the installation structure of a visual positioning detection unit in the preferred embodiment of the present invention;
FIG. 2 is a schematic view of the installation structure of the LED sign unit in the preferred embodiment of the present invention;
FIG. 3 is a schematic flow chart of a low data volume binocular vision shearer positioning and pose detection method according to a preferred embodiment of the invention;
fig. 4 is a schematic diagram of a three-dimensional coordinate system defined in a preferred embodiment of the present invention.
The figure includes: 1. a binocular camera module; 2. and positioning the supporting rod.
Detailed Description
The technical solution of the present invention will be more clearly and completely explained below with reference to the description of the preferred embodiments of the present invention in conjunction with the accompanying drawings.
Examples
A binocular vision coal mining machine positioning and pose detection system with low data volume comprises an LED mark unit, a vision positioning detection unit and an upper computer, wherein the vision positioning detection unit is shown in figure 1 and consists of a plurality of binocular camera modules 1 and positioning support rods 2 thereof, and the LED mark unit comprises positioning mark points 3.1-3.12 and registration mark points 4.1-4.6 as shown in figure 2.
Two digital cameras and treater have been integrated to two binocular camera modules inside, the treater can carry out the processing of certain degree to the digital image of shooing, the opening of this binocular camera module of simultaneous control and two adjacent binocular camera modules stops, there is data transmission wire connection between the adjacent binocular camera module, see figure 1, binocular camera module 1 passes through the bolt fastening with the one end of its location bracing piece 2 and is in the same place, the other end of location bracing piece 2 is fixed on hydraulic support's floorbar, fixed effect can make the camera lens of binocular camera module 1 remain throughout towards the wall of waiting to cut coal moving the frame in-process.
Further, the optical axes of the two cameras of the binocular camera module 1 are parallel, both are perpendicular to the coal wall, the relative pose is fixed, and further an internal parameter matrix of the cameras and a pose transformation matrix between the two cameras are obtained through a camera calibration technology.
Referring to the attached drawings 1 and 2, a plurality of binocular camera modules 1 are installed in a manner that two adjacent binocular camera modules 1 are installed in parallel, the installation interval is required to enable the visual fields of the adjacent modules to have a certain overlapping range, the first and last binocular camera modules are located at two ends of a fully mechanized mining face, and registration mark points 4.1-4.6 are respectively located in the overlapping range of the adjacent binocular camera modules 1.
Referring to fig. 2, the positioning mark points 3.1-3.12 are a plurality of LED circular light sources with different colors, wherein the positioning mark points 3.2-3.6 are on the same horizontal line, the positioning mark points 3.8-3.12 are on the same horizontal line, the positioning mark points 3,1 and 3,7 are installed at the center point of the connection between the rocker arm and the roller, the positioning mark points 3.2 and 3.6 are installed at the hinge point between the rocker arm and the body, the positioning mark points 3.4 and 3.10 are installed on the symmetrical axis of the middle part of the body, the positioning mark points 3.8 and 3.12 are installed at the hinge point between the lower end of the body and the hydraulic cylinder, and the other positioning mark points 3.3, 3.5, 3.9 and 3.11 are used as auxiliary positioning mark points and are uniformly installed on the contour line of the body to ensure that each binocular module can shoot more than 3 positioning mark points, and when the length of the body of the coal mining machine increases, the number of the auxiliary positioning mark points is increased to meet the requirements; registration mark points 4.1-4.6 are mounted on the sides of the middle trough of the scraper conveyor, preferably using LED circular light sources of a different color than the registration mark points.
Specifically, as shown in fig. 3, a low data volume binocular vision coal mining machine positioning and pose detection method includes the following steps:
step S1: when a coal mining machine of the fully mechanized mining face walks on a scraper conveyor to operate and is located at different positions, the binocular camera module 1 is started to continuously shoot the coal mining machine, and the shooting speed is once every 0.1 second;
step S2: the processor can use various algorithms to process the shot image, establish a pixel coordinate system on an image plane, take the upper left corner of the image as a coordinate origin, take an X-axis square to the right, and take a Y-axis positive direction downwards, and extract the pixel coordinates of the positioning mark point and the registration mark point in the image;
the image processing method comprises the following specific steps: firstly, converting an image from an RGB space to an HSV space, identifying a specific color, determining the position of a positioning mark point in the image on a coal mining machine through the color, removing image noise by using Gaussian filtering, carrying out binarization processing on the image, carrying out morphological processing on the binarized image by using an expansion and corrosion algorithm, searching and drawing a graph contour, removing a contour with too small contour length and a contour with a father contour, carrying out minimum circumcircle fitting on the remaining contour, and taking the center of a circumcircle as the pixel coordinates of the positioning mark point and the registration mark point. From this, it can be derived that the nth position coordinate point is homogeneous in the imaging plane of the camera on the left side of the module N
Figure BDA0002714867280000051
Homogeneous coordinate of nth location coordinate point in imaging plane of right camera of module N
Figure BDA0002714867280000052
Homogeneous coordinates of the m-th registration coordinate point in the imaging plane of the camera on the left side of module N
Figure BDA0002714867280000053
Homogeneous coordinates of the m-th registration coordinate point in the imaging plane of the camera on the right side of module N
Figure BDA0002714867280000054
Wherein m, N ═ 1,2,3, …;
Figure BDA0002714867280000055
step S3: calculating to obtain a coordinate transformation matrix of an adjacent binocular camera module through matrix transformation by combining the obtained pixel coordinates of the registration mark points and a parameter matrix in the camera;
referring to the attached figure 4, the center of a binocular camera module 1 at the starting point of a fully mechanized mining face is used as the origin of an absolute coordinate system, the centers of the other modules are used as the origins of a relative coordinate system, a Z axis points to the front of a camera (namely is vertical to an imaging plane), positive directions of an X axis and a Y axis are parallel to a pixel coordinate system, each binocular camera module can shoot a registration mark point on each of the left side and the right side, the registration mark point is registered with the left binocular camera module and is referred to by a left registration mark point, the registration mark point is registered with the right binocular camera module and is referred to by a right registration mark point, the binocular camera is calibrated by the prior art, and according to the parallax principle, the coordinates are used for calibrating the binocular camera by the parallax principle
Figure BDA0002714867280000056
And
Figure BDA0002714867280000057
to obtain a leftThe three-dimensional homogeneous coordinate of the registration mark points on the right side in the relative coordinate system of the Nth binocular camera module is
Figure BDA0002714867280000058
And
Figure BDA0002714867280000059
setting the coordinate transformation matrix of two adjacent binocular camera modules as
Figure BDA00027148672800000510
Then
Figure BDA00027148672800000511
Figure BDA00027148672800000512
…
Figure BDA0002714867280000061
Step S4: calibrating a binocular camera by the prior art through the obtained pixel coordinates of the positioning mark points and the parameter matrix in the camera, and obtaining the three-dimensional homogeneous coordinates of the nth positioning mark point in the relative coordinate system of the nth binocular camera module according to the parallax principle
Figure BDA0002714867280000062
Figure BDA0002714867280000063
Step S5: and judging the volume of the coal mining machine in the visual field according to the positions of the positioning mark points distributed in the visual field, thereby realizing the start-stop control of the binocular camera module in the step S1: when the volume of the coal mining machine in the visual field exceeds one half, a processor in the binocular camera module sends a signal according to the traveling direction of the coal mining machine, and the next adjacent binocular camera module is started; when the volume of the coal mining machine in the visual field is zero, a processor in the binocular camera module sends a signal, and the binocular camera module is closed; in other cases, the processor does not send out a start-stop instruction;
the method for judging the walking direction of the coal mining machine comprises the steps of continuously shooting two pictures by using the same binocular camera module, extracting the three-dimensional coordinates of the same positioning mark point, and comparing the values of X-axis coordinates, wherein when the X-axis coordinates are increased, the walking is performed rightward, and when the X-axis coordinates are decreased, the walking is performed leftward;
step S6: outputting the coordinate transformation matrix obtained in the step S3 to an upper computer to obtain a transformation matrix of the relative coordinate system of the Nth binocular camera module relative to the absolute coordinate system
Figure BDA0002714867280000064
Figure BDA0002714867280000065
By transforming matrices
Figure BDA0002714867280000066
Obtaining the three-dimensional homogeneous coordinate Y of each positioning mark point in the absolute coordinate systemn(n＝1-12)：
Figure BDA0002714867280000067
Using Y at different pointsnAnd (4) calculating coordinates to obtain parameters of the rocker arm angle of the coal mining machine, the attitude angle of the machine body and the position coordinates under the world coordinate system, and finally displaying the parameters. Referring to the arrangement of the three-dimensional coordinate plane in fig. 4, the attitude angle of the machine body is divided into a pitch angle (the included angle between the projection of the centerline of the machine body of the coal mining machine on the XOY plane and the advancing direction of the machine body), a roll angle (the included angle between the projection of the transverse axis of the machine body of the coal mining machine on the YOZ plane and the Y axis), and a yaw angle (the included angle between the projection of the centerline of the machine body of the coal mining machine on the XOZ plane and the advancing direction of the machine body)Angle). The specific calculation method comprises the following steps:
three-dimensional coordinates of the localization marker points:
Figure BDA0002714867280000071
position coordinates: y is4
Rocker angle θ:
Figure BDA0002714867280000072
the attitude angle of the body:
pitch angle α:
Figure BDA0002714867280000073
roll angle β:
Figure BDA0002714867280000074
yaw angle γ:
Figure BDA0002714867280000075
according to the invention, a small number of binocular camera modules are used for continuous photographing at the same time, and meanwhile, only specific coordinate values are output to the upper computer, so that the output data volume is effectively reduced, and the requirement on the communication condition of the fully mechanized mining face is not high; the luminous LED marker points are used, so that the problems of poor illumination condition and low visibility of the fully mechanized mining face are solved, the attitude information of the coal mining machine is convenient to obtain, the detection precision is high, the acquisition speed is high, the influence of other factors is small, the automation degree is high, and an important guarantee is provided for realizing unmanned mining of a mine.
The above detailed description merely describes preferred embodiments of the present invention and does not limit the scope of the invention. Without departing from the spirit and scope of the present invention, it should be understood that various changes, substitutions and alterations can be made herein by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims and their equivalents. The scope of the invention is defined by the claims.

Claims (3)

1. The detection method of the binocular vision coal mining machine positioning and pose detection system based on low data volume is characterized in that the binocular vision coal mining machine positioning and pose detection system based on low data volume comprises an LED mark unit, a vision positioning detection unit and an upper computer;
the LED mark unit comprises a positioning mark point and a registration mark point, the positioning mark point is arranged on the coal mining machine, and the registration mark point is arranged on the scraper conveyor; the visual positioning detection unit comprises a plurality of binocular camera modules (1) which are arranged along the conveying direction of the scraper conveyor, the lens of each binocular camera module (1) faces the wall to be mined, the vision fields of adjacent binocular camera modules (1) have an overlapping range, and the registration mark points are located in the overlapping range;
two video cameras and a processor are integrated in the binocular camera modules (1), and data transmission connection is carried out between the adjacent binocular camera modules (1); the processor is used for processing the shot digital images and controlling the start and stop of the binocular camera module and the two adjacent binocular camera modules (1); the upper computer calculates to obtain the three-dimensional coordinates of the positioning mark points according to the output image processing result, so as to realize the positioning and pose detection of the coal mining machine;
the binocular camera module (1) is characterized in that a positioning support rod (2) is arranged at the bottom of the binocular camera module (1) and is fixed on a base of a hydraulic support through the positioning support rod (2), and the height of the positioning support rod (2) is the same as that of a machine body of a coal mining machine;
optical axes of two cameras on the binocular camera module (1) are parallel and are perpendicular to the coal wall, and relative poses of the two cameras are fixed;
the positioning mark points adopt LED circular light sources with different colors, and the mounting positions of the positioning mark points on the coal mining machine comprise a central point at the connecting part of the rocker arm and the roller, a hinge point of the rocker arm and the machine body, a machine body contour line angular point and a key point;
the registration mark points adopt LED circular light sources with different colors from the positioning mark points and are arranged on the side edge of the middle groove of the scraper conveyor;
the method comprises the following steps:
s1, when the coal mining machine of the fully mechanized mining face walks on the scraper conveyor for operation, the binocular camera module (1) is started to continuously shoot the coal mining machine;
the S2 processor processes the shot digital image and extracts the pixel coordinates of the positioning mark point and the registration mark point;
s3, calculating a coordinate transformation matrix of the adjacent binocular camera module through the obtained pixel coordinates of the registration mark points and the camera intrinsic parameter matrix through matrix transformation, and uploading the coordinate transformation matrix to an upper computer;
s4, obtaining three-dimensional homogeneous coordinates of the positioning mark points in a binocular camera module relative coordinate system through the obtained pixel coordinates of the positioning mark points and the camera intrinsic parameter matrix, and uploading the coordinates to an upper computer;
s5, judging the volume of the coal mining machine in the visual field according to the distribution positions of the positioning mark points in the visual field of the binocular camera module, thereby realizing the start-stop control of the binocular camera module (1) in the step S1: when the volume of the coal mining machine in the visual field exceeds one half, a processor of the binocular camera module (1) sends a signal according to the traveling direction of the coal mining machine, and the next adjacent binocular camera module (1) is started; when the volume of the coal mining machine in the visual field is zero, a processor of the binocular camera module (1) sends a signal, and the binocular camera module (1) is closed; in other cases, the processor does not send out a start-stop instruction;
s6, according to the coordinate transformation matrix obtained in the step S3, a transformation matrix of the binocular camera module relative to the coordinate system and the world coordinate system is obtained through calculation, and the three-dimensional homogeneous coordinates of the positioning mark points in the world coordinate system are obtained through combination with the three-dimensional homogeneous coordinates obtained in the step S4, so that the rocker arm angle, the body attitude angle parameter and the position coordinates of the coal mining machine in the world coordinate system are further obtained.
2. The detection method of the low-data-volume binocular vision shearer positioning and pose detection system according to claim 1, wherein the image processing method adopted by the processor in the step S2 comprises the following steps: firstly, converting a shot image from an RGB space to an HSV space, identifying colors, determining the position of a positioning mark point in the image on a coal mining machine through the colors, removing image noise by using Gaussian filtering, carrying out binarization processing on the image, carrying out morphological processing on the binarized image by using an expansion and corrosion algorithm, searching and drawing a graph contour, removing a contour with too small contour length and a contour with a parent contour, carrying out minimum circumcircle fitting on the remaining contour, and taking the center of a circumcircle as the pixel coordinates of the positioning mark point and the registration mark point.
3. The detection method of the low-data-volume binocular vision shearer positioning and pose detection system according to claim 1, wherein the method for judging the walking direction of the shearer in the step S5 comprises the following steps: and extracting three-dimensional coordinates of the same positioning mark point in a relative coordinate system of the binocular camera module by using two images continuously shot by the same binocular camera module, and judging the walking direction of the coal mining machine by comparing the change of the coordinates twice.
CN202011070727.9A 2020-10-09 2020-10-09 Low-data-volume binocular vision coal mining machine positioning and pose detection system and method Active CN112161571B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011070727.9A CN112161571B (en) 2020-10-09 2020-10-09 Low-data-volume binocular vision coal mining machine positioning and pose detection system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011070727.9A CN112161571B (en) 2020-10-09 2020-10-09 Low-data-volume binocular vision coal mining machine positioning and pose detection system and method

Publications (2)

Publication Number Publication Date
CN112161571A CN112161571A (en) 2021-01-01
CN112161571B true CN112161571B (en) 2022-02-22

Family

ID=73861620

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011070727.9A Active CN112161571B (en) 2020-10-09 2020-10-09 Low-data-volume binocular vision coal mining machine positioning and pose detection system and method

Country Status (1)

Country Link
CN (1) CN112161571B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113012236B (en) * 2021-03-31 2022-06-07 武汉理工大学 Intelligent robot polishing method based on crossed binocular vision guidance
CN113409390B (en) * 2021-06-08 2024-02-09 煤炭科学研究总院有限公司 Self-moving tail gesture detection method and device and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107767442A (en) * 2017-10-16 2018-03-06 浙江工业大学 A kind of foot type three-dimensional reconstruction and measuring method based on Kinect and binocular vision
CN108469254A (en) * 2018-03-21 2018-08-31 南昌航空大学 A kind of more visual measuring system overall calibration methods of big visual field being suitable for looking up and overlooking pose
CN108731672A (en) * 2018-05-30 2018-11-02 中国矿业大学 Coalcutter attitude detection system and method based on binocular vision and inertial navigation
CN109798831A (en) * 2018-12-28 2019-05-24 辽宁红沿河核电有限公司 A kind of Binocular vision photogrammetry method for fuel assembly
CN109903383A (en) * 2019-04-11 2019-06-18 中国矿业大学 A kind of coalcutter accurate positioning method in the threedimensional model of working face coal seam

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107747941B (en) * 2017-09-29 2020-05-15 歌尔股份有限公司 Binocular vision positioning method, device and system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107767442A (en) * 2017-10-16 2018-03-06 浙江工业大学 A kind of foot type three-dimensional reconstruction and measuring method based on Kinect and binocular vision
CN108469254A (en) * 2018-03-21 2018-08-31 南昌航空大学 A kind of more visual measuring system overall calibration methods of big visual field being suitable for looking up and overlooking pose
CN108731672A (en) * 2018-05-30 2018-11-02 中国矿业大学 Coalcutter attitude detection system and method based on binocular vision and inertial navigation
CN109798831A (en) * 2018-12-28 2019-05-24 辽宁红沿河核电有限公司 A kind of Binocular vision photogrammetry method for fuel assembly
CN109903383A (en) * 2019-04-11 2019-06-18 中国矿业大学 A kind of coalcutter accurate positioning method in the threedimensional model of working face coal seam

Also Published As

Publication number Publication date
CN112161571A (en) 2021-01-01

Similar Documents

Publication Publication Date Title
CN110232736B (en) Method for quickly constructing three-dimensional scene of underground fully-mechanized mining face
CN109270534B (en) Intelligent vehicle laser sensor and camera online calibration method
CN112161571B (en) Low-data-volume binocular vision coal mining machine positioning and pose detection system and method
CN110246175A (en) Intelligent Mobile Robot image detecting system and method for the panorama camera in conjunction with holder camera
CN105447853A (en) Flight device, flight control system and flight control method
CN105346706A (en) Flight device, and flight control system and method
CN111735445B (en) Monocular vision and IMU (inertial measurement Unit) integrated coal mine tunnel inspection robot system and navigation method
CN112036210B (en) Method and device for detecting obstacle, storage medium and mobile robot
CN109166113B (en) Cutting control system of coal cutter and cutting track planning method
CN109115173A (en) Machine body of boring machine pose monocular vision measurement method based on straight line location model
CN108596173B (en) Single-camera full-view line number real-time recognition device and detection method thereof
CN108198417B (en) A kind of road cruising inspection system based on unmanned plane
CN205693769U (en) A kind of motion cameras positioning capturing quick to panorama target system
CN102879404B (en) System for automatically detecting medical capsule defects in industrial structure scene
CN103886107A (en) Robot locating and map building system based on ceiling image information
CN113115008B (en) Pipe gallery master-slave operation inspection system and method
CN108564628B (en) Cutting head vision positioning and orientation system for heading machine automation
CN112686889B (en) Hydraulic support pushing progress detection method and system based on monocular vision automatic label
CN109991900A (en) Embedded guiding processing system
CN112132078A (en) Coal rock interface recognition system based on image and thermal imaging tracking
CN110861875B (en) Multi-camera scraper conveyor chute position correction method and system
CN106713701B (en) A kind of crowd hazards collecting method and system based on image processing techniques
CN107339950A (en) A kind of track quick track switching operating operation sleeper bolt method for detecting position
CN109115172A (en) Coalcutter fuselage position and posture detection method based on machine vision
CN210530832U (en) Shield tail clearance measuring device for shield tunneling machine

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant