WO2023226574A1 - Scanning and observation system for coal-mine mechanical arm - Google Patents
Scanning and observation system for coal-mine mechanical arm Download PDFInfo
- Publication number
- WO2023226574A1 WO2023226574A1 PCT/CN2023/083776 CN2023083776W WO2023226574A1 WO 2023226574 A1 WO2023226574 A1 WO 2023226574A1 CN 2023083776 W CN2023083776 W CN 2023083776W WO 2023226574 A1 WO2023226574 A1 WO 2023226574A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- map
- environment
- module
- dynamic
- modeling
- Prior art date
Links
- 230000003068 static effect Effects 0.000 claims abstract description 18
- 238000000034 method Methods 0.000 claims abstract description 16
- 238000013507 mapping Methods 0.000 claims abstract description 6
- 239000003245 coal Substances 0.000 claims description 20
- 230000008447 perception Effects 0.000 claims description 12
- 238000010276 construction Methods 0.000 claims description 7
- 238000005516 engineering process Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 4
- 230000007774 longterm Effects 0.000 claims description 4
- 238000005259 measurement Methods 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 239000000203 mixture Substances 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 description 6
- 230000007613 environmental effect Effects 0.000 description 4
- 238000013439 planning Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 239000011378 shotcrete Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004387 environmental modeling Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- E—FIXED CONSTRUCTIONS
- E21—EARTH DRILLING; MINING
- E21F—SAFETY DEVICES, TRANSPORT, FILLING-UP, RESCUE, VENTILATION, OR DRAINING IN OR OF MINES OR TUNNELS
- E21F17/00—Methods or devices for use in mines or tunnels, not covered elsewhere
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
- G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06V10/143—Sensing or illuminating at different wavelengths
Definitions
- the invention belongs to the field of coal mine robots and relates to a coal mine manipulator scanning and observation system.
- Coal mine surveying is a key step in the construction and production period of coal mines. Since coal mine surveying involves both the surface and underground, it must not only serve coal mine production and construction, but also provide information for safety production in order to plan project safety production plans. Coal mine production measurement puts forward higher requirements for data accuracy. Any negligence or data deviation may lead to serious accidents.
- coal mine construction and mining In the process of coal mine construction and mining, it plays a very important role in the planning and design, mining and production, exploration and construction, operation management and future scrapping of coal mines.
- the object of the present invention is to provide a coal mine manipulator scanning and observation system.
- a coal mine manipulator scanning and observation system which includes a geographical information system, an online perception module, a modeling system, a laser radar, an infrared camera, a dynamic and static information recognition module, and an underground environment module;
- the geographical information system and the online perception module are respectively connected with the modeling system signals;
- the laser radar and infrared camera are respectively connected to the dynamic and static information recognition module via signals;
- the modeling system and the dynamic and static information identification module are respectively connected with signals to the underground environment module;
- the geographical information system uses three-dimensional laser scanning to construct a mine environment model and a map description of the mine environment;
- the online perception module uses laser and visual collection perception methods to perform SLAM modeling and update the environment map;
- the lidar transmits image signals to the dynamic and static information recognition module through point cloud registration and combined with the visual semantics of the infrared camera;
- the modeling system and the dynamic and static information identification module jointly transmit the data to the underground environment module; use the dynamic registration results of lidar and vision to determine the map update area and confirm whether it is a long-term change area; based on the original map, use the combination Multi-sensor laser SLAM technology updates the environment map; loop detection and graph optimization methods are used to optimize map quality to ensure the consistency of the updated map with the original map.
- the modeling process of the tunnel environment model includes acquisition of point cloud data, preprocessing of point cloud data and three-dimensional environment modeling;
- the acquisition of point cloud data is affected by the special environment of underground tunnels.
- a laser scanner is used to scan the underground tunnels to obtain Obtain high-precision three-dimensional point cloud data; the obtained three-dimensional point cloud data has errors due to the influence of various factors, and the point cloud data needs to be preprocessed to provide data for obtaining a high-precision model of the underground tunnel environment.
- the construction of a tunnel environment model and a map description of the tunnel environment through three-dimensional laser scanning is specifically:
- Map has been positioned by ndt. Run bag and ndt positioning to generate a point table corresponding to ndt and gnss.
- the trajectory of the point is a polyline, not a straight line;
- the local map will be in the coordinate system with the gps of the first frame of the first bag as the origin. If all submap If the same GPS origin is selected, all submaps will be in the same coordinate system to complete the map splicing.
- the specific steps of performing SLAM modeling to update the environment map are:
- S11 Predict robot status: Based on the motion model, predict the current position status of the robot;
- S12 Predict measured quantities: Based on the obtained predicted positions, predict which measured quantities should be obtained;
- the environment map includes:
- Grid maps Divide the map into m*n grids, and the value in each grid represents whether it is occupied;
- Landmark-based maps Based on landmark mapping, the determined positions of certain landmarks in the map are known.
- the environment map includes:
- Occupancy maps For each grid cell, it represents whether it is occupied;
- Reflection maps For each unit, represents the probability of sensor beam reflection.
- the invention uses a laser scanner to construct a high-precision mine and tunnel environment model, and combines the geographical information system to construct a high-precision map description of the mine and tunnel environment; it uses lidar and infrared cameras for dynamic registration to realize the scene Identification and estimation of information; using laser real-time mapping method to update changing environmental models and maps to achieve dynamic updating of large-scale coal mine tunnel scene models.
- Figure 1 is a technical roadmap of the present invention
- Figure 2 is a system module diagram of the present invention.
- FIG. 1 is a coal mine manipulator scanning and observation system.
- the system includes a geographical information system, an online perception module, a modeling system, a laser radar, an infrared camera, a dynamic and static information recognition module, and an underground environment module;
- the geographical information system and the online perception module are respectively connected with the modeling system signals;
- the laser radar and infrared camera are respectively connected to the dynamic and static information recognition module via signals;
- the modeling system and the dynamic and static information identification module are respectively connected with signals to the underground environment module;
- the geographical information system uses three-dimensional laser scanning to construct a mine environment model and a map description of the mine environment;
- the online perception module uses laser and visual collection perception methods to perform SLAM modeling and update the environment map;
- the lidar transmits image signals to the dynamic and static information recognition module through point cloud registration and combined with the visual semantics of the infrared camera;
- the modeling system and the dynamic and static information identification module jointly transmit the data to the underground environment module; use the dynamic registration results of lidar and vision to determine the map update area and confirm whether it is a long-term change area; based on the original map, use the combination Multi-sensor laser SLAM technology updates the environment map; loop detection and graph optimization methods are used to optimize map quality to ensure the consistency of the updated map with the original map.
- the modeling process of the tunnel environment model includes acquisition of point cloud data, preprocessing of point cloud data and three-dimensional environment modeling;
- the acquisition of point cloud data is affected by the special environment of underground tunnels.
- a laser scanner is used to scan the underground tunnels to obtain high-precision three-dimensional point cloud data.
- the acquired three-dimensional point cloud data has errors due to the influence of various factors, and point cloud needs to be performed.
- Data preprocessing provides data for obtaining a high-precision model of the underground tunnel environment.
- the construction of a tunnel environment model and a map description of the tunnel environment through three-dimensional laser scanning is specifically:
- Map has been positioned by ndt. Run bag and ndt positioning to generate a point table corresponding to ndt and gnss.
- the trajectory of the point is a polyline, not a straight line;
- the local map will be in the coordinate system with the gps of the first frame of the first bag as the origin. If all submap If the same GPS origin is selected, all submaps will be in the same coordinate system to complete the map splicing.
- the specific steps of performing SLAM modeling to update the environment map are:
- S11 Predict robot status: Based on the motion model, predict the current position status of the robot;
- S12 Predict measured quantities: Based on the obtained predicted positions, predict which measured quantities should be obtained;
- the environment map includes:
- Grid maps Divide the map into m*n grids, and the value in each grid represents whether it is occupied;
- Landmark-based maps Based on landmark mapping, the determined positions of certain landmarks in the map are known.
- the environment map includes:
- Occupancy maps For each grid cell, it represents whether it is occupied;
- Reflection maps For each unit, represents the probability of sensor beam reflection.
- 3Use loop detection and graph optimization methods to further optimize the map quality to ensure the consistency of the updated map with the original map.
Abstract
The present invention belongs to the field of coal-mine robots, and relates to a scanning and observation system for a coal-mine mechanical arm. The system comprises a geographic information system, an online sensing module, a modeling system, a LIDAR, an infrared camera, a dynamic and static information identification module and an underground environment module, wherein the geographic information system and the online sensing module are in signal connection with the modeling system, respectively; the LIDAR and the infrared camera are in signal connection with the dynamic and static information identification module, respectively; and the modeling system and the dynamic and static information identification module are in signal connection with the underground environment module, respectively. In the present invention, a high-precision laneway environment model is constructed by using a laser scanner, and a high-precision map description of a laneway environment is constructed in combination with a geographic information system; and a changed environment model and map are updated by using a laser real-time mapping method, thereby realizing the dynamic updating of a wide-range coal-mine laneway scenario model.
Description
本发明属于煤矿机器人领域，涉及一种煤矿机械手扫描观测系统。The invention belongs to the field of coal mine robots and relates to a coal mine manipulator scanning and observation system.
煤矿测量是煤矿建设和生产时期的关键步骤。由于煤矿测量工作涉及地面和井下，不但要为煤矿生产建设服务，也要为安全生产提供信息，以供规划出项目安全生产方案。煤矿生产测量，对数据精确度提出了更高的要求，任何疏忽或数据偏差都可能导致严重的事故发生。Coal mine surveying is a key step in the construction and production period of coal mines. Since coal mine surveying involves both the surface and underground, it must not only serve coal mine production and construction, but also provide information for safety production in order to plan project safety production plans. Coal mine production measurement puts forward higher requirements for data accuracy. Any negligence or data deviation may lead to serious accidents.
在煤矿建设和采矿过程中，为煤矿的规划设计、开采生产、勘探建设、运营管理以及日后报废等工作中承担十分重要的责任与作用。In the process of coal mine construction and mining, it plays a very important role in the planning and design, mining and production, exploration and construction, operation management and future scrapping of coal mines.
发明内容Contents of the invention
有鉴于此，本发明的目的在于提供一种煤矿机械手扫描观测系统。In view of this, the object of the present invention is to provide a coal mine manipulator scanning and observation system.
为达到上述目的，本发明提供如下技术方案：In order to achieve the above objects, the present invention provides the following technical solutions:
一种煤矿机械手扫描观测系统，该系统包括地理信息系统、在线感知模块、建模系统、激光雷达、红外相机、动静态信息识别模块和井下环境模块；A coal mine manipulator scanning and observation system, which includes a geographical information system, an online perception module, a modeling system, a laser radar, an infrared camera, a dynamic and static information recognition module, and an underground environment module;
所述地理信息系统与在线感知模块分别与建模系统信号连接；The geographical information system and the online perception module are respectively connected with the modeling system signals;
所述激光雷达与红外相机分别与动静态信息识别模块信号连接；The laser radar and infrared camera are respectively connected to the dynamic and static information recognition module via signals;
所述建模系统与动静态信息识别模块分别与井下环境模块信号连接；The modeling system and the dynamic and static information identification module are respectively connected with signals to the underground environment module;
所述地理信息系统通过三维激光扫描，构建井巷环境模型以及井巷环境的地图描述；The geographical information system uses three-dimensional laser scanning to construct a mine environment model and a map description of the mine environment;
所述在线感知模块通过激光和视觉采集的感知方式，进行SLAM建模更新环境地图；The online perception module uses laser and visual collection perception methods to perform SLAM modeling and update the environment map;
所述激光雷达通过点云配准，结合所述红外相机的视觉语义，将图像信号传输至动静态信息识别模块；The lidar transmits image signals to the dynamic and static information recognition module through point cloud registration and combined with the visual semantics of the infrared camera;
所述建模系统和动静态信息识别模块联合数据传至井下环境模块；利用激光雷达与视觉的动态配准结果，确定地图更新区域，确认是否为长期变化区域；在原有地图基础上，利用结合多传感器的激光SLAM技术更新环境地图；利用回环检测和图优化方法优化地图质量，保证更新后的地图与原始地图的一致性。The modeling system and the dynamic and static information identification module jointly transmit the data to the underground environment module; use the dynamic registration results of lidar and vision to determine the map update area and confirm whether it is a long-term change area; based on the original map, use the combination Multi-sensor laser SLAM technology updates the environment map; loop detection and graph optimization methods are used to optimize map quality to ensure the consistency of the updated map with the original map.
可选的，所述井巷环境模型的建模过程包括点云数据的获取、点云数据的预处理和三维环境建模；Optionally, the modeling process of the tunnel environment model includes acquisition of point cloud data, preprocessing of point cloud data and three-dimensional environment modeling;
点云数据的获取受井下巷道特殊环境的影响，选用激光扫描仪对井下巷道进行扫描，获
取高精度三维点云数据；获取的三维点云数据受各种因素的影响存在误差，需要进行点云数据的预处理，为进行得到井下巷道环境的高精度模型提供数据。The acquisition of point cloud data is affected by the special environment of underground tunnels. A laser scanner is used to scan the underground tunnels to obtain Obtain high-precision three-dimensional point cloud data; the obtained three-dimensional point cloud data has errors due to the influence of various factors, and the point cloud data needs to be preprocessed to provide data for obtaining a high-precision model of the underground tunnel environment.
可选的，所述通过三维激光扫描，构建井巷环境模型以及井巷环境的地图描述具体为：Optionally, the construction of a tunnel environment model and a map description of the tunnel environment through three-dimensional laser scanning is specifically:
录制相关区域带有GNSS信息的bag；Record the bag with GNSS information in the relevant area;
进行hdl_slam slam构图；Carry out hdl_slam slam composition;
把map绕x轴旋转180，同时z轴方向提高2m；Rotate the map 180 degrees around the x-axis and increase the z-axis direction by 2m;
把map绕z轴旋转90，使机机械手车头与朝向地图上前进的方向；Rotate the map 90 degrees around the z-axis so that the robot head is facing the direction on the map;
map已经进行ndt定位，运行bag及ndt定位生成ndt与gnss的对应点表，点的轨迹是一条折线，不能是直线；Map has been positioned by ndt. Run bag and ndt positioning to generate a point table corresponding to ndt and gnss. The trajectory of the point is a polyline, not a straight line;
利用生成的点表ndt或gnss进行再次转换，使地图方位与gps结合且方位一致；Use the generated point table ndt or gnss to convert again, so that the map orientation is combined with GPS and the orientation is consistent;
把地图转换到gps坐标系下，如果选择本次bag第一帧gps作为原点，则地图的ndt原点与gps原点重合；Convert the map to the GPS coordinate system. If the first GPS frame of this bag is selected as the origin, the ndt origin of the map coincides with the GPS origin;
如果选择某一给定gps坐标作为原点，第一个bag第一帧的gps，则转换完成后，则本地图在第一个bag第一帧的gps为原点的坐标系下，如果所有的submap都选择同一gps原点，则所有的submap在同一坐标系下，完成地图拼接。If a given gps coordinate is selected as the origin, the gps of the first frame of the first bag, then after the conversion is completed, the local map will be in the coordinate system with the gps of the first frame of the first bag as the origin. If all submap If the same GPS origin is selected, all submaps will be in the same coordinate system to complete the map splicing.
可选的，所述进行SLAM建模更新环境地图具体为：Optionally, the specific steps of performing SLAM modeling to update the environment map are:
S11：预测机器人状态：基于运动模型，预测机器人当前位置状态；S11: Predict robot status: Based on the motion model, predict the current position status of the robot;
S12：预测测量量：基于得到的预测位置，预测应该得到哪些测量量；S12: Predict measured quantities: Based on the obtained predicted positions, predict which measured quantities should be obtained;
S13：测量：测量当前真实环境信息；S13: Measurement: Measure the current real environment information;
S14：数据融合：利用EKF将S12和S13中的信息融合；S14: Data fusion: Use EKF to fuse the information in S12 and S13;
S15：更新机器人状态：根据S14的结果，更新机器人位置状态，收敛误差；S15: Update robot status: Based on the results of S14, update the robot position status and convergence error;
S16：更新地图：根据S14和S15更新地图数据。S16: Update map: Update map data according to S14 and S15.
可选的，所述环境地图包括：Optionally, the environment map includes:
栅格图grid maps：将地图分为m*n的栅格，每个栅格内的数值代表是否被占用；Grid maps: Divide the map into m*n grids, and the value in each grid represents whether it is occupied;
地标图landmark-based maps：基于地标建图，已知某些标志物在地图中的确定位置。Landmark-based maps: Based on landmark mapping, the determined positions of certain landmarks in the map are known.
可选的，所述环境地图包括：Optionally, the environment map includes:
占用图occupancy maps：对于每个栅格单元，代表是否被占用；Occupancy maps: For each grid cell, it represents whether it is occupied;
反射图reflection maps：对于每个单元，表示传感器束反射的概率。Reflection maps: For each unit, represents the probability of sensor beam reflection.
本发明的有益效果在于：本发明利用激光扫描仪构建高精度井巷环境模型，结合地理信息系统构建井巷环境的高精度地图描述；利用激光雷达和红外相机进行动态配准，实现场景
信息的识别与估测；利用激光实时建图方法更新变化的环境模型和地图，实现大范围煤矿巷道场景模型的动态更新。The beneficial effects of the present invention are: the invention uses a laser scanner to construct a high-precision mine and tunnel environment model, and combines the geographical information system to construct a high-precision map description of the mine and tunnel environment; it uses lidar and infrared cameras for dynamic registration to realize the scene Identification and estimation of information; using laser real-time mapping method to update changing environmental models and maps to achieve dynamic updating of large-scale coal mine tunnel scene models.
本发明的其他优点、目标和特征在某种程度上将在随后的说明书中进行阐述，并且在某种程度上，基于对下文的考察研究对本领域技术人员而言将是显而易见的，或者可以从本发明的实践中得到教导。本发明的目标和其他优点可以通过下面的说明书来实现和获得。Other advantages, objects, and features of the present invention will, to the extent that they are set forth in the description that follows, and to the extent that they will become apparent to those skilled in the art upon examination of the following, or may be derived from This invention is taught by practicing it. The objects and other advantages of the invention may be realized and obtained by the following description.
为了使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明作优选的详细描述，其中：In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be described in detail below in conjunction with the accompanying drawings, in which:
图1为本发明技术路线图；Figure 1 is a technical roadmap of the present invention;
图2为本发明系统模块图。Figure 2 is a system module diagram of the present invention.
以下通过特定的具体实例说明本发明的实施方式，本领域技术人员可由本说明书所揭露的内容轻易地了解本发明的其他优点与功效。本发明还可以通过另外不同的具体实施方式加以实施或应用，本说明书中的各项细节也可以基于不同观点与应用，在没有背离本发明的精神下进行各种修饰或改变。需要说明的是，以下实施例中所提供的图示仅以示意方式说明本发明的基本构想，在不冲突的情况下，以下实施例及实施例中的特征可以相互组合。The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. The following embodiments and the features in the embodiments can be combined with each other as long as there is no conflict.
其中，附图仅用于示例性说明，表示的仅是示意图，而非实物图，不能理解为对本发明的限制；为了更好地说明本发明的实施例，附图某些部件会有省略、放大或缩小，并不代表实际产品的尺寸；对本领域技术人员来说，附图中某些公知结构及其说明可能省略是可以理解的。The drawings are only for illustrative purposes, and represent only schematic diagrams rather than actual drawings, which cannot be understood as limitations of the present invention. In order to better illustrate the embodiments of the present invention, some components of the drawings will be omitted. The enlargement or reduction does not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
本发明实施例的附图中相同或相似的标号对应相同或相似的部件；在本发明的描述中，需要理解的是，若有术语“上”、“下”、“左”、“右”、“前”、“后”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本发明和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此附图中描述位置关系的用语仅用于示例性说明，不能理解为对本发明的限制，对于本领域的普通技术人员而言，可以根据具体情况理解上述术语的具体含义。In the drawings of the embodiments of the present invention, the same or similar numbers correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms "upper", "lower", "left" and "right" The orientation or positional relationship indicated by "front", "rear", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must be It has a specific orientation and is constructed and operated in a specific orientation. Therefore, the terms describing the positional relationships in the drawings are only for illustrative purposes and cannot be understood as limitations of the present invention. For those of ordinary skill in the art, they can determine the specific position according to the specific orientation. Understand the specific meaning of the above terms.
请参阅图1和图2，为一种煤矿机械手扫描观测系统，该系统包括地理信息系统、在线感知模块、建模系统、激光雷达、红外相机、动静态信息识别模块和井下环境模块；Please refer to Figure 1 and Figure 2, which is a coal mine manipulator scanning and observation system. The system includes a geographical information system, an online perception module, a modeling system, a laser radar, an infrared camera, a dynamic and static information recognition module, and an underground environment module;
所述地理信息系统与在线感知模块分别与建模系统信号连接；
The geographical information system and the online perception module are respectively connected with the modeling system signals;
所述激光雷达与红外相机分别与动静态信息识别模块信号连接；The laser radar and infrared camera are respectively connected to the dynamic and static information recognition module via signals;
所述建模系统与动静态信息识别模块分别与井下环境模块信号连接；The modeling system and the dynamic and static information identification module are respectively connected with signals to the underground environment module;
所述地理信息系统通过三维激光扫描，构建井巷环境模型以及井巷环境的地图描述；The geographical information system uses three-dimensional laser scanning to construct a mine environment model and a map description of the mine environment;
所述在线感知模块通过激光和视觉采集的感知方式，进行SLAM建模更新环境地图；The online perception module uses laser and visual collection perception methods to perform SLAM modeling and update the environment map;
所述激光雷达通过点云配准，结合所述红外相机的视觉语义，将图像信号传输至动静态信息识别模块；The lidar transmits image signals to the dynamic and static information recognition module through point cloud registration and combined with the visual semantics of the infrared camera;
所述建模系统和动静态信息识别模块联合数据传至井下环境模块；利用激光雷达与视觉的动态配准结果，确定地图更新区域，确认是否为长期变化区域；在原有地图基础上，利用结合多传感器的激光SLAM技术更新环境地图；利用回环检测和图优化方法优化地图质量，保证更新后的地图与原始地图的一致性。The modeling system and the dynamic and static information identification module jointly transmit the data to the underground environment module; use the dynamic registration results of lidar and vision to determine the map update area and confirm whether it is a long-term change area; based on the original map, use the combination Multi-sensor laser SLAM technology updates the environment map; loop detection and graph optimization methods are used to optimize map quality to ensure the consistency of the updated map with the original map.
可选的，所述井巷环境模型的建模过程包括点云数据的获取、点云数据的预处理和三维环境建模；Optionally, the modeling process of the tunnel environment model includes acquisition of point cloud data, preprocessing of point cloud data and three-dimensional environment modeling;
点云数据的获取受井下巷道特殊环境的影响，选用激光扫描仪对井下巷道进行扫描，获取高精度三维点云数据；获取的三维点云数据受各种因素的影响存在误差，需要进行点云数据的预处理，为进行得到井下巷道环境的高精度模型提供数据。The acquisition of point cloud data is affected by the special environment of underground tunnels. A laser scanner is used to scan the underground tunnels to obtain high-precision three-dimensional point cloud data. The acquired three-dimensional point cloud data has errors due to the influence of various factors, and point cloud needs to be performed. Data preprocessing provides data for obtaining a high-precision model of the underground tunnel environment.
可选的，所述通过三维激光扫描，构建井巷环境模型以及井巷环境的地图描述具体为：Optionally, the construction of a tunnel environment model and a map description of the tunnel environment through three-dimensional laser scanning is specifically:
录制相关区域带有GNSS信息的bag；Record the bag with GNSS information in the relevant area;
进行hdl_slam slam构图；Carry out hdl_slam slam composition;
把map绕x轴旋转180，同时z轴方向提高2m；Rotate the map 180 degrees around the x-axis and increase the z-axis direction by 2m;
把map绕z轴旋转90，使机机械手车头与朝向地图上前进的方向；Rotate the map 90 degrees around the z-axis so that the robot head is facing the direction on the map;
map已经进行ndt定位，运行bag及ndt定位生成ndt与gnss的对应点表，点的轨迹是一条折线，不能是直线；Map has been positioned by ndt. Run bag and ndt positioning to generate a point table corresponding to ndt and gnss. The trajectory of the point is a polyline, not a straight line;
利用生成的点表ndt或gnss进行再次转换，使地图方位与gps结合且方位一致；Use the generated point table ndt or gnss to convert again, so that the map orientation is combined with GPS and the orientation is consistent;
把地图转换到gps坐标系下，如果选择本次bag第一帧gps作为原点，则地图的ndt原点与gps原点重合；Convert the map to the GPS coordinate system. If the first GPS frame of this bag is selected as the origin, the ndt origin of the map coincides with the GPS origin;
如果选择某一给定gps坐标作为原点，第一个bag第一帧的gps，则转换完成后，则本地图在第一个bag第一帧的gps为原点的坐标系下，如果所有的submap都选择同一gps原点，则所有的submap在同一坐标系下，完成地图拼接。If a given gps coordinate is selected as the origin, the gps of the first frame of the first bag, then after the conversion is completed, the local map will be in the coordinate system with the gps of the first frame of the first bag as the origin. If all submap If the same GPS origin is selected, all submaps will be in the same coordinate system to complete the map splicing.
可选的，所述进行SLAM建模更新环境地图具体为：Optionally, the specific steps of performing SLAM modeling to update the environment map are:
S11：预测机器人状态：基于运动模型，预测机器人当前位置状态；
S11: Predict robot status: Based on the motion model, predict the current position status of the robot;
S12：预测测量量：基于得到的预测位置，预测应该得到哪些测量量；S12: Predict measured quantities: Based on the obtained predicted positions, predict which measured quantities should be obtained;
S13：测量：测量当前真实环境信息；S13: Measurement: Measure the current real environment information;
S14：数据融合：利用EKF将S12和S13中的信息融合；S14: Data fusion: Use EKF to fuse the information in S12 and S13;
S15：更新机器人状态：根据S14的结果，更新机器人位置状态，收敛误差；S15: Update robot status: Based on the results of S14, update the robot position status and convergence error;
S16：更新地图：根据S14和S15更新地图数据。S16: Update map: Update map data according to S14 and S15.
可选的，所述环境地图包括：Optionally, the environment map includes:
栅格图grid maps：将地图分为m*n的栅格，每个栅格内的数值代表是否被占用；Grid maps: Divide the map into m*n grids, and the value in each grid represents whether it is occupied;
地标图landmark-based maps：基于地标建图，已知某些标志物在地图中的确定位置。Landmark-based maps: Based on landmark mapping, the determined positions of certain landmarks in the map are known.
可选的，所述环境地图包括：Optionally, the environment map includes:
占用图occupancy maps：对于每个栅格单元，代表是否被占用；Occupancy maps: For each grid cell, it represents whether it is occupied;
反射图reflection maps：对于每个单元，表示传感器束反射的概率。Reflection maps: For each unit, represents the probability of sensor beam reflection.
研究井巷环境高精度建模技术，利用激光扫描仪构建高精度井巷环境模型，结合地理信息系统构建井巷环境的高精度地图描述。环境建模是喷浆机器人在复杂动态环境下的全场景精确定位与路径规划的重要前提，对喷浆定位精度及路径规划具有重要影响。其环境高精度建模过程一般为点云数据的获取、点云数据的预处理和三维环境建模三个阶段。点云数据的获取受井下巷道特殊环境的影响，选用激光扫描仪对井下巷道进行扫描，获取高精度三维点云数据。获取的三维点云数据受各种因素的影响会存在一定的误差，需要进行点云数据的预处理，为进行得到井下巷道环境的高精度模型提供数据。Research high-precision modeling technology of mine and tunnel environment, use laser scanners to build high-precision mine and tunnel environment models, and combine with geographic information systems to construct high-precision map descriptions of mine and tunnel environment. Environmental modeling is an important prerequisite for the accurate positioning and path planning of the shotcrete robot in the entire scene in a complex dynamic environment, and has an important impact on the positioning accuracy and path planning of the shotcrete robot. Its environment high-precision modeling process generally consists of three stages: point cloud data acquisition, point cloud data preprocessing, and three-dimensional environment modeling. The acquisition of point cloud data is affected by the special environment of underground tunnels. A laser scanner is used to scan the underground tunnels to obtain high-precision three-dimensional point cloud data. The acquired three-dimensional point cloud data will have certain errors due to various factors, and the point cloud data needs to be preprocessed to provide data for a high-precision model of the underground tunnel environment.
研究环境模型动态更新技术，利用激光实时建图方法更新变化的环境模型和地图，实现大范围煤矿巷道场景模型的动态更新。其步骤包括：Research the dynamic update technology of environmental models, use laser real-time mapping method to update changing environmental models and maps, and realize the dynamic update of large-scale coal mine tunnel scene models. The steps include:
①利用激光雷达与视觉的动态配准结果，确定地图更新区域，确认是否为长期变化区域；①Use the dynamic registration results of lidar and vision to determine the map update area and confirm whether it is a long-term change area;
②在原有高精度地图基础上，利用结合多传感器的激光SLAM技术更新环境地图；② Based on the original high-precision map, use laser SLAM technology combined with multiple sensors to update the environmental map;
③利用回环检测和图优化方法进一步优化地图质量，保证更新后的地图与原始地图的一致性。③Use loop detection and graph optimization methods to further optimize the map quality to ensure the consistency of the updated map with the original map.
最后说明的是，以上实施例仅用以说明本发明的技术方案而非限制，尽管参照较佳实施例对本发明进行了详细说明，本领域的普通技术人员应当理解，可以对本发明的技术方案进行修改或者等同替换，而不脱离本技术方案的宗旨和范围，其均应涵盖在本发明的权利要求范围当中。
Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified. Modifications or equivalent substitutions without departing from the purpose and scope of the technical solution shall be included in the scope of the claims of the present invention.
Claims (6)
- 一种煤矿机械手扫描观测系统，其特征在于：该系统包括地理信息系统、在线感知模块、建模系统、激光雷达、红外相机、动静态信息识别模块和井下环境模块；A coal mine manipulator scanning and observation system, which is characterized in that: the system includes a geographical information system, an online perception module, a modeling system, a laser radar, an infrared camera, a dynamic and static information identification module and an underground environment module;所述地理信息系统与在线感知模块分别与建模系统信号连接；The geographical information system and the online perception module are respectively connected with the modeling system signals;所述激光雷达与红外相机分别与动静态信息识别模块信号连接；The laser radar and infrared camera are respectively connected to the dynamic and static information recognition module via signals;所述建模系统与动静态信息识别模块分别与井下环境模块信号连接；The modeling system and the dynamic and static information identification module are respectively connected with signals to the underground environment module;所述地理信息系统通过三维激光扫描，构建井巷环境模型以及井巷环境的地图描述；The geographical information system uses three-dimensional laser scanning to construct a mine environment model and a map description of the mine environment;所述在线感知模块通过激光和视觉采集的感知方式，进行SLAM建模更新环境地图；The online perception module uses laser and visual collection perception methods to perform SLAM modeling and update the environment map;所述激光雷达通过点云配准，结合所述红外相机的视觉语义，将图像信号传输至动静态信息识别模块；The lidar transmits image signals to the dynamic and static information recognition module through point cloud registration and combined with the visual semantics of the infrared camera;所述建模系统和动静态信息识别模块联合数据传至井下环境模块；利用激光雷达与视觉的动态配准结果，确定地图更新区域，确认是否为长期变化区域；在原有地图基础上，利用结合多传感器的激光SLAM技术更新环境地图；利用回环检测和图优化方法优化地图质量，保证更新后的地图与原始地图的一致性。The modeling system and the dynamic and static information identification module jointly transmit the data to the underground environment module; use the dynamic registration results of lidar and vision to determine the map update area and confirm whether it is a long-term change area; based on the original map, use the combination Multi-sensor laser SLAM technology updates the environment map; loop detection and graph optimization methods are used to optimize map quality to ensure the consistency of the updated map with the original map.
- 根据权利要求1所述的一种煤矿机械手扫描观测系统，其特征在于：所述井巷环境模型的建模过程包括点云数据的获取、点云数据的预处理和三维环境建模；A coal mine manipulator scanning and observation system according to claim 1, characterized in that: the modeling process of the tunnel environment model includes acquisition of point cloud data, preprocessing of point cloud data and three-dimensional environment modeling;点云数据的获取受井下巷道特殊环境的影响，选用激光扫描仪对井下巷道进行扫描，获取高精度三维点云数据；获取的三维点云数据受各种因素的影响存在误差，需要进行点云数据的预处理，为进行得到井下巷道环境的高精度模型提供数据。The acquisition of point cloud data is affected by the special environment of underground tunnels. A laser scanner is used to scan the underground tunnels to obtain high-precision three-dimensional point cloud data. The acquired three-dimensional point cloud data has errors due to the influence of various factors, and point cloud needs to be performed. Data preprocessing provides data for obtaining a high-precision model of the underground tunnel environment.
- 根据权利要求1所述的一种煤矿机械手扫描观测系统，其特征在于：所述通过三维激光扫描，构建井巷环境模型以及井巷环境的地图描述具体为：A coal mine manipulator scanning and observation system according to claim 1, characterized in that: the construction of a tunnel environment model and a map description of the tunnel environment through three-dimensional laser scanning is specifically:录制相关区域带有GNSS信息的bag；Record the bag with GNSS information in the relevant area;进行hdl_slam slam构图；Carry out hdl_slam slam composition;把map绕x轴旋转180，同时z轴方向提高2m；Rotate the map 180 degrees around the x-axis and increase the z-axis direction by 2m;把map绕z轴旋转90，使机机械手车头与朝向地图上前进的方向；Rotate the map 90 degrees around the z-axis so that the robot head is facing the direction on the map;map已经进行ndt定位，运行bag及ndt定位生成ndt与gnss的对应点表，点的轨迹是一条折线，不能是直线；Map has been positioned by ndt. Run bag and ndt positioning to generate a point table corresponding to ndt and gnss. The trajectory of the point is a polyline, not a straight line;利用生成的点表ndt或gnss进行再次转换，使地图方位与gps结合且方位一致；Use the generated point table ndt or gnss to convert again, so that the map orientation is combined with GPS and the orientation is consistent;把地图转换到gps坐标系下，如果选择本次bag第一帧gps作为原点，则地图的ndt原点与gps原点重合；Convert the map to the GPS coordinate system. If the first GPS frame of this bag is selected as the origin, the ndt origin of the map coincides with the GPS origin;如果选择某一给定gps坐标作为原点，第一个bag第一帧的gps，则转换完成后，则本地 图在第一个bag第一帧的gps为原点的坐标系下，如果所有的submap都选择同一gps原点，则所有的submap在同一坐标系下，完成地图拼接。If a given GPS coordinate is selected as the origin, the GPS of the first frame of the first bag, then after the conversion is completed, the local The map is in the coordinate system where the GPS of the first frame of the first bag is the origin. If all submaps select the same GPS origin, then all submaps will be in the same coordinate system to complete the map splicing.
- 根据权利要求1所述的一种煤矿机械手扫描观测系统，其特征在于：所述进行SLAM建模更新环境地图具体为：A coal mine manipulator scanning and observation system according to claim 1, characterized in that: performing SLAM modeling to update the environment map is specifically:S11：预测机器人状态：基于运动模型，预测机器人当前位置状态；S11: Predict robot status: Based on the motion model, predict the current position status of the robot;S12：预测测量量：基于得到的预测位置，预测应该得到哪些测量量；S12: Predict measured quantities: Based on the obtained predicted positions, predict which measured quantities should be obtained;S13：测量：测量当前真实环境信息；S13: Measurement: Measure the current real environment information;S14：数据融合：利用EKF将S12和S13中的信息融合；S14: Data fusion: Use EKF to fuse the information in S12 and S13;S15：更新机器人状态：根据S14的结果，更新机器人位置状态，收敛误差；S15: Update robot status: Based on the results of S14, update the robot position status and convergence error;S16：更新地图：根据S14和S15更新地图数据。S16: Update map: Update map data according to S14 and S15.
- 根据权利要求4所述的一种煤矿机械手扫描观测系统，其特征在于：所述环境地图包括：A coal mine robot scanning and observation system according to claim 4, characterized in that: the environment map includes:栅格图grid maps：将地图分为m*n的栅格，每个栅格内的数值代表是否被占用；Grid maps: Divide the map into m*n grids, and the value in each grid represents whether it is occupied;地标图landmark-based maps：基于地标建图，已知某些标志物在地图中的确定位置。Landmark-based maps: Based on landmark mapping, the determined positions of certain landmarks in the map are known.
- 根据权利要求4所述的一种煤矿机械手扫描观测系统，其特征在于：所述环境地图包括：A coal mine robot scanning and observation system according to claim 4, characterized in that: the environment map includes:占用图occupancy maps：对于每个栅格单元，代表是否被占用；Occupancy maps: For each grid cell, it represents whether it is occupied;反射图reflection maps：对于每个单元，表示传感器束反射的概率。 Reflection maps: For each unit, represent the probability of sensor beam reflection.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210578662.1 | 2022-05-25 | ||
| CN202210578662.1A CN114972970A (en) | 2022-05-25 | 2022-05-25 | Coal mine mechanical arm scanning observation system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2023226574A1 true WO2023226574A1 (en) | 2023-11-30 |
Family
ID=82955601
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2023/083776 WO2023226574A1 (en) | 2022-05-25 | 2023-03-24 | Scanning and observation system for coal-mine mechanical arm |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114972970A (en) |
| WO (1) | WO2023226574A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114972970A (en) * | 2022-05-25 | 2022-08-30 | 中煤科工集团重庆研究院有限公司 | Coal mine mechanical arm scanning observation system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103512579A (en) * | 2013-10-22 | 2014-01-15 | 武汉科技大学 | Map building method based on thermal infrared camera and laser range finder |
| CN109059942A (en) * | 2018-08-22 | 2018-12-21 | 中国矿业大学 | A kind of high-precision underground navigation map building system and construction method |
| CN110285813A (en) * | 2019-07-01 | 2019-09-27 | 东南大学 | A kind of man-machine co-melting navigation device of indoor mobile robot and method |
| CN110363846A (en) * | 2019-08-21 | 2019-10-22 | 江苏盈丰电子科技有限公司 | A kind of underground 3D laser imaging intelligent inspection system and its application method |
| CN114972970A (en) * | 2022-05-25 | 2022-08-30 | 中煤科工集团重庆研究院有限公司 | Coal mine mechanical arm scanning observation system |
-
2022
- 2022-05-25 CN CN202210578662.1A patent/CN114972970A/en active Pending
-
2023
- 2023-03-24 WO PCT/CN2023/083776 patent/WO2023226574A1/en unknown
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103512579A (en) * | 2013-10-22 | 2014-01-15 | 武汉科技大学 | Map building method based on thermal infrared camera and laser range finder |
| CN109059942A (en) * | 2018-08-22 | 2018-12-21 | 中国矿业大学 | A kind of high-precision underground navigation map building system and construction method |
| CN110285813A (en) * | 2019-07-01 | 2019-09-27 | 东南大学 | A kind of man-machine co-melting navigation device of indoor mobile robot and method |
| CN110363846A (en) * | 2019-08-21 | 2019-10-22 | 江苏盈丰电子科技有限公司 | A kind of underground 3D laser imaging intelligent inspection system and its application method |
| CN114972970A (en) * | 2022-05-25 | 2022-08-30 | 中煤科工集团重庆研究院有限公司 | Coal mine mechanical arm scanning observation system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114972970A (en) | 2022-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109696663B (en) | Vehicle-mounted three-dimensional laser radar calibration method and system | |
| CN101241011B (en) | High precision positioning and posture-fixing device on laser radar platform and method | |
| CN103472823B (en) | A kind of grating map creating method of intelligent robot | |
| WO2019136714A1 (en) | 3d laser-based map building method and system | |
| CN107966989A (en) | A kind of robot autonomous navigation system | |
| CN110361027A (en) | Robot path planning method based on single line laser radar Yu binocular camera data fusion | |
| Brenner | Extraction of features from mobile laser scanning data for future driver assistance systems | |
| CN110211228A (en) | For building the data processing method and device of figure | |
| CN110412616A (en) | A kind of mining area underground mining stope acceptance method and device | |
| CN104180793A (en) | Device and method for obtaining mobile spatial information for digital city construction | |
| WO2023226574A1 (en) | Scanning and observation system for coal-mine mechanical arm | |
| CA2539783A1 (en) | Method and device for determining the actual position of a geodetic instrument | |
| CN112146682B (en) | Sensor calibration method and device for intelligent automobile, electronic equipment and medium | |
| CN110412596A (en) | A kind of robot localization method based on image information and laser point cloud | |
| CN109491383A (en) | Multirobot positions and builds drawing system and method | |
| Soheilian et al. | Generation of an integrated 3D city model with visual landmarks for autonomous navigation in dense urban areas | |
| Chen et al. | 3D LiDAR-GPS/IMU calibration based on hand-eye calibration model for unmanned vehicle | |
| CN111402702A (en) | Map construction method, device and system | |
| CN207965645U (en) | A kind of robot autonomous navigation system | |
| Zhang et al. | Continuous GNSS-RTK aided by LiDAR/inertial odometry with intelligent GNSS selection in urban canyons | |
| CN113592951A (en) | Method and device for calibrating external parameters of vehicle-road cooperative middle-road side camera and electronic equipment | |
| CN112684797A (en) | Obstacle map construction method | |
| Wang et al. | Micro aerial vehicle navigation with visual-inertial integration aided by structured light | |
| CN114397631B (en) | Large-scale radar antenna array plane precision measurement method and system | |
| TW469392B (en) | Computer-aided synchronous measuring method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23810649 Country of ref document: EP Kind code of ref document: A1 |