WO2021016749A1

WO2021016749A1 - Multi-data fusion-based positioning method, movable platform and storage medium

Info

Publication number: WO2021016749A1
Application number: PCT/CN2019/097957
Authority: WO
Inventors: 冯国强
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-07-26
Filing date: 2019-07-26
Publication date: 2021-02-04
Also published as: CN112105961A

Abstract

A multi-data fusion-based positioning method, a movable platform and a storage medium. The method comprises: acquiring GNSS data, inertial navigation system data, driving status data, and at least one kind of SLAM sensor data of a movable platform; performing mutual verification on the GNSS data, the inertial navigation system data, the driving status data, and the at least one kind of SLAM sensor data to obtain verified data, and determining a target data fusion manner according to the verified data; and fusing the GNSS data, the inertial navigation system data, the driving status data and the at least one kind of SLAM sensor data of the movable platform in accordance with an instruction of the target data fusion manner to obtain target information, and determining the position of the movable platform according to the target information. By means of the embodiments of the present invention, the movable platform in different environments may be positioned on the basis of different data fusion manners, which effectively ensures the positioning accuracy.

Description

Positioning method, movable platform and storage medium based on multi-data fusion

Technical field

The present invention relates to the technical field of positioning, in particular to a positioning method based on multi-data fusion, a movable platform and a storage medium.

Background technique

Positioning technology can provide position and other information for the movable platform, which is a prerequisite for path planning, motion control and autonomous decision-making of the movable platform. At present, the more mature method is to use the Inertial Measurement Unit (IMU) and the Global Navigation Satellite System (Global Navigation Satellite System, GNSS) to combine to achieve real-time positioning. However, the global navigation satellite system GNSS has the problem of frequent signal loss in complex environments such as urban canyons, tunnels, or wild jungles. Frequent signal loss will cause the mobile platform to be unable to use the global satellite navigation system GNSS to accurately locate the situation. In this case, the movable platform can only use the inertial measurement module IMU for positioning, but the inertial measurement module IMU has low positioning accuracy and cannot meet the precise positioning requirements of the movable platform.

Summary of the invention

The embodiment of the invention discloses a positioning method, a movable platform and a storage medium based on multi-data fusion, which can position the movable platform in different environments based on different data fusion methods, effectively ensuring positioning accuracy.

On the one hand, an embodiment of the present invention discloses a positioning method based on multiple data fusion, which is applied to a movable platform, and the method includes:

Acquiring global satellite navigation system (GNSS) data, inertial navigation system data, driving state data, and at least one real-time positioning and mapping (SLAM) sensor data of the mobile platform;

Perform mutual verification on the GNSS data, the inertial navigation system data, the driving state data, and at least one SLAM sensor data to obtain data that has passed the verification, and determine the target data fusion based on the data that has passed the verification the way;

Perform fusion processing on the GNSS data, inertial navigation system data, driving state data, and at least one SLAM sensor data of the movable platform according to the instructions of the target data fusion mode to obtain target information, and determine the target information according to the target information. Describe the location of the movable platform.

On the other hand, an embodiment of the present invention discloses a movable platform, including: a memory and a processor,

The memory is used to store program instructions;

The processor is configured to execute program instructions stored in the memory, and when the program instructions are executed, the processor is configured to:

Acquiring GNSS data, inertial navigation system data, driving state data, and at least one SLAM sensor data of the movable platform;

Correspondingly, the embodiment of the present invention also discloses a computer-readable storage medium in which a computer program is stored, and when the computer program is executed by a processor, the positioning method based on multiple data fusion as described above is implemented A step of.

In the embodiment of the present invention, the GNSS data, inertial navigation system data, driving state data and at least one SLAM sensor data of the movable platform are mutually verified to obtain the data that has passed the verification, and the data is determined according to the data that has passed the verification. Target data fusion mode instructions fusion process the GNSS data of the movable platform, inertial navigation system data, driving status data and at least one SLAM sensor data to obtain the target information, and determine the position of the movable platform based on the target information. Different data fusion methods locate mobile platforms in different environments to effectively ensure positioning accuracy.

Description of the drawings

In order to explain the technical solutions in the embodiments of the present invention more clearly, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative labor, other drawings can be obtained from these drawings.

Figure 1 is a schematic structural diagram of a movable platform disclosed in an embodiment of the present invention;

2 is a schematic flowchart of a positioning method based on multiple data fusion disclosed in an embodiment of the present invention;

3 is a schematic flowchart of another positioning method based on multiple data fusion disclosed in an embodiment of the present invention;

FIG. 4 is a schematic diagram of the conversion relationship between filtering modes disclosed in an embodiment of the present invention;

5 is a schematic diagram of the conversion relationship between sub-modes in the position observation mode disclosed in the embodiment of the present invention;

Fig. 6 is a schematic structural diagram of another movable platform disclosed in an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

Please refer to FIG. 1, which is a schematic structural diagram of a movable platform provided by an embodiment of the present invention. As shown in Figure 1, the mobile platform is configured with a global satellite navigation system GNSS101, an inertial navigation system INS or a strapdown inertial navigation system SINS102, and a sensor module 103 for collecting driving state data; the mobile platform is also configured with at least one A positioning module 104 for acquiring real-time positioning and map construction (Simultaneous Localization And Mapping, SLAM) sensor data. Among them, the inertial navigation system INS or strapdown inertial navigation system SINS102 may include an inertial measurement module IMU, which may include gyroscopes and accelerometers, etc.; including the inertial measurement module may be a low-precision microelectromechanical system (MEMS) IMU, It can also be a fiber type or laser type IMU. The positioning module 104 can be carried on the fuselage 106 of the movable platform through the platform 105 of the movable platform. The platform 105 can drive the positioning module 104 around one or more of the yaw axis, roll axis, and pitch axis. Rotate to adjust the posture for acquiring SLAM sensor data; the positioning module 104 can also be directly carried on the body 106 of the movable platform. In some embodiments, the positioning module 104 may be completely fixed to the pan/tilt 104, or may be partially fixed to the pan/tilt 104, and the other part may be directly carried on the body 106 of the movable platform.

Among them, there may be one or more GNSS systems 101, one or more inertial navigation systems, and one or more sensor modules 103 for collecting driving state data, and positioning modules for acquiring SLAM sensor data. 104 can also be one or more. The positioning module 104 used to acquire SLAM sensor data may be a positioning module based on image sensors, a positioning module based on lidar, or the like. The sensor module 103 for collecting driving state data may be an odometer or the like. It should be noted that the movable platform shown in FIG. 1 is described by taking a vehicle as an example. The movable platform in the embodiment of the present invention may also be an unmanned aerial vehicle (UAV), an unmanned ship, a mobile robot, etc. Removable equipment.

The positioning method based on multiple data fusion described in the embodiment of the present invention can be applied to the movable platform shown in FIG. 1, specifically: the movable platform obtains its GNSS data, inertial navigation system data, driving state data, and at least one SLAM Sensor data: The acquired GNSS data, inertial navigation system data, driving state data and at least one SLAM sensor data are mutually verified to obtain data that has passed the verification, and the target data fusion method is determined based on the data that has passed the verification. The data fusion method can be used to indicate the type or type of data to be used for data fusion. Further, the movable platform performs fusion processing on the GNSS data, inertial navigation system data, driving state data, and at least one SLAM sensor data of the movable platform according to the instructions of the target data fusion mode to obtain target information, and determine the target information according to the target information. The location of the mobile platform. By adopting the above method, the movable platform in different environments can be positioned based on different data fusion methods, effectively ensuring the positioning accuracy. The detailed description is given below.

Please refer to FIG. 2, which is a schematic flowchart of a positioning method based on multiple data fusion according to an embodiment of the present invention. The positioning method based on multi-data fusion described in the embodiment of the present invention is applied to a mobile platform configured with a global satellite navigation GNSS system, an inertial navigation system INS or a strapdown inertial navigation system SINS for collecting A sensor module for driving state data, and at least one positioning module for acquiring SLAM sensor data. Wherein, the method includes the following steps:

S201. Acquire GNSS data, inertial navigation system data, driving state data, and at least one SLAM sensor data of the movable platform.

In the embodiment of the present invention, the GNSS data is also the observation data output by the global satellite navigation system GNSS configured with the movable platform, including the carrier phase data and speed of the movable platform. The global satellite navigation system GNSS may be a single-point type global satellite navigation system and/or a differential type global satellite navigation system. The inertial navigation system data includes INS data and/or SINS data. The INS data is also the observation data output by the inertial navigation system INS configured on the mobile platform, and the SINS data is the output of the strapdown inertial navigation system SINS configured on the mobile platform. data. Inertial navigation system data includes measurement data of gyroscope and accelerometer in inertial navigation system, etc. The measurement data of gyroscope includes the angular velocity of the movable platform, and the measurement data of accelerometer includes the acceleration of the movable platform. The driving state data is the observation data output by the sensor module configured to collect driving state data on the movable platform. The sensor module used to collect driving state data may be an odometer, and the driving state data collected by the odometer includes movable The speed and acceleration of the platform, etc.; if the movable platform is a vehicle, the driving status data includes the wheel speed and acceleration of the vehicle; if the movable platform is a drone, the driving status data includes the ground speed and Ground acceleration, etc.

SLAM sensor data is also the observation data output by the positioning module configured to obtain SLAM sensor data on the mobile platform. The positioning module used to obtain SLAM sensor data includes a positioning module based on image sensors and/or a lidar as The main positioning module, the SLAM sensor data specifically includes the positioning data output by the image sensor-based positioning module and/or the positioning data output by the lidar-based positioning module. The positioning data output by the positioning module based on the image sensor includes the speed of the movable platform and the environmental feature information extracted from the image of the environment where the movable platform is located. The positioning data output by the positioning module based on lidar includes the speed of the movable platform and the point cloud data of the environment where the movable platform is located.

S202. Perform mutual verification on the GNSS data, the inertial navigation system data, the driving state data, and at least one SLAM sensor data to obtain data that has passed the verification.

In the embodiment of the present invention, the GNSS system, the inertial navigation system, the sensor module for collecting driving state data, and the positioning module for acquiring SLAM sensor data all add identification information to the respective output data, and the identification information is used to indicate the output Whether the data is valid; therefore, the mobile platform can detect whether the data is available through the identification information of the data. The output information of the GNSS system, inertial navigation system, sensor module for collecting driving state data, and positioning module for acquiring SLAM sensor data also includes the reference data output frequency; therefore, the movable platform can detect the output data of the above-mentioned system or module. The real frequency is the same as the reference data output frequency included in the information output by the system or module (or within a preset error range) to determine whether the data output by the system or module is available.

The mobile platform is based on the identification information of each data in the data set composed of GNSS data, inertial navigation system data, driving state data, and at least one SLAM sensor data and the output data of the corresponding device (that is, the aforementioned system or module) Frequency, to detect the various data to determine from the various data that the identification information is valid, and the true frequency of the corresponding device output data is consistent with the output frequency of the reference data output by the corresponding device (or in advance Set the first data within the error range), the first data is also the data available in the various data; and the data set composed of the first data is determined as the data set that passes the detection.

Further, the mobile platform performs mutual verification on the data in the detected data set to obtain data that has passed the verification. Specifically, the mobile platform can use the data output by the target device in the above-mentioned system or module and pass the detection as reference data, and use the reference data to output and pass the detection on the device other than the target device in the system or module. The data is verified, and the data passed the verification is obtained. The target device is a system or module whose default data accuracy of the mobile platform is minimally affected by environmental factors, and the data output by the target device is highly reliable. In an optional embodiment, the target device may be a sensor module for collecting driving state data in the aforementioned system or module, and the sensor module may specifically be an odometer. For example, if the detected data set includes the speed output by the odometer and the speed output by the GNSS system, the speed output by the odometer can be used to verify the speed output by the GNSS system. If the output speed is consistent or within the preset error range, it is determined that the data output by the GNSS system and passed the test is reliable, and the data output by the GNSS system and passed the test is determined as the data passed by the verification; otherwise, the GNSS system The output data that passed the test is excluded.

In an optional embodiment, after the mobile platform determines the data set that has passed the test, it performs self-inspection on the second data output by the same system or module in the data set that has passed the test, so that the data that has not passed the self-inspection Exclude the data set that passed the self-test. For example, when the movable platform is a vehicle, the odometer can output the respective wheel speeds of the four wheels of the vehicle; if the four wheel speeds are compared, it is detected that a wheel speed is significantly different from the other three wheel speeds. It is determined that a certain wheel speed is abnormal, the self-inspection fails, and the abnormal wheel speed is excluded from the data set that has passed the detection. For another example, the GNSS system can simultaneously output the carrier phase data and speed of the movable platform. By processing the carrier phase data, the speed of the movable platform can also be obtained; compare the processed speed with the output speed of the GNSS system, if If the difference between the two exceeds the preset error range, it can be determined that the data output by the GNSS system is abnormal and the self-inspection has failed, and the data output by the GNSS system can be excluded from the data set that has passed the test. Further, the mobile platform performs mutual verification on the data in the data set that has passed the self-inspection to obtain data that has passed the verification. The embodiment of the present invention can effectively isolate inaccurate data by detecting, self-checking, and mutual verification on the data output by the above-mentioned system or module, thereby ensuring that the determined data fusion method is optimal and positioning accuracy.

In another optional embodiment, since the coordinate systems used by the GNSS system, the inertial navigation system, the sensor module for collecting driving state data, and the positioning module for acquiring SLAM sensor data are usually different, in order to facilitate data comparison and Processing, the mobile platform is based on GNSS data, inertial navigation system data, driving state data and at least one kind of SLAM sensor data in the data set consisting of the identification information of each data and the frequency of the corresponding device output data Before the item data is detected, the data in the above data set is converted to the reference coordinate system to obtain the data set after coordinate conversion; then the identification information and corresponding data in the data set after coordinate conversion are obtained. The frequency of the output data of the device is to detect various data in the data set after coordinate conversion, and obtain the data set that passed the test.

S203: Determine a target data fusion mode according to the data passed the verification.

In the embodiment of the present invention, when the verified data includes GNSS data, it indicates that the accuracy of the data output by the GNSS system under the current environment is high. At this time, the data output by the GNSS system can be used for accurate positioning, and the movable platform will use GNSS The data-based data fusion method is determined as the target data fusion method.

If the at least one type of SLAM sensor data includes the positioning data output by the image sensor-based positioning module, then when the verified data includes the positioning data output by the image sensor-based positioning module and does not include GNSS data, It shows that the accuracy of the data output by the GNSS system in the current environment is low, and the positioning data output by the positioning module based on image sensors is high; at this time, the data output by the GNSS system cannot be used for accurate positioning, but the image sensor can be used as the main The positioning data output by the positioning module of the image sensor is used for accurate positioning; the movable platform determines the data fusion method based on the positioning data output by the positioning module mainly based on the image sensor as the target data fusion method. The image sensor may be a monocular image sensor, a binocular image sensor, a multi-eye image sensor, a fish-eye image sensor, or a compound-eye image sensor. The monocular image sensor can obtain the surrounding image information based on the machine vision (Machine View) through the image information returned by the image sensor, and then perform positioning or map construction. Based on the positioning and map construction information, the positioning of the sensor carrier can be determined. In addition to monocular image sensor data, you can also obtain positioning information through binocular image sensors, multi-eye image sensors, etc., and also increase the robustness of positioning and map construction based on the depth obtained by binocular or multi-eye. Finally, the positioning results are more accurate. The positioning information can be used as verification data to be verified with GNSS data and inertial sensor data to retain high-confidence data.

If the at least one type of SLAM sensor data includes positioning data output by a positioning module based on lidar, then when the data passed the verification includes positioning data output by a positioning module based on lidar and does not include GNSS data, It shows that the accuracy of the data output by the GNSS system is low under the current situation, and the positioning data output by the positioning module based on lidar is high; at this time, the data output by the GNSS system cannot be used for accurate positioning, but the lidar can be used as the main The positioning data output by the positioning module of the mobile platform performs accurate positioning; the movable platform determines the method of data fusion based on the positioning data output by the positioning module mainly based on lidar as the target data fusion method.

If the at least one type of SLAM sensor data includes the positioning data output by the positioning module mainly based on lidar and the positioning data output by the positioning module mainly based on image sensors, then when the data passed the verification includes the mainly based on lidar When the positioning data output by the positioning module and the positioning data output by the image sensor-based positioning module do not include GNSS data; the movable platform will use the positioning data output by the lidar-based positioning module and the positioning data The positioning data output by the positioning module mainly based on the image sensor is determined as the target data fusion method as the main data fusion method.

In another optional embodiment, if the at least one type of SLAM sensor data includes positioning data output by a positioning module based on lidar, and/or positioning data output by a positioning module based on image sensors, when When the verified data includes the positioning data output by the positioning module based on lidar and/or the positioning data output by the positioning module based on image sensors, and includes GNSS data; the movable platform combines GNSS data and lidar The positioning data output by the main positioning module and/or the positioning data output by the positioning module mainly based on the image sensor are used as the main reference data for data fusion to determine the target data fusion method.

S204: Perform fusion processing on the GNSS data, inertial navigation system data, driving state data, and at least one SLAM sensor data of the movable platform according to the instruction of the target data fusion mode, to obtain target information.

In the embodiment of the present invention, the indication of the target data fusion mode is to specify the type or type of data to be used for fusion of the GNSS data, inertial navigation system data, driving status data, and at least one SLAM sensor data of the movable platform. Process to get target information. For example, if the target data fusion method is based on the positioning data output by the lidar-based positioning module, the movable platform will use the positioning data output by the lidar-based positioning module as the main data fusion method. With reference to data, the GNSS data, inertial navigation system data, driving state data, and at least one SLAM sensor data of the movable platform are fused to obtain target information. Other integration methods can be deduced by analogy, so I won't repeat them here.

Among them, the target information after fusion processing includes subsequent data used to determine the position of the movable platform. When the target data fusion method is based on GNSS data, the target information may include carrier phase data in the GNSS data, and angular velocity and acceleration in the driving state data. When the target data fusion method is based on the positioning data output by the image sensor-based positioning module, the target information may include environmental feature information extracted from the image of the environment where the movable platform is located Wait. When the target data fusion method is based on the positioning data output by the lidar-based positioning module, the target information may include the point cloud data of the environment where the movable platform is located and/or the point cloud data Feature point information extracted from cloud data, etc.

By adopting the positioning method based on multiple data fusion provided by the embodiment of the present invention, it is possible to select the one with high data accuracy when the GNSS data accuracy is low due to the mobile platform being in a complex environment such as urban canyons, tunnels, or wild jungles. The positioning data output by the positioning module based on lidar and/or the positioning data output by the positioning module based on image sensors are mainly used for data fusion to ensure that the target information obtained after data fusion has high accuracy. So as to meet the positioning accuracy requirements of the movable platform.

S205: Determine the position of the movable platform according to the target information.

In the embodiment of the present invention, the target information includes positioning data or positioning information. The movable platform can determine the position of the movable platform in the high-precision map according to the target information. The high-precision map may be an offline high-precision map downloaded in advance by the mobile platform. The high-precision map records map information that can be verified with the positioning data. For example, based on the positioning information obtained by the vision sensor or lidar, landmarks matching the positioning information can be obtained. In the high-precision map, through the matching of the landmarks Ways to determine where the platform can be moved.

In order to better understand the positioning method based on multiple data fusion provided by the embodiment of the present invention, the following examples will be used for detailed description. Since the use of higher-precision fiber-type or laser-type IMUs will drastically increase equipment costs, the inertial navigation system in the embodiments of the present invention can use low-precision micro-electromechanical IMUs with low energy consumption, small size, and low cost, and based on the foregoing The positioning method of data fusion realizes the purpose of fusing multi-sensor data or redundant sensor data, and finally provides low-cost, accurate and reliable positioning information for the mobile platform of autonomous driving. Please also refer to FIG. 3, which shows a schematic flowchart of another positioning method based on multiple data fusion. As shown in Figure 3, the sensor modules of the movable platform include an inertial measurement module, an odometer, a positioning module based on lidar, a positioning module based on image sensors, and a global satellite navigation system. Among them, the laser radar-based positioning module mainly uses laser point cloud data to achieve positioning, and its module can integrate sensors such as inertial measurement modules or odometers. The image-based positioning module uses image information to achieve positioning. For example, it uses image information obtained by a visual odometer to achieve positioning. The module can also integrate sensors such as inertial measurement modules or odometers. The global satellite navigation system can provide both single-point positioning results and higher-precision differential (RTK) positioning results. It should be noted that in order to ensure the safety of autonomous driving on the mobile platform, the mobile platform can carry multiple sets of the same sensors, for example, two sets of GNSS systems can be installed at the same time. The above-mentioned inertial measurement module corresponds to the aforementioned inertial navigation system, the above-mentioned odometer corresponds to the aforementioned sensor module for collecting driving state data, the positioning module based on lidar and the positioning module based on image sensors correspond to the aforementioned The described positioning module for acquiring SLAM sensor data.

The above-mentioned sensor module configured on the movable platform can provide some observation data for navigation and positioning. As shown in Figure 3, the inertial measurement module can output observation data such as acceleration and angular velocity of the movable platform, the odometer can output observation data such as the speed of the movable platform, and the positioning module based on lidar can output the position of the movable platform. Observation data such as, heading, etc. The positioning module based on image sensors can output observation data such as the position, attitude and speed of the movable platform, and the global satellite navigation system can output the GNSS position and GNSS speed in the single-point positioning results of the movable platform. Observation data can also output the RTK position, RTK speed, dual-antenna route and other observation data in the differential positioning results of the movable platform. It can be seen that different sensor modules may output the same observation data. The main goal of this scheme is to manage and verify these rich sensor data, and design several different filter modes, that is, the data fusion method described above. In this way, the best filtering mode can be selected based on the current observation data of various sensor modules, so as to obtain higher positioning accuracy. Moreover, in the case that some observation data is not available, based on the currently available observation data, the filter mode of the filter can be degraded accordingly, and the filter is also a device for data fusion processing. In this way, the movable platform can perform different processing strategies according to different filtering modes. For example, when the filtering mode is low, the movable platform can be controlled to actively stop moving.

In order to facilitate the subsequent description, the observation data output by the above-mentioned sensor modules are marked as follows: the speed output by the odometer is recorded as odo_v; the position and heading output by the positioning module mainly based on laser radar are recorded as laser_p and laser_yaw; The position and attitude output by the main positioning module are recorded as vo_pq, the output velocity is recorded as vo_v, the output gravity observation is recorded as vo_gravity; the GNSS position and GNSS velocity output by the global satellite navigation system are recorded as gnss_p, gnss_v, and the output RTK The position and RTK speed are recorded as rtk_p and rtk_v respectively, and the output dual-antenna heading is recorded as rtk_yaw. Therefore, the available sensor module observation data are shown in Table 1:

laser_p

laser_yaw

vo_pq

vo_v

vo_gravity

rtk_p

rtk_v

rtk_yaw

gnss_p

gnss_v

odo_v

Based on the above observation data, the following 6 filtering modes can be designed:

1. Invalid mode FS_NONE (the filter is in an invalid state); 2. Image observation mode FS_VPQ (the filter has the relative observation of the visual odometer VIO), the relative observation with VIO means that there is the output of the positioning module mainly based on the image sensor The position observation data of, other similar descriptions are the same; 3. Position observation mode FS_POSI (the filter is in position mode, with global position observation); 4. Image and position observation mode FS_POSI_VPQ (the filter has both VIO relative observation and global position Observation); 5. RTK observation mode FS_POSI_RTK (the filter has both RTK relative observation and global position observation); 6. Image and RTK observation mode FS_POSI_VPQ_RTK (the filter has both RTK relative observation, VIO relative observation and global position observation). Among them, the priority of the above 6 filtering modes from low to high are: FS_NONE, FS_VPQ, FS_POSI, FS_POSI_VPQ, FS_POSI_RTK, FS_POSI_VPQ_RTK. Please refer to Fig. 4 together. Fig. 4 shows the conversion relationship among the above 6 filtering modes, and the conversion conditions for the conversion from the filtering mode with low priority to the filtering mode with high priority are marked. For the filtering mode with high priority The conversion conditions for the conversion to the filtering mode with lower priority can be deduced by analogy, and will not be repeated here.

In addition, since the position observation data laser_p and gnss_p both provide global position information, 3 sub-modes can be set in the position observation mode FS_POSI, which are respectively marked as invalid position observation mode FS_POSI_NOE, lidar position observation mode FS_POSI_LASER, and GNSS position observation Mode FS_POSI_GNSS. Please also refer to FIG. 5, which shows the conversion relationship among the three sub-modes in the position observation mode FS_POSI. When there is lidar positioning observation, that is, there is position observation data output by the positioning module mainly based on lidar, the sub-mode will preferentially jump to the FS_POSI_LASER sub-mode; when there is no lidar positioning observation but GNSS observation In this case, the sub-mode will jump to the FS_POSI_GNSS sub-mode. If there is GNSS observation, that is, there is the position observation data output by the GNSS system; when there is neither lidar positioning observation nor GNSS observation, the sub-mode will jump Go to the FS_POSI_NONE sub-mode.

Further, the steps in Figure 3 are introduced:

(1) Coordinate conversion: Different sensor modules usually have different coordinate systems, so it is necessary to convert the coordinate system of each sensor module to the coordinate system set by the filter, such as the commonly used northeast sky coordinate system. Among them, the output of the positioning module mainly based on lidar is already the data in the northeast sky coordinate system, and the data output by other sensor modules need to undergo coordinate conversion before they can be used in the filter.

(2) Information statistics, corresponding to the data detection process in the previous article: Information statistics mainly detect whether the data output by each sensor module is based on the identification information of the data mark output by each sensor module and the data frequency output by each sensor module. Normal, that is, to count which data currently output by each sensor module is available and which is not available.

(3) Module data self-check: each sensor module can usually give a variety of data. Through mutual verification between the various data output by the same sensor module, it can be judged whether the data currently output by the sensor module is available. Therefore, the module data self-check will filter the available data given by the information statistics again.

(4) Data mutual inspection between modules: In addition to the module data self-inspection, the observation data output by each sensor module can also be mutually verified. For example, the observation data output by the odometer is generally more reliable. The speed output by the odometer can be used to verify the speed of the GNSS output, so as to judge whether the observation data currently output by the GNSS is available. Therefore, the inter-module data mutual inspection will further filter the available data given by the module data self-inspection.

(5) Mode selection, filter configuration, mode selection corresponds to the data fusion method determination process in the previous article: According to the available data obtained in step (4), determine which filtering mode the filter can be in, and then select one of the available filtering modes The filter mode with the highest priority is used as the final filter mode of the filter. The filter mode corresponds to the data fusion method in the previous article; further, the filter is configured according to the selected filter mode, such as reducing or increasing the state dimension and configuration of the filter Observation data of the filter, etc.

In some embodiments, the observation data required by the above 6 filtering modes are shown in Table 2:

Among them, WITHOUT indicates that the observation data is not required, OPTION indicates that the observation data is optional, and NECE_0 and NECE_1 indicate that the observation data must be available. For example, to enter the FS_POSI_RTK mode, the necessary observation data must have rtk_p; for another example, to enter the FS_POSI mode, the necessary observation data are laser_p and laser yaw, or there is observation data gnss_p.

(6) Filter data fusion: Use the configured filter to filter and fuse the data output by the inertial measurement module and the available data obtained in step (4) above or the data output by sensor modules other than the inertial measurement module to obtain Location information of the mobile platform.

Using the above method, through the module data self-check and the module data mutual check, the observation data output by the inaccurate or even faulty sensor module can be effectively isolated. For example, when the mobile platform just exits the tunnel or in the case of an urban canyon environment, the identification information of the GNSS position observation data output by the GNSS system indicates that the GNSS position observation data is valid, but the positioning result of the GNSS system is reliable at this time The performance and accuracy are poor, and the corresponding observation data output by the GNSS system can be verified through the speed output by the odometer, which can avoid the problem of introducing the poorly reliable observation data currently output by the GNSS system into the filter and causing the positioning result deviation. For another example, when an abnormality occurs in the positioning module based on lidar, if the observation data output by the GNSS system is accurate, the observation data output by the GNSS system can be used to compare the observation data output by the positioning module based on lidar Perform verification. In short, through the self-check and mutual check of the module data, the contamination of the filter by the wrong observation data can be greatly reduced.

In addition, the above method can ensure that the movable platform has high positioning accuracy in different environments. For example, when the movable platform causes frequent loss of GNSS signals in an obstructed environment, the observation data output by the positioning module based on lidar or the positioning module based on image sensors or sensor modules such as odometer can be used. The input filter performs data fusion, which can still output high-precision positioning results at this time. In addition, the switching between filtering modes, that is, the switching of the data fusion mode, can ensure that the filter can be configured flexibly and reasonably in the case of various sensor failures or effective conditions, and ensure the accuracy of the output results, while giving the current filtering Mode, can facilitate the corresponding operation of the movable platform. For example, when the positioning module based on lidar adopts the Monte Carlo positioning method based on grid map, the corresponding grid map needs to be collected in advance. When the movable platform moves to an area without a map, the position can be observed The sub-mode of the mode is switched from the FS_POSI_LASER sub-mode to the FS_POSI_GNSS sub-mode, so that the global position observation data provided by the GNSS system can be used to ensure the global positioning accuracy. For another example, if the mobile platform in the tunnel is unable to find a suitable grid due to the similar texture of the surrounding environment, which causes the laser positioning of the positioning module based on lidar to fail, you can switch the filtering mode from FS_POSI_VPQ mode to The FS_VPQ mode can use the observation data provided by the image sensor-based positioning module to ensure the accuracy of the positioning results.

Please refer to FIG. 6, which is a schematic structural diagram of another movable platform according to an embodiment of the present invention. The movable platform described in the embodiment of the present invention includes: a processor 601, a communication interface 602, and a memory 603. Among them, the processor 601, the communication interface 602, and the memory 603 may be connected through a bus or in other ways. The embodiment of the present invention takes the connection through a bus as an example.

The processor 601 may be a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), or a combination of a CPU, GPU, and NP. The processor 601 may also be a core in a multi-core CPU, a multi-core GPU, or a multi-core NP for implementing communication identification binding.

The processor 601 may be a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a general array logic (generic array logic, GAL) or any combination thereof.

The communication interface 602 can be used for the interaction of sending and receiving information or signaling, and the receiving and transmitting of signals. The memory 603 may mainly include a storage program area and a storage data area. The storage program area may store an operating system and a storage program required by at least one function (such as text storage function, location storage function, etc.); the storage data area may store Data (such as image data, text data) created according to the use of the device, etc., and may include application storage programs, etc. In addition, the memory 603 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

The memory 603 is also used to store program instructions. The processor 601 is configured to execute program instructions stored in the memory 603, and when the program instructions are executed, the processor 601 is configured to:

The methods executed by the processor in the embodiments of the present invention are all described from the perspective of the processor. It can be understood that the processor in the embodiments of the present invention requires the cooperation of other hardware structures to execute the foregoing methods. The embodiments of the present invention do not make detailed descriptions and restrictions on the specific implementation process.

In an embodiment, when the processor 601 determines the target data fusion mode according to the data passed the verification, it is specifically configured to:

When the data that has passed the verification includes the GNSS data, a data fusion method based on GNSS data is determined as the target data fusion method.

In an embodiment, the at least one type of SLAM sensor data includes: positioning data output by a positioning module based on an image sensor.

When the verified data includes the positioning data output by the image sensor-based positioning module and does not include the GNSS data, the positioning data output by the image sensor-based positioning module is The main data fusion method is determined as the target data fusion method.

In an embodiment, the at least one type of SLAM sensor data includes: positioning data output by a positioning module based on lidar.

When the verified data includes the positioning data output by the lidar-based positioning module and does not include the GNSS data, the positioning data output by the lidar-based positioning module is The main data fusion method is determined as the target data fusion method.

In an embodiment, the at least one type of SLAM sensor data includes: positioning data output by a positioning module based on lidar, and positioning data output by a positioning module based on image sensors.

When the verified data includes the positioning data output by the positioning module based on lidar and the positioning data output by the positioning module based on image sensors, and does not include the GNSS data, it will be The method of data fusion between the positioning data output by the positioning module mainly based on lidar and the positioning data output by the positioning module mainly based on image sensors is determined as the target data fusion manner.

In an embodiment, the processor 601 performs mutual verification on the GNSS data, the inertial navigation system data, the driving state data, and at least one SLAM sensor data, and when the data passed the verification is obtained, the specific Used for:

Convert each item of data in a data set composed of the GNSS data, the inertial navigation system data, the driving state data, and the at least one SLAM sensor data to a reference coordinate system to obtain a coordinate-transformed data set ; Perform mutual verification on the data in the data set after the coordinate conversion to obtain data that has passed the verification.

Based on the GNSS data, the inertial navigation system data, the driving state data, and at least one type of SLAM sensor data, the identification information of each data item and the frequency of the corresponding device output data The item data is detected to obtain a data set that has passed the test; the data in the data set that has passed the test are mutually verified to obtain the data that has passed the verification.

In an embodiment, when the processor determines the position of the movable platform according to the target information, it is specifically configured to:

The position of the movable platform is determined in a high-precision map according to the target information.

In specific implementation, the processor 601, the communication interface 602, and the memory 603 described in the embodiment of the present invention can execute the implementation described in the multi-data fusion-based positioning method provided in the embodiment of the present invention. Repeat.

In the embodiment of the present invention, the GNSS data, inertial navigation system data, driving state data, and at least one SLAM sensor data of the movable platform are mutually verified by the processor to obtain the data that has passed the verification, and the data is determined according to the data that has passed the verification. The target data fusion method is indicated by fusing the GNSS data, inertial navigation system data, driving status data and at least one SLAM sensor data of the movable platform to obtain target information, and determine the position of the movable platform according to the target information, thereby The mobile platform in different environments can be positioned based on different data fusion methods, effectively ensuring positioning accuracy.

An embodiment of the present invention also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the positioning based on multiple data fusion described in the above method embodiment is implemented method.

The embodiment of the present invention also provides a computer program product containing instructions, which when running on a computer, causes the computer to execute the positioning method based on multiple data fusion described in the above method embodiment.

It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described sequence of actions. Because according to the present invention, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the involved actions and modules are not necessarily required by the present invention.

The steps in the method of the embodiment of the present invention can be adjusted, merged, and deleted in order according to actual needs.

The modules in the device of the embodiment of the present invention can be combined, divided, and deleted according to actual needs.

A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium, and the storage medium can include: Flash disk, read-only memory (Read-Only Memory, ROM), random access device (Random Access Memory, RAM), magnetic disk or optical disk, etc.

The above provides a detailed introduction to a positioning method based on multiple data fusion and a movable platform provided by the embodiments of the present invention. Specific examples are used in this article to explain the principles and implementation of the present invention. The description of the above embodiments is only It is used to help understand the method and core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and the scope of application. In summary, this The contents of the description should not be construed as limiting the present invention.

Claims

A positioning method based on multiple data fusion, applied to a movable platform, characterized in that the method includes:

Acquiring global satellite navigation system (GNSS) data, inertial navigation system data, driving state data, and at least one real-time positioning and mapping (SLAM) sensor data of the mobile platform;

Perform mutual verification on the GNSS data, the inertial navigation system data, the driving state data, and at least one SLAM sensor data to obtain data that has passed the verification, and determine the target data fusion based on the data that has passed the verification the way;

Perform fusion processing on the GNSS data, inertial navigation system data, driving state data, and at least one SLAM sensor data of the movable platform according to the instructions of the target data fusion mode to obtain target information, and determine the target information according to the target information. Describe the location of the movable platform.
The method according to claim 1, wherein the determining a target data fusion mode according to the data passed in the verification comprises:

When the data that has passed the verification includes the GNSS data, a data fusion method based on GNSS data is determined as the target data fusion method.
The method according to claim 1, wherein the at least one type of SLAM sensor data comprises: positioning data output by a positioning module based on image sensors.
The method according to claim 3, wherein the determining a target data fusion mode according to the data passed in the verification comprises:

When the verified data includes the positioning data output by the image sensor-based positioning module and does not include the GNSS data, the positioning data output by the image sensor-based positioning module is The main data fusion method is determined as the target data fusion method.
The method according to claim 1, wherein the at least one type of SLAM sensor data comprises: positioning data output by a positioning module based on lidar.
The method according to claim 5, wherein the determining a target data fusion mode according to the data passed in the verification comprises:

When the verified data includes the positioning data output by the lidar-based positioning module and does not include the GNSS data, the positioning data output by the lidar-based positioning module is The main data fusion method is determined as the target data fusion method.
The method according to claim 1, wherein the at least one type of SLAM sensor data comprises: positioning data output by a positioning module based on lidar, and positioning data output by a positioning module based on image sensors.
The method according to claim 7, wherein the determining the target data fusion mode according to the data passed the verification comprises:

When the verified data includes the positioning data output by the positioning module based on lidar and the positioning data output by the positioning module based on image sensors, and does not include the GNSS data, it will be The method of data fusion between the positioning data output by the positioning module mainly based on lidar and the positioning data output by the positioning module mainly based on image sensors is determined as the target data fusion manner.
The method according to any one of claims 1 to 8, wherein the GNSS data, the inertial navigation system data, the driving state data, and at least one SLAM sensor data are mutually verified , Get the data that passed the verification, including:

Convert each item of data in a data set composed of the GNSS data, the inertial navigation system data, the driving state data, and the at least one SLAM sensor data to a reference coordinate system to obtain a coordinate-transformed data set ；

Perform mutual verification on the data in the data set after the coordinate conversion to obtain data that has passed the verification.
The method according to any one of claims 1 to 8, wherein the GNSS data, the inertial navigation system data, the driving state data, and at least one SLAM sensor data are mutually verified , Get the data that passed the verification, including:

Based on the GNSS data, the inertial navigation system data, the driving state data, and at least one type of SLAM sensor data, the identification information of each data item and the frequency of the corresponding device output data Item data is tested, and the data set that passed the test is obtained;

Perform mutual verification on the data in the detected data set to obtain data that has passed the verification.
The method according to any one of claims 1 to 8, wherein the determining the position of the movable platform according to the target information comprises:

The position of the movable platform is determined in a high-precision map according to the target information.
A movable platform, which is characterized by comprising: a memory and a processor,

The memory is used to store program instructions;

The processor is configured to execute program instructions stored in the memory, and when the program instructions are executed, the processor is configured to:

Acquiring GNSS data, inertial navigation system data, driving state data, and at least one SLAM sensor data of the movable platform;

Perform mutual verification on the GNSS data, the inertial navigation system data, the driving state data, and at least one SLAM sensor data to obtain data that has passed the verification, and determine the target data fusion based on the data that has passed the verification the way;

Perform fusion processing on the GNSS data, inertial navigation system data, driving state data, and at least one SLAM sensor data of the movable platform according to the instructions of the target data fusion mode to obtain target information, and determine the target information according to the target information. Describe the location of the movable platform.
The mobile platform according to claim 12, wherein when the processor determines the target data fusion mode according to the data passed the verification, it is specifically configured to:

When the data that has passed the verification includes the GNSS data, a data fusion method based on GNSS data is determined as the target data fusion method.
The movable platform according to claim 12, wherein the at least one type of SLAM sensor data comprises: positioning data output by a positioning module mainly based on image sensors.
The mobile platform according to claim 14, wherein when the processor determines the target data fusion mode according to the data passed the verification, it is specifically configured to:

When the verified data includes the positioning data output by the image sensor-based positioning module and does not include the GNSS data, the positioning data output by the image sensor-based positioning module is The main data fusion method is determined as the target data fusion method.
The mobile platform according to claim 12, wherein the at least one type of SLAM sensor data comprises: positioning data output by a positioning module based on lidar.
The mobile platform according to claim 16, wherein when the processor determines the target data fusion mode according to the data passed the verification, it is specifically configured to:

When the verified data includes the positioning data output by the lidar-based positioning module and does not include the GNSS data, the positioning data output by the lidar-based positioning module is The main data fusion method is determined as the target data fusion method.
The mobile platform according to claim 12, wherein the at least one type of SLAM sensor data comprises: positioning data output by a positioning module based on lidar, and positioning output by a positioning module based on image sensors data.
The mobile platform according to claim 18, wherein when the processor determines the target data fusion mode according to the data passed the verification, it is specifically configured to:

When the verified data includes the positioning data output by the positioning module based on lidar and the positioning data output by the positioning module based on image sensors, and does not include the GNSS data, it will be The method of data fusion between the positioning data output by the positioning module mainly based on lidar and the positioning data output by the positioning module mainly based on image sensors is determined as the target data fusion manner.
The mobile platform according to any one of claims 12 to 19, wherein the processor combines the GNSS data, the inertial navigation system data, the driving state data, and at least one SLAM sensor data When performing mutual verification and obtaining data that has passed the verification, it is specifically used to:

Convert each item of data in a data set composed of the GNSS data, the inertial navigation system data, the driving state data, and the at least one SLAM sensor data to a reference coordinate system to obtain a coordinate-transformed data set ；

Perform mutual verification on the data in the data set after the coordinate conversion to obtain data that has passed the verification.
The mobile platform according to any one of claims 12 to 19, wherein the processor combines the GNSS data, the inertial navigation system data, the driving state data, and at least one SLAM sensor data When performing mutual verification and obtaining data that has passed the verification, it is specifically used to:

Based on the GNSS data, the inertial navigation system data, the driving state data, and at least one type of SLAM sensor data, the identification information of each data item and the frequency of the corresponding device output data Item data is tested, and the data set that passed the test is obtained;

Perform mutual verification on the data in the detected data set to obtain data that has passed the verification.
The movable platform according to any one of claims 12 to 19, wherein when the processor determines the position of the movable platform according to the target information, it is specifically configured to:

The position of the movable platform is determined in a high-precision map according to the target information.
A computer-readable storage medium in which a computer program is stored, characterized in that: when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented .