WO2022036792A1 - 多数据源的slam方法、设备及计算机可读存储介质 - Google Patents

多数据源的slam方法、设备及计算机可读存储介质 Download PDF

Info

Publication number
WO2022036792A1
WO2022036792A1 PCT/CN2020/115672 CN2020115672W WO2022036792A1 WO 2022036792 A1 WO2022036792 A1 WO 2022036792A1 CN 2020115672 W CN2020115672 W CN 2020115672W WO 2022036792 A1 WO2022036792 A1 WO 2022036792A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
map
sweeping robot
data
state information
Prior art date
Application number
PCT/CN2020/115672
Other languages
English (en)
French (fr)
Inventor
潘俊威
Original Assignee
苏州三六零机器人科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州三六零机器人科技有限公司 filed Critical 苏州三六零机器人科技有限公司
Publication of WO2022036792A1 publication Critical patent/WO2022036792A1/zh

Links

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0242—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using non-visible light signals, e.g. IR or UV signals
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20—Instruments for performing navigational calculations
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0223—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving speed control of the vehicle
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0253—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0257—Control of position or course in two dimensions specially adapted to land vehicles using a radar
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • G05D1/028—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using a RF signal
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • G05D1/0285—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using signals transmitted via a public communication network, e.g. GSM network
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks

Definitions

  • the present application relates to the field of robotics, and in particular, to a multi-data source SLAM method, device, and computer-readable storage medium.
  • the current sweeping robot can use sensor data to construct a map, it still lacks the ability to perceive the real external environment, especially for the real ground environment.
  • the current sweeping robot can only obtain discrete two-dimensional sampling information at a specific height (usually the height of the sensor) or the sparse feature point cloud information in the top space of the sweeping robot, but lacks the ground and object information in the lower space on the ground.
  • the geometric structure and semantic information of the ground and the lower space on the ground cannot be reconstructed in the map, so that the current cleaning robot cannot solve the problem of collision with low obstacles.
  • the main purpose of this application is to provide a multi-data source SLAM method, a sweeping robot, a device and a readable storage medium, which aims to solve the problem of the sweeping robot caused by the lack of information about the low-level environment in the current environment map modeling process of the sweeping robot. Problems with poor handling of low obstacles.
  • the present application provides a multi-data source SLAM method
  • the multi-data source SLAM method includes the following steps:
  • the motion equation and the observation equation are fused by using the Bayesian recursive estimation algorithm to obtain the fused state information of the sweeping robot;
  • the depth map obtained by the sweeping robot is used to update the probability information of voxels in the map through a static Bayesian filtering algorithm to construct an environment map.
  • the step of establishing an observation equation for each sensor includes:
  • a first type of observation equation and a second type of observation equation are established, respectively.
  • the method before the step of fusing the motion equation and the observation equation by using a Bayesian recursive estimation algorithm, the method further includes:
  • Bayesian inference is performed on the fusion result and the registration result to obtain preliminary state information of the cleaning robot.
  • the motion equation and the observation equation are fused using a Bayesian recursive estimation algorithm
  • the step of acquiring the fused state information of the sweeping robot includes:
  • the Bayesian recursive estimation algorithm is used for fusion to obtain the fused state information of the sweeping robot at the current moment.
  • the steps before the step of updating the probability information of voxels in the map through a static Bayesian filtering algorithm to construct a map by using the depth map obtained by the sweeping robot according to the fused state information, the steps further include:
  • the step of updating the probability information of voxels in the map through a static Bayesian filtering algorithm to construct the map includes:
  • the probability information of voxels in the map is updated based on a static Bayesian filtering algorithm to construct a map.
  • the step of updating the probability information of voxels in the map through a static Bayesian filtering algorithm to construct a map using the depth map obtained by the sweeping robot according to the fused state information it further includes:
  • Semantic segmentation and edge extraction are performed on the objects in the infrared image to obtain object information
  • Bayesian inference is performed according to the object information to obtain an inference result, and the inference result is updated into the environment map.
  • the step of acquiring the infrared image it further includes:
  • the method further includes:
  • Bayesian inference is performed on the object information based on the reprojection error.
  • the multi-data source SLAM method further includes:
  • the present application also provides a SLAM device with multiple data sources, and the SLAM device with multiple data sources includes:
  • the first establishment module is used to establish the motion equation of the sweeping robot according to the motion constraints
  • the second establishment module is used to establish an observation equation for each sensor according to the acquired measurement data of each sensor of the sweeping robot;
  • a first fusion module configured to use a Bayesian recursive estimation algorithm to fuse the motion equation and the observation equation to obtain the fused state information of the sweeping robot;
  • the building module is used to update the probability information of voxels in the map by using the depth map obtained by the sweeping robot according to the fused state information to construct an environment map.
  • the first establishment module includes:
  • a first acquisition unit used for acquiring measurement data of each sensor of the sweeping robot
  • a dividing unit configured to divide the measurement data into fast-changing data and slow-changing data according to the sensor type corresponding to the measurement data
  • a establishing unit configured to establish a first type of observation equation and a second type of observation equation according to the measurement data belonging to the fast-changing data type and the slowly-changing data type, respectively.
  • the multi-data source SLAM device further includes:
  • a second fusion module configured to fuse the observation equations of the first type to obtain a fusion result
  • an initial value module for using the fusion result as the initial value of the point cloud registration algorithm of the second type of observation equation
  • a registration module configured to obtain a registration result according to the point cloud registration algorithm and the initial value
  • the first inference module is configured to perform Bayesian inference on the fusion result and the registration result to obtain preliminary state information of the cleaning robot.
  • the first fusion module includes:
  • a second obtaining unit configured to obtain the preliminary state information obtained by the observation equation at the current moment
  • the fusion unit is configured to use the Bayesian recursive estimation algorithm to perform fusion according to the preliminary state information and the motion equation to obtain the fusion state information of the sweeping robot at the current moment.
  • the building blocks include:
  • a determining unit configured to determine the location information of the sweeping robot in the map according to the fused state information
  • a third obtaining unit configured to obtain the distance information between each object and the position information by using the point cloud information projected by the depth map
  • the updating unit is configured to update the probability information of voxels in the map based on the static Bayesian filtering algorithm according to the distance information to construct the map.
  • the multi-data source SLAM device further includes:
  • a first acquisition module used for acquiring an infrared image
  • An update module configured to perform Bayesian inference according to the object information to obtain an inference result, and update the inference result to the environment map.
  • the multi-data source SLAM device further includes:
  • a second acquiring module configured to acquire the confidence level of each point in the infrared image and display the confidence level through a confidence level histogram
  • a deletion module is used to delete the noise points whose confidence level does not meet the preset condition in the infrared image.
  • the multi-data source SLAM device further includes:
  • a reprojection module configured to reproject the inference result to obtain a reprojection error
  • a second inference module configured to perform Bayesian inference on the object information based on the reprojection error.
  • the present application also provides a multi-data source SLAM device, the multi-data source SLAM device comprising: a memory, a processor, and a multi-data source SLAM program stored on the memory and executable on the processor , when the multi-data source SLAM program is executed by the processor to realize the steps of the above-mentioned multi-data source SLAM method
  • the present application further provides a readable storage medium, where a computer program is stored on the readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned multi-data source SLAM method are implemented.
  • the motion equation of the sweeping robot is established according to the motion constraints; the observation equation is established for each sensor according to the acquired measurement data of each sensor of the sweeping robot; the Bayesian recursive estimation algorithm is used to calculate the motion equation and the observation equation. Fusion to obtain the fused state information of the sweeping robot; according to the fused state information, using the depth map obtained by the sweeping robot, the probability information of voxels in the map is updated through a static Bayesian filtering algorithm to construct an environment map.
  • the observation equation is established by using the data of a variety of different sensors, so that more accurate position information of the sweeping robot can be obtained. At the same time, the motion equation can ensure the accuracy of the obtained position information. It makes the information in the final map more abundant, and improves the perception and processing ability of the sweeping robot to surrounding obstacles.
  • FIG. 1 is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of the present application;
  • FIG. 2 is a schematic flowchart of the first embodiment of the SLAM method for multiple data sources of the present application
  • step S30 in FIG. 2 is a schematic flowchart of the steps before step S30 in FIG. 2 in the third embodiment of the SLAM method for multiple data sources of the present application;
  • FIG. 4 is a schematic flowchart of the steps after step S40 in FIG. 2 in the sixth embodiment of the multi-data source SLAM method of the present application;
  • FIG. 5 is a schematic diagram of a system structure of an embodiment of a SLAM device with multiple data sources of the present application.
  • FIG. 1 is a schematic structural diagram of a terminal of a hardware operating environment involved in the solution of the embodiment of the present application.
  • the terminal in this embodiment of the present application is a SLAM device with multiple data sources.
  • the terminal may include: a processor 1001 , such as a CPU, a network interface 1004 , a user interface 1003 , a memory 1005 , and a communication bus 1002 .
  • the communication bus 1002 is used to realize the connection and communication between these components.
  • the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.
  • the network interface 1004 may include a standard wired interface and a wireless interface (eg, a WI-FI interface).
  • the memory 1005 may be high-speed RAM memory, or may be non-volatile memory, such as disk memory.
  • the memory 1005 may also be a storage device independent of the aforementioned processor 1001 .
  • the terminal may further include a camera, an RF (Radio Frequency, radio frequency) circuits, sensors, audio circuits, WiFi modules, etc.
  • sensors such as light sensors, motion sensors and other sensors.
  • the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and/or when the terminal device moves to the ear Backlight.
  • the terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.
  • terminal structure shown in FIG. 1 does not constitute a limitation on the terminal, and may include more or less components than the one shown, or combine some components, or arrange different components.
  • the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a SLAM program with multiple data sources.
  • the network interface 1004 is mainly used to connect to the background server and perform data communication with the background server;
  • the user interface 1003 is mainly used to connect to the client (client) and perform data communication with the client;
  • the processor 1001 can be used to invoke a multi-source SLAM program stored in memory 1005 and perform the following operations:
  • the motion equation and the observation equation are fused by using the Bayesian recursive estimation algorithm to obtain the fused state information of the sweeping robot;
  • the depth map obtained by the sweeping robot is used to update the probability information of voxels in the map through a static Bayesian filtering algorithm to construct an environment map.
  • the present application provides a multi-data source SLAM method.
  • the method includes:
  • Step S10 establishing the motion equation of the sweeping robot according to the motion constraints
  • the SLAM (Simultaneous Localization And Mapping) method is a commonly used method in the field of sweeping robots, which enables the sweeping robot to scan a map of the surrounding environment while moving, so that the sweeping robot can autonomously locate and navigate.
  • the sweeping robot needs to obtain the relevant data of the surrounding environment through sensors such as radar for its own positioning and the construction of the surrounding map.
  • sensors such as radar for its own positioning and the construction of the surrounding map.
  • the motion equation of the sweeping robot can be established through motion constraints such as Newton's second law.
  • the equation is used to express the conditional relationship that should be satisfied between two different states of the sweeping robot when the motion constraints are satisfied, wherein one independent variable in the motion equation is the time interval between the two states. For each sweeping robot, its motion equation is uniquely determined. At the same time, the motion equation is the most basic condition that the sweeping robot should meet, and it is also the prior information for judging the current state of the sweeping robot.
  • Step S20 establishing an observation equation for each sensor according to the acquired measurement data of each sensor of the sweeping robot;
  • the sensors to be used mainly include IMU (Inertial Measurement Unit, inertial measurement unit), wheel encoder, radar, depth camera, etc., among which IMU and vehicle encoder are mainly used to obtain motion data such as acceleration, angular velocity, rotational speed, etc. of the sweeping robot, while radar and depth camera can be used with It is used to obtain information about the surrounding environment or objects, such as the location and size of obstacles.
  • IMU Inertial Measurement Unit
  • inertial measurement unit wheel encoder
  • radar depth camera
  • It is used to obtain information about the surrounding environment or objects, such as the location and size of obstacles.
  • Step S30 using a Bayesian recursive estimation algorithm to fuse the motion equation and the observation equation to obtain the fused state information of the sweeping robot;
  • the state judgment of the sweeping robot at the current moment is an estimated value.
  • this estimated value may be exactly the same as the actual state, or there may be certain errors. Of course, , even if there is an error, this error is in the controllable range.
  • the motion equation is used as a priori information. The motion equation can be used to ensure that the error between the final state quantity and the real state quantity is small, and the observation equation at the current moment can be obtained.
  • the Bayesian recursive estimation algorithm obtains the posterior distribution of the parameter according to the prior distribution of the parameter and a series of observed values, and then obtains the expected value of the parameter as its final value.
  • a variance amount of the parameter is defined to evaluate the accuracy or confidence of the parameter estimate.
  • the motion equation is the prior distribution in the Bayesian recursive estimation algorithm, and the observation equation can provide a series of observation values at different times, so that the state quantity of the sweeping robot at the current moment and the corresponding confidence level can be obtained.
  • the confidence level represents the estimation The probability that the value and state parameters are within a certain allowable error range.
  • Bayesian will continuously use the state quantity obtained in the previous time to calculate the next state quantity, thus forming a recursive process.
  • the reliability of the motion equation and the observation equation will affect the fusion state information of the sweeping robot finally obtained by the Bayesian recursive estimation algorithm.
  • the state information display includes the position, attitude, linear velocity, angular velocity and acceleration of the sweeping robot.
  • Step S40 according to the fused state information, using the depth map obtained by the sweeping robot, and updating the probability information of the voxels in the map through a static Bayesian filtering algorithm to construct an environment map;
  • the current position and posture of the sweeping robot can be known.
  • the depth map can be obtained by the depth camera, and the depth map can directly record the absolute distance between the object in the world space and the depth camera.
  • the positional relationship between the objects around the sweeping robot and the sweeping robot can be known.
  • the state at the current moment is determined by the state at the previous moment and the action at the current moment, and the process at the current moment is only related to the state at the current moment.
  • the probability information of the voxels in the map at the current moment can be obtained from the position information of the object provided by the depth map at the current moment and the corresponding probability information of the voxels in the map at the previous moment to construct the environment.
  • Map that is, through the probability information of each voxel in the map to indicate whether there is an object in the voxel, and by combining the information of a voxel, you can know where the environmental information in the map exists and where the object's edge is.
  • the motion equation of the sweeping robot is established according to the motion constraints; the observation equation is established for each sensor according to the acquired measurement data of each sensor of the sweeping robot; the Bayesian recursive estimation algorithm is used to analyze the motion equation and the The observation equations are fused to obtain the fused state information of the sweeping robot; according to the fused state information, the depth map obtained by the sweeping robot is used to update the probability information of voxels in the map through a static Bayesian filtering algorithm to construct an environment map.
  • the observation equation is established by using the data of a variety of different sensors, so that more accurate position information of the sweeping robot can be obtained.
  • the motion equation can ensure the accuracy of the obtained position information. It makes the information in the final map more abundant, and improves the perception and processing ability of the sweeping robot to surrounding obstacles.
  • a second embodiment of the multi-data source SLAM method is provided.
  • the second embodiment a second embodiment of the multi-data source SLAM method is provided.
  • Step S20 includes:
  • Step A1 acquiring the measurement data of each sensor of the sweeping robot
  • Sensors include IMU, wheel encoder, radar, depth camera, etc.
  • the gyroscope and accelerometer in the IMU can obtain various accelerations in the three-axis direction of the sweeping robot.
  • the radar can obtain the point cloud information of the surrounding environment, and the depth map and infrared map of the surrounding environment can be obtained through the depth camera.
  • Step A2 according to the sensor type corresponding to the measurement data, divide the measurement data into fast-changing data and slow-changing data;
  • Each sensor has corresponding data acquisition characteristics.
  • One of the important images is the frequency of data acquisition by the sensor.
  • it is divided into fast-changing data and slow-changing data.
  • Fast-changing data is a sensor with a higher scanning frequency, such as The data obtained by gyroscopes and accelerometers, etc.
  • the slow-changing data is the data obtained by sensors with low scanning frequency, such as radar, depth camera, etc.
  • the data obtained by the sensors corresponding to the fast-changing data such as gyroscopes or accelerometers
  • the acquisition period of the fast-changing data is shorter, and the acquisition period of the second slow-changing data is longer.
  • Step A3 according to the measurement data belonging to the fast-changing data type and the slow-changing data type, respectively establishing a first type of observation equation and a second type of observation equation;
  • observation equation After dividing the data into fast-changing data and slow-changing data, correspondingly, the established observation equations are divided into a first type of observation equation and a second type of observation equation.
  • the observation equation can be abstracted into the following expression:
  • z k,j is the observation data
  • y j is the landmark point
  • x k is the position of the sweeping robot
  • v k, j is the noise existing during the observation
  • h() is an abstract functional relationship.
  • the time interval between time k and time k+1 of the first type of observation equation and the second type of observation equation is different.
  • the data obtained by the observation equations are classified according to the types of sensors, and the corresponding types of observation equations are established respectively, and the observation equations are divided into different types to make the fusion result more accurate in the subsequent fusion process.
  • a third embodiment of the multi-data source SLAM method is provided.
  • the third embodiment
  • step S30 it includes:
  • Step S31 fuse the observation equations of the first type to obtain a fusion result
  • the first type of observation equation is an observation equation constructed by using fast-changing data, and by fusing the first type of observation equation at the initial moment, such as the speed, acceleration, angular velocity and other observation data obtained from a gyroscope and an accelerometer, so that The fusion result is obtained to obtain the initial state information of the sweeping robot.
  • Step S32 using the fusion result as the initial value of the point cloud registration algorithm of the second type of observation equation
  • the second type of observation equation is the observation equation constructed by using slowly changing data. Because the time interval of data acquisition for slow-changing data is longer, and the time interval for data acquisition for fast-changing data is shorter. When the sweeping robot does not fail, the state data of the sweeping robot obtained through fast-changing data or slow-changing data should be the same. Therefore, the fusion result of the first type of observation equation can be used as the initial value of the second type of observation equation.
  • Step S33 obtaining a registration result according to the point cloud registration algorithm and the initial value
  • the point cloud registration algorithm is used to obtain the pose transformation between two different frame images. Because two different images need to be compared, the fusion result of the first type of observation equation is used as the initial value, that is, the initial frame, and then the point cloud registration algorithm starts from the initial frame to obtain the position of the adjacent two frames of images.
  • the pose transformation result is the registration result, so that the pose information of the sweeping robot at each moment can be obtained.
  • Step S34 performing Bayesian inference on the fusion result and the registration result to obtain preliminary state information of the sweeping robot
  • Bayesian inference can use the existing knowledge, which is the existing knowledge in this application.
  • the fusion result of the first type and the registration result of the observation equation of the second type are used to infer the probability of a new event.
  • what is obtained by Bayesian inference is the preliminary state of the sweeping robot at the current moment. The best estimate of the information and the corresponding confidence.
  • the corresponding state information is obtained for the first type of observation equation and the second type of observation equation respectively, and then Bayesian inference is used to fuse the results obtained from the two types of observation equations to obtain the sweeping robot the initial state information, so that the obtained initial state information is more accurate.
  • a fourth embodiment of the multi-data source SLAM method is provided.
  • the fourth embodiment
  • Step S30 includes:
  • Step B1 obtaining the preliminary state information obtained by the observation equation at the current moment
  • the preliminary state information of the current moment is obtained through the methods of steps S31 to S34 in the third embodiment.
  • Step B2 according to the preliminary state information, combined with the equation of motion, using the Bayesian recursive estimation algorithm to perform fusion, to obtain the fusion state information of the sweeping robot at the current moment;
  • the preliminary state information is obtained from the observation equation, but not combined with the motion equation.
  • the equations of motion can also be expressed as expressions related to the time interval between two states.
  • the motion equation is related to the motion constraint, and the motion constraint is to ensure that the final state information will not violate the objective physical laws, that is, the accuracy of the state information can be guaranteed.
  • the Bayesian recursive estimation algorithm obtains the posterior distribution of the parameter based on the prior distribution of the parameter and a series of observed values, and then obtains the expected value of the parameter as its final value.
  • the motion equation is the prior distribution in the Bayesian recursive estimation algorithm, and the observation equation can provide a series of observation values at different times, so that the state quantity and corresponding confidence of the sweeping robot at the current moment can be obtained. Therefore, through the Bayesian recursive estimation algorithm, the sweeping robot uses the confidence of the previous moment to calculate the prediction confidence of the current moment, and then uses the observation value of the observation equation and the prediction confidence of the current moment to obtain the Bayesian confidence and use this to update the current status information of the sweeping robot.
  • the current state information and the corresponding confidence level of the sweeping robot are continuously updated through the Bayesian recursive estimation algorithm by using the motion equation and the observation equation.
  • the Bayesian recursive estimation algorithm makes the obtained state information of the sweeping robot. The closest to the real state information, thereby improving the accuracy of the entire positioning.
  • a fifth embodiment of the multi-data source SLAM method is provided.
  • the fifth embodiment
  • step S40 it also includes:
  • Step C1 performing down-sampling processing on the depth map obtained by the sweeping robot
  • the depth map obtained by the depth camera is down-sampled, and the image resolution of the depth map is appropriately reduced. While reducing the amount of data contained in the depth map, try not to affect the probability update of voxels in the map using the depth map later.
  • Step S40 includes:
  • Step C2 determining the location information of the sweeping robot in the map according to the fused state information
  • the state information includes the posture, linear velocity, angular velocity, acceleration and other information of the sweeping robot. Therefore, the optimal estimation of the state information of the sweeping robot obtained by the improved Bayesian recursive estimation algorithm is used, and then obtained according to the estimated state information.
  • the current posture of the sweeping robot is the position and posture information.
  • Step C3 using the point cloud information projected by the depth map to obtain the distance information of each object and the position information;
  • Each pixel value in the depth map represents the distance from the object to the camera plane.
  • the corresponding point cloud information can be projected according to the depth map, that is, the position of each object in the surrounding environment. and edge information.
  • the distance information between each object and the current position information of the cleaning robot can be known.
  • Step C4 according to the distance information, update the probability information of voxels in the map based on the static Bayesian filtering algorithm to construct the map;
  • the specific Bayesian filtering algorithm is to judge the state of the current time by the state information of the previous time and the action of the current time. In this application, it uses the depth map according to the probability information of each voxel at the previous time and the current time. The obtained observation results are used to obtain the new probability information of each voxel at the current moment, so that the sweeping robot can continuously update and construct the map.
  • the depth map can accurately detect the location information of low objects in the environment, so that more abundant and complete map information can be obtained in the process of constructing the map.
  • the state information of the sweeping robot can be determined by combining the motion equation and the observation equation. , so that the accurate positioning of the sweeping robot can be realized, and the accuracy of the established map can be further improved.
  • the point cloud information of the depth map is used to determine the position information of each object in the environment, and the probability information of voxels in the map is continuously updated according to the static Bayesian filtering algorithm, so as to construct the map.
  • the depth map can contain low
  • the information of dwarf objects ensures the accuracy of the object information of the final constructed map.
  • a sixth embodiment of the multi-data source SLAM method is provided.
  • the sixth embodiment
  • step S40 it also includes:
  • Step S50 acquiring an infrared image
  • decoration materials are complex and diverse, and some materials are difficult to measure by optical sensors, such as light-transmitting materials, reflective materials, and light-absorbing materials, so that the depth camera has problems in the measurement of problems involving these materials.
  • optical sensors such as light-transmitting materials, reflective materials, and light-absorbing materials
  • the depth camera has problems in the measurement of problems involving these materials.
  • Inability to measure or inaccurate measurement leads to defects in the positioning accuracy of the final map and its consistency with the actual environment. Therefore, it is also necessary to use the infrared image obtained by the depth camera.
  • Step S60 performing semantic segmentation and edge extraction on the objects in the infrared image to obtain object information
  • the one in the infrared image shows the spectral information of different objects, and the position and contour information of the corresponding material can be obtained for light-transmitting materials, reflective materials and light-absorbing materials.
  • For the objects in the infrared image perform semantic return and edge extraction, that is, according to the previously trained semantic information such as desks, stools, etc., to judge each object in the infrared image, and then perform preliminary segmentation of the corresponding objects and obtain the preliminary information of the objects. position information, and then use the spectral information in the infrared image to extract the edge of each object to obtain accurate object position information and edge information.
  • Step S70 performing Bayesian inference according to the object information to obtain an inference result, and updating the inference result to the environment map;
  • the object information includes the position information and edge information of the object, that is, where the object is in the room and the position information corresponding to the boundary points in the edge of the object, so that the specific information of the object can be constructed through the position information and edge information of the object.
  • the Bayesian inference here is to use the existing probability information of each voxel in the existing map, combined with the object information extracted through semantic segmentation and edge extraction, to update the probability information of each voxel in the map, so as to update the map and be able to
  • the relevant information of the objects that are difficult to be measured by the optical sensor is added to the constructed map, so as to ensure the consistency of the final constructed map information and the actual environment information.
  • more object information in the environment is provided through the infrared image, and the probability information of each voxel in the map is updated through the Bayesian inference method to obtain a map that is more consistent with the actual environment.
  • a seventh embodiment of the multi-data source SLAM method is provided.
  • the seventh embodiment
  • step S50 it also includes:
  • Step D1 obtaining the confidence level of each point in the infrared image and displaying the confidence level through a confidence level histogram
  • the confidence histogram method is to count the confidence information of each point in the infrared image, and count the confidence of each point through the histogram, and display the statistical results of the confidence according to the histogram, and the distribution of the overall confidence in the infrared image can be obtained. condition.
  • Step D2 delete the noise whose confidence level does not meet the preset condition in the infrared image
  • the number of points in the infrared image that exist in different confidence value intervals can be known through the histogram, where the confidence value is lower than the preset value and the confidence interval in which the number of points is contained is smaller than the preset number of intervals That is, the points that do not meet the preset conditions are also called noise points in the infrared image, and the noise points are removed to improve the accuracy of the subsequent map construction process.
  • the infrared image can also be down-sampled to reduce the data pressure on the processor of the cleaning robot.
  • the points in the infrared image are analyzed through the confidence histogram, and the noise points that do not meet the preset conditions are deleted, thereby improving the accuracy when constructing the map.
  • an eighth embodiment of the multi-data source SLAM method is provided.
  • the eighth embodiment
  • step S60 it also includes:
  • Step E1 based on the environment map, reproject the inference result to obtain a reprojection error
  • the point cloud information in the infrared image obtained by the depth camera is reprojected to the point cloud information with the radar as the reference plane, and the error between the two point cloud information is judged as the reprojection error.
  • Step E2 performing Bayesian inference on the object information based on the reprojection error
  • reprojection is performed according to different coordinate systems and the reprojection error is obtained, and Bayesian inference is performed by using the reprojection error to ensure the consistency of the information of each voxel in the constructed environment map.
  • a ninth embodiment of the multi-data source SLAM method is provided.
  • the ninth embodiment
  • Multi-source SLAM methods also include:
  • Step F1 compare the result error of the observation equation and the motion equation
  • Step F2 if the result error is greater than a preset threshold, increase the corresponding error times;
  • Step F3 if the number of errors is greater than the preset number of times, output prompt information
  • the motion equation is an equation constructed based on motion constraints to ensure the rationality of the finally obtained state information of the sweeping robot.
  • the observation equation is established by the observation data of the sensor. Therefore, if the error between the process equation and the motion equation exceeds the preset threshold.
  • the accuracy of the sensor is detected by comparing the error between the motion equation and the observation equation.
  • prompt information is output to prompt the user.
  • an embodiment of the present application further proposes a SLAM device with multiple data sources, and the SLAM device with multiple data sources includes:
  • the first establishment module is used to establish the motion equation of the sweeping robot according to the motion constraints
  • the second establishment module is used to establish an observation equation for each sensor according to the acquired measurement data of each sensor of the sweeping robot;
  • a first fusion module configured to use a Bayesian recursive estimation algorithm to fuse the motion equation and the observation equation to obtain the fused state information of the sweeping robot;
  • the building module is used to update the probability information of voxels in the map by using the depth map obtained by the sweeping robot according to the fused state information to construct an environment map.
  • the first establishment module includes:
  • a first acquisition unit used for acquiring measurement data of each sensor of the sweeping robot
  • a dividing unit configured to divide the measurement data into fast-changing data and slow-changing data according to the sensor type corresponding to the measurement data
  • a establishing unit configured to establish a first type of observation equation and a second type of observation equation according to the measurement data belonging to the fast-changing data type and the slowly-changing data type, respectively.
  • the multi-data source SLAM device further includes:
  • a second fusion module configured to fuse the observation equations of the first type to obtain a fusion result
  • an initial value module for using the fusion result as the initial value of the point cloud registration algorithm of the second type of observation equation
  • a registration module configured to obtain a registration result according to the point cloud registration algorithm and the initial value
  • the first inference module is configured to perform Bayesian inference on the fusion result and the registration result to obtain preliminary state information of the cleaning robot.
  • the first fusion module includes:
  • a second obtaining unit configured to obtain the preliminary state information obtained by the observation equation at the current moment
  • the fusion unit is configured to use the Bayesian recursive estimation algorithm to perform fusion according to the preliminary state information and the motion equation to obtain the fusion state information of the sweeping robot at the current moment.
  • the multi-data source SLAM device further includes:
  • the downsampling module is used to downsample the depth map obtained by the sweeping robot.
  • the building blocks include:
  • a determining unit configured to determine the location information of the sweeping robot in the map according to the fused state information
  • a third obtaining unit configured to obtain the distance information between each object and the position information by using the point cloud information projected by the depth map
  • the updating unit is configured to update the probability information of voxels in the map based on the static Bayesian filtering algorithm according to the distance information to construct the map.
  • the multi-data source SLAM device further includes:
  • a first acquisition module used for acquiring an infrared image
  • An update module configured to perform Bayesian inference according to the object information to obtain an inference result, and update the inference result to the environment map.
  • the multi-data source SLAM device further includes:
  • a second acquiring module configured to acquire the confidence level of each point in the infrared image and display the confidence level through a confidence level histogram
  • a deletion module is used to delete the noise points whose confidence level does not meet the preset condition in the infrared image.
  • the multi-data source SLAM device further includes:
  • a reprojection module configured to reproject the inference result to obtain a reprojection error
  • the second inference module is configured to perform Bayesian inference on the object information based on the reprojection error.
  • the multi-data source SLAM device further includes:
  • a comparison module for comparing the result error of the observation equation and the motion equation
  • the output module is configured to output prompt information if the number of errors is greater than the preset number of times.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Automation & Control Theory (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Algebra (AREA)
  • Electromagnetism (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Manipulator (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

一种多数据源的SLAM方法、装置、设备和可读存储介质，该方法包括：根据运动约束建立扫地机器人的运动方程（S10）；根据获取的扫地机器人的各传感器的测量数据，对各传感器建立观测方程（S20）；利用贝叶斯递归估计算法对运动方程和观测方程进行融合，获取扫地机器人的融合后的状态信息（S30）；根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建环境地图（S40）。

Description

多数据源的SLAM方法、设备及计算机可读存储介质
本申请要求于2020年8月21日申请的、申请号为202010853891.0、名称为“多数据源的SLAM方法、装置、扫地机器人及可读介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及机器人技术领域，尤其涉及一种多数据源的SLAM方法、设备及计算机可读存储介质。
 背景技术
目前的扫地机器人，虽然可以利用传感器数据构建出地图，但是对于真实的外部环境还是缺乏感知能力，尤其是对于真实的地面环境。目前的扫地机器人仅能获取特定高度（通常为传感器所在高度）的离散二维采样信息或者扫地机器人顶部空间稀疏的特征点云信息，但是缺少地面以及地面上较低空间中的物体信息。从而无法在地图中重建地面以及地面上较低空间中的几何结构和语义信息，从而导致目前的扫地机器人无法解决与低矮障碍物的碰撞问题。
 技术解决方案
本申请的主要目的在于提供一种多数据源的SLAM方法、扫地机器人、装置及可读存储介质，旨在解决目前扫地机器人的环境地图建模过程中对于低矮环境的信息缺失导致的扫地机器人对低矮障碍物处理能力较差的问题。
为实现上述目的，本申请提供的一种多数据源的SLAM方法，所述多数据源的SLAM方法包括以下步骤：
根据运动约束建立扫地机器人的运动方程；
根据获取的扫地机器人的各传感器的测量数据，对各传感器建立观测方程；
利用贝叶斯递归估计算法对所述运动方程和所述观测方程进行融合，获取扫地机器人的融合后的状态信息；
根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建环境地图。
在一实施例中，根据获取的扫地机器人的各传感器的测量数据，对各传感器建立观测方程的步骤包括：
获取扫地机器人的各传感器的测量数据；
根据所述测量数据对应的传感器类型，将所述测量数据划分为快变数据和慢变数据；
根据属于所述快变数据类型和所述慢变数据类型的所述测量数据，分别建立第一类型的观测方程和第二类型的观测方程。
在一实施例中，利用贝叶斯递归估计算法对所述运动方程和所述观测方程进行融合的步骤之前，还包括：
将所述第一类型的观测方程进行融合以得到融合结果；
将所述融合结果作为第二类型的观测方程的点云配准算法的初值；
根据所述点云配准算法以及所述初值，得到配准结果；
对所述融合结果与所述配准结果进行贝叶斯推断以获取扫地机器人的初步状态信息。
在一实施例中，利用贝叶斯递归估计算法对所述运动方程和所述观测方程进行融合，获取扫地机器人的融合后的状态信息的步骤包括：
获取当前时刻由所述观测方程得到的初步状态信息；
根据所述初步状态信息，结合运动方程，利用所述贝叶斯递归估计算法进行融合，得到当前时刻扫地机器人的融合后的状态信息。
在一实施例中，根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建地图的步骤之前还包括：
对扫地机器人获取的深度图进行降采样处理。
在一实施例中，根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建地图的步骤包括：
根据所述融合后的状态信息确定扫地机器人在地图中的位置信息；
利用深度图投影的点云信息，获取各物体与所述位置信息的距离信息；
根据所述距离信息，基于静态贝叶斯滤波算法更新地图中体素的概率信息以构建地图。
在一实施例中，根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建地图的步骤之后，还包括：
获取红外图；
对所述红外图中的物体进行语义分割和边缘提取以得到物体信息；
根据所述物体信息进行贝叶斯推断以获取推断结果，并将所述推断结果更新到所述环境地图中。
在一实施例中，获取红外图的步骤之后，还包括：
获取所述红外图中各点的置信度并通过置信度直方图显示所述置信度；
删除所述红外图中置信度不满足预设条件的噪点。
在一实施例中，根据所述物体信息进行贝叶斯推断以获取推断结果的步骤之后，还包括：
对所述推断结果进行重投影以获取重投影误差；
基于所述重投影误差对所述物体信息进行贝叶斯推断。
在一实施例中，多数据源的SLAM方法还包括：
比较所述观测方程与所述运动方程的结果误差；
若所述结果误差大于预设阈值，则增加相应的出错次数；
若所述出错次数大于预设次数，则输出提示信息。
本申请还提供一种多数据源的SLAM装置，所述多数据源的SLAM装置包括：
第一建立模块，用于根据运动约束建立扫地机器人的运动方程；
第二建立模块，用于根据获取的扫地机器人的各传感器的测量数据，对各传感器建立观测方程；
第一融合模块，用于利用贝叶斯递归估计算法对所述运动方程和所述观测方程进行融合，获取扫地机器人的融合后的状态信息；
构建模块，用于根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建环境地图。
在一实施例中，第一建立模块包括：
第一获取单元，用于获取扫地机器人的各传感器的测量数据；
划分单元，用于根据所述测量数据对应的传感器类型，将所述测量数据划分为快变数据和慢变数据；
建立单元，用于根据属于所述快变数据类型和所述慢变数据类型的所述测量数据，分别建立第一类型的观测方程和第二类型的观测方程。
在一实施例中，多数据源的SLAM装置还包括：
第二融合模块，用于将所述第一类型的观测方程进行融合以得到融合结果；
初值模块，用于将所述融合结果作为第二类型的观测方程的点云配准算法的初值；
配准模块，用于根据所述点云配准算法以及所述初值，得到配准结果；
第一推断模块，用于对所述融合结果与所述配准结果进行贝叶斯推断以获取扫地机器人的初步状态信息。
在一实施例中，第一融合模块包括：
第二获取单元，用于获取当前时刻由所述观测方程得到的初步状态信息；
融合单元，用于根据所述初步状态信息，结合运动方程，利用所述贝叶斯递归估计算法进行融合，得到当前时刻扫地机器人的融合后的状态信息。
在一实施例中，构建模块包括：
确定单元，用于根据所述融合后的状态信息确定扫地机器人在地图中的位置信息；
第三获取单元，用于利用深度图投影的点云信息，获取各物体与所述位置信息的距离信息；
更新单元，用于根据所述距离信息，基于静态贝叶斯滤波算法更新地图中体素的概率信息以构建地图。
在一实施例中，多数据源的SLAM装置还包括：
第一获取模块，用于获取红外图；
得到模块，用于对所述红外图中的物体进行语义分割和边缘提取以得到物体信息；
更新模块，用于根据所述物体信息进行贝叶斯推断以获取推断结果，并将所述推断结果更新到所述环境地图中。
在一实施例中，多数据源的SLAM装置还包括：
第二获取模块，用于获取所述红外图中各点的置信度并通过置信度直方图显示所述置信度；
删除模块，用于删除所述红外图中置信度不满足预设条件的噪点。
在一实施例中，多数据源的SLAM装置还包括：
重投影模块，用于对所述推断结果进行重投影以获取重投影误差；
第二推断模块，用于基于所述重投影误差对所述物体信息进行贝叶斯推断。
本申请还提供一种多数据源的SLAM设备，所述多数据源的SLAM设备包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的多数据源的SLAM程序，所述多数据源的SLAM程序被所述处理器执行时实现如上述的多数据源的SLAM方法的步骤
本申请还提供一种可读存储介质，所述可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如上述的多数据源的SLAM方法的步骤。
本申请中根据运动约束建立扫地机器人的运动方程；根据获取的扫地机器人的各传感器的测量数据，对各传感器建立观测方程；利用贝叶斯递归估计算法对所述运动方程和所述观测方程进行融合，获取扫地机器人的融合后的状态信息；根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建环境地图。利用多种不同传感器的数据建立观测方程，从而能够得到扫地机器人更为精确的位置信息，同时运动方程能够保证获取的位置信息的准确性，同时多传感器能够通过周围环境中更加多样的信息，从而使最终更加的地图中的信息更加丰富，提高扫地机器人对周围障碍物的感知与处理能力。
 附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请的实施例，并与说明书一起用于解释本申请的原理。
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例方案涉及的硬件运行环境的装置结构示意图；
图2为本申请多数据源的SLAM方法第一实施例的流程示意图；
图3为本申请多数据源的SLAM方法第三实施例中对图2步骤S30之前步骤的流程示意图；
图4为本申请多数据源的SLAM方法第六实施例中对图2步骤S40之后步骤的流程示意图；
图5为本申请多数据源的SLAM装置一实施例的系统结构示意图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
 本发明的实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
在后续的描述中，使用用于表示元件的诸如“模块”、“部件”或“单元”的后缀仅为了有利于本申请的说明，其本身没有特定的意义。因此，“模块”、“部件”或“单元”可以混合地使用。
如图1所示，图1是本申请实施例方案涉及的硬件运行环境的终端结构示意图。
本申请实施例终端为多数据源的SLAM设备。
如图1所示，该终端可以包括：处理器1001，例如CPU，网络接口1004，用户接口1003，存储器1005，通信总线1002。其中，通信总线1002用于实现这些组件之间的连接通信。用户接口1003可以包括显示屏（Display）、输入单元比如键盘（Keyboard），可选用户接口1003还可以包括标准的有线接口、无线接口。网络接口1004可选的可以包括标准的有线接口、无线接口（如WI-FI接口）。存储器1005可以是高速RAM存储器，也可以是稳定的存储器（non-volatile memory），例如磁盘存储器。存储器1005可选的还可以是独立于前述处理器1001的存储装置。
在一实施例中，终端还可以包括摄像头、RF（Radio Frequency，射频）电路，传感器、音频电路、WiFi模块等等。其中，传感器比如光传感器、运动传感器以及其他传感器。具体地，光传感器可包括环境光传感器及接近传感器，其中，环境光传感器可根据环境光线的明暗来调节显示屏的亮度，接近传感器可在终端设备移动到耳边时，关闭显示屏和/或背光。当然，终端设备还可配置陀螺仪、气压计、湿度计、温度计、红外线传感器等其他传感器，在此不再赘述。
本领域技术人员可以理解，图1中示出的终端结构并不构成对终端的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
如图1所示，作为一种计算机存储介质的存储器1005中可以包括操作系统、网络通信模块、用户接口模块以及多数据源的SLAM程序。
在图1所示的终端中，网络接口1004主要用于连接后台服务器，与后台服务器进行数据通信；用户接口1003主要用于连接客户端（用户端），与客户端进行数据通信；而处理器1001可以用于调用存储器1005中存储的多数据源的SLAM程序，并执行以下操作：
根据运动约束建立扫地机器人的运动方程；
根据获取的扫地机器人的各传感器的测量数据，对各传感器建立观测方程；
利用贝叶斯递归估计算法对所述运动方程和所述观测方程进行融合，获取扫地机器人的融合后的状态信息；
根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建环境地图。
基于上述终端硬件结构，提出本申请各个实施例。
本申请提供一种多数据源的SLAM方法。
参照图2，在多数据源的SLAM方法第一实施例中，该方法包括：
步骤S10，根据运动约束建立扫地机器人的运动方程；
SLAM（Simultaneous Localization And Mapping，同步定位与建图）方法是扫地机器人领域中常用的一种方法，能够使扫地机器人一边移动一边扫描出周围环境的地图，从而使扫地机器人能够自主定位与导航。在SLAM方法中，扫地机器人需要通过传感器如雷达来获取周围环境的相关数据来进行自身的定位以及周围地图的构建。在现实生活中，受扫地机器人或物理规律如牛顿第二定律的限制，扫地机器人的运动状态的变化不可能是任意的，因此通过运动约束如牛顿第二定律可以建立扫地机器人的运动方程，运动方程用于表示满足运动约束时扫地机器人的两个不同状态间应该满足的条件关系，其中，运动方程中的一个自变量是两个状态间的时间间隔。对于每一个扫地机器人，其运动方程是唯一确定的。同时运动方程是扫地机器人应满足的最基础的条件，也是对于扫地机器人当前时刻状态判断的先验信息。
步骤S20，根据获取的扫地机器人的各传感器的测量数据，对各传感器建立观测方程；
在本申请中，需要使用的传感器主要包括IMU（Inertial Measurement Unit，惯性测量单元）、车轮编码器、雷达、深度相机等，其中IMU和车辆编码器主要是用于获取扫地机器人的运动数据如加速度、角速度、转速等，而雷达和深度相机则可以用于获取周围环境或物体的信息如障碍物的位置、尺寸等。因为各传感器获取的测量数据甚至数据的类型不同，因此对于各传感器分别建立相应的观测方程，观测方程表示根据当前时刻的状态量与各传感器获取的测量数据的关系，根据观测方程可以由扫地机器人的前一个状态推测出后一个时刻扫地机器人的状态。
步骤S30，利用贝叶斯递归估计算法对所述运动方程和所述观测方程进行融合，获取扫地机器人的融合后的状态信息；
因为传感器的误差，构建地图时的建图误差等原因，导致扫地机器人对于当前时刻的状态判断是一个估计值，同时这个估计值可能与实际的状态可能完全相同，也可能存在一定的误差，当然，即使存在误差，这个误差也在可控的范围中。在获取扫地机器人融合后的状态信息时，以运动方程作为先验信息，运动方程可以用于保证最终得到的状态量与真实的状态量的误差较小，而通过当前时刻的观测方程则可以获取当前时刻的状态量，贝叶斯递归估计算法根据参数的先验分布和一系列观察值，求出参数的后验分布，然后求出参数的期望值，作为其最终值。另外还定义了参数的一个方差量，来评估参数估计的准确程度或者置信度。其中运动方程为贝叶斯递归估计算法中的先验分布，而观测方程则可以提供不同时刻的一系列观测值，从而能够得到扫地机器人当前时刻的状态量以及相应的置信度，置信度表示估计值与状态参数在一定允许的误差范围内的概率。贝叶斯根据递归算法会不断利用前一次求得的状态量来计算下一次的状态量，从而形成递归的过程。在贝叶斯递归估计算法中，运动方程以及观测方程的可靠性都会影响最终通过贝叶斯递归估计算法求得的扫地机器人的融合后的状态信息。状态信息展示包括扫地机器人的位置、姿态、线速度、角速度和加速度等。
步骤S40，根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建环境地图；
通过融合后的状态信息，可以知道扫地机器人当前的位置以及姿态。深度图可以由深度相机获取，深度图可以直接记录物体在世界空间与深度相机的绝对距离。通过深度图的相关信息，即可知道扫地机器人周围的物体与扫地机器人的位置关系。在贝叶斯滤波算法中，当前时刻的状态由之前时刻的状态以及当前时刻的动作确定，同时当前时刻的过程只与当前时刻的状态有关。根据静态贝叶斯滤波算法，则可以由当前时刻的深度图提供的物体的位置信息以及相应的前一时刻中地图的体素的概率信息获取当前时刻的地图中体素的概率信息来构建环境地图，即通过地图中各体素的概率信息来表示体素中是否存在物体，通过一个个体素的信息综合起来即可知道地图中环境信息如在何处存在物体以及物体的边缘都在哪里。
在本实施例中，根据运动约束建立扫地机器人的运动方程；根据获取的扫地机器人的各传感器的测量数据，对各传感器建立观测方程；利用贝叶斯递归估计算法对所述运动方程和所述观测方程进行融合，获取扫地机器人的融合后的状态信息；根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建环境地图。利用多种不同传感器的数据建立观测方程，从而能够得到扫地机器人更为精确的位置信息，同时运动方程能够保证获取的位置信息的准确性，同时多传感器能够通过周围环境中更加多样的信息，从而使最终更加的地图中的信息更加丰富，提高扫地机器人对周围障碍物的感知与处理能力。
进一步地，在本申请多数据源的SLAM方法第一实施例的基础上，提供多数据源的SLAM方法第二实施例，在第二实施例中，
步骤S20包括：
步骤A1，获取扫地机器人的各传感器的测量数据；
传感器包括IMU、车轮编码器、雷达、深度相机等，其中IMU中的陀螺仪、加速度计可以获取扫地机器人三轴方向上的各种加速度。雷达可以获取周围环境的点云信息，通过深度相机可以获取周围环境的深度图以及红外图。
步骤A2，根据所述测量数据对应的传感器类型，将所述测量数据划分为快变数据和慢变数据；
各传感器都有对应的获取数据的特性，其中一个重要的图像就是传感器获取数据的频率，根据传感器获取数据的频率分为快变数据和慢变数据，快变数据为扫描频率较高的传感器如陀螺仪和加速度计等获取的数据，慢变数据为是扫描频率较低的传感器如雷达、深度相机等获取的数据，快变数据对应的传感器如陀螺仪或加速度计等其获取的数据可以说是实时获取或者说是连续获取的，而慢变数据对应的传感器如雷达或深度相机等获取相应的数据时的时间间隔较大，存在比较明显的不连续性。从而快变数据的获取周期较小，二慢变数据的获取周期较长。
步骤A3，根据属于所述快变数据类型和所述慢变数据类型的所述测量数据，分别建立第一类型的观测方程和第二类型的观测方程；
将数据分为快变数据与慢变数据后，对应的将建立的观测方程分为第一类型的观测方程和第二类型的观测方程。观测方程可以抽象为如下的表达式：
z k,j=h(y j,x k,v k,j)
其中z k,j为观测数据，y j为路标点，x k为扫地机器人的位置，v k,j为观测时存在的噪音，h()为一个抽象出的函数关系。
因为快变数据与慢变数据的状态变化间的时间间隔不同，所以第一类型的观测方程与第二类型的观测方程的k时刻与k+1时刻间的时间间隔不同。
在本实施例中，根据传感器的类型对于观测方程获取的数据进行分类，在分别建立相应类型的观测方程，将观测方程分为不同的类型能够在之后的融合过程中是融合结果更加准确。
进一步地，参照图2和图3，在本申请多数据源的SLAM方法上述实施例的基础上，提供多数据源的SLAM方法第三实施例，在第三实施例中，
步骤S30之前包括：
步骤S31，将所述第一类型的观测方程进行融合以得到融合结果；
第一类型的观测方程为利用快变数据构建的观测方程，则通过将初始时刻时第一类型的观测方程进行融合，如根据陀螺仪、加速度计获取的速度、加速度、角速度等观测数据，从而得到融合结果以获取扫地机器人的初始状态信息。
步骤S32，将所述融合结果作为第二类型的观测方程的点云配准算法的初值；
同时第二类型的观测方程为利用慢变数据构建的观测方程。因为慢变数据的数据获取的时间间隔较长，而快变数据的数据获取的时间间隔较短。当扫地机器人未出现故障，通过快变数据或慢变数据获取的扫地机器人的状态数据应该是相同的，因此可以将第一类型的观测方程的融合结果作为第二类型的观测方程的初值。
步骤S33，根据所述点云配准算法以及所述初值，得到配准结果；
点云配准算法是用于获取两个不同的帧图像间的位姿变换。因为需要将两帧不同的图像进行对比，因此通过第一类型的观测方程的融合结果作为初值即初始帧，然后不断通过点云配准算法从初始帧开始，得到相邻两帧图像的位姿变换结果即配准结果，从而能够获取各个时刻中扫地机器人的位姿信息。
步骤S34，对所述融合结果与所述配准结果进行贝叶斯推断以获取扫地机器人的初步状态信息；
当获取第二类型的观测方程的配准结果后，根据配准结果以及对应时刻的第一类型的观测方程的融合结果，贝叶斯推断可以利用已有的知识，在本申请中是已有的第一类型的融合结果以及第二类型的观测方程的配准结果，去推断出一个新的事件发生的概率，在本申请中，通过贝叶斯推断得到的是扫地机器人当前时刻的初步状态信息的最优估计以及相应的置信度。
在本实施例中，先分别对于第一类型的观测方程以及第二类型的观测方程求得相应的状态信息，再利用贝叶斯推断将两种类型的观测方程得到的结果融合起来得到扫地机器人的初始状态信息，从而是得到的初步状态信息更加准确。
进一步地，在本申请多数据源的SLAM方法上述实施例的基础上，提供多数据源的SLAM方法第四实施例，在第四实施例中，
步骤S30包括：
步骤B1，获取当前时刻由所述观测方程得到的初步状态信息；
对于每个时刻，都有对应的第一类型的观测方程与第二类型的观测方程，先通过第三实施例中的步骤S31至S34的方法获取当前时刻的初步状态信息。
步骤B2，根据所述初步状态信息，结合运动方程，利用所述贝叶斯递归估计算法进行融合，得到当前时刻扫地机器人的融合后的状态信息；
初步状态信息是由观测方程得到的，但没有结合运动方程，一般的，运动方程可以简化表示为：x k=f(x k-1,u k,w k)，x k和x k-1分别为k时刻与k-1时刻扫地机器人的位置，u k为传感器的读数，w k为噪声。运动方程还可以表示为与两个状态间的时间间隔相关的表达式。运动方程与运动约束相关，运动约束是为了保证最终得到的状态信息不会与客观的物理规律想违背，即可以保证状态信息的准确性。贝叶斯递归估计算法根据参数的先验分布和一系列观察值，求出参数的后验分布，然后求出参数的期望值，作为其最终值。在本申请中，运动方程为贝叶斯递归估计算法中的先验分布，而观测方程则可以提供不同时刻的一系列观测值，从而能够得到扫地机器人当前时刻的状态量以及相应的置信度。因此，通过贝叶斯递归估计算法，扫地机器人利用前一时刻的置信度计算得到当前时刻的预测置信度，再利用当前时刻观测方程的观测值以及预测置信度得到贝叶斯置信度并以此来更新扫地机器人当前的状态信息。
在本实施例中，利用运动方程与观测方程，通过贝叶斯递归估计算法，不断地更新扫地机器人当前的状态信息以及相应的置信度，贝叶斯递归估计算法使获取的扫地机器人的状态信息最接近真实的状态信息，从而提高整个定位的准确性。
进一步地，在本申请多数据源的SLAM方法上述实施例的基础上，提供多数据源的SLAM方法第五实施例，在第五实施例中，
步骤S40之前还包括：
步骤C1，对扫地机器人获取的深度图进行降采样处理；
因为扫地机器人中的处理器对于数据的处理能力有限，因此，为了适当减轻处理器对于数据的处理压力，对于深度相机获取的深度图进行降采样处理，将深度图的图像分辨率适当降低，从而使深度图中包含的数据量减少的同时尽量不影响之后利用深度图对于地图中体素的概率更新。
步骤S40包括：
步骤C2，根据所述融合后的状态信息确定扫地机器人在地图中的位置信息；
状态信息中包括扫地机器人的位姿、线速度、角速度、加速度等信息，因此，利用之前提高贝叶斯递归估计算法求得的扫地机器人的状态信息的最优估计，之后根据估计的状态信息获取扫地机器人当前的位姿即位置以及姿态信息。
步骤C3，利用深度图投影的点云信息，获取各物体与所述位置信息的距离信息；
深度图中的每个像素值代表的是物体到相机平面的距离，根据深度图中各像素值的距离信息，从而能够根据深度图投影出相应的点云信息，即周围环境中各物体的位置以及边缘信息。从而能够知道各物体与当前扫地机器人的位置信息之间的距离信息。
步骤C4，根据所述距离信息，基于静态贝叶斯滤波算法更新地图中体素的概率信息以构建地图；
根据距离信息判断占据在地图中的哪个体素中，之后再根据当前时刻获取的位置信息，以及前一时刻中各体素本来的概率信息，从而对于地图中各体素当前时刻的概率信息进行更新。具体贝叶斯滤波算法就是通过前一时刻的状态信息以及当前时刻的动作从而来判断当前时刻的状态，在本申请中，则是根据各体素前一时刻的概率信息以及当前时刻利用深度图获取的观测结果来求当前时刻各体素新的概率信息，从而扫地机器人能够不断更新构建地图。同时深度图那个准确检测到环境中包括低矮物体的位置信息，从而在构建地图的过程中能够获得更加丰富完善的地图信息，同时结合利用运动方程以及观测方程进行的扫地机器人的状态信息的确定，从而能够实现根据准确的扫地机器人的定位，进一步提高建立的地图的精度。
在本实施例中，利用深度图的点云信息确定环境中各物体的位置信息，并根据静态贝叶斯滤波算法不断更新地图中个体素的概率信息，从而构建地图，同时深度图能够包含低矮物体的信息，保证最终构建的地图的物体信息的准确性。
进一步地，参照图2和图4，在本申请多数据源的SLAM方法上述实施例的基础上，提供多数据源的SLAM方法第六实施例，在第六实施例中，
步骤S40之后，还包括：
步骤S50，获取红外图；
在实际的家居环境中，装修材料是复杂多样的，有一些材料是光学传感器难以测量的，如透光材料、反光材料、吸光材料，从而使深度相机对于包含这些材料的问题的测量存在问题如无法测量到或测量不准，导致最终更加的地图的定位精度以及与实际环境的一致性存在缺陷，因此这里还需要利用通过深度相机获取的红外图。
步骤S60，对所述红外图中的物体进行语义分割和边缘提取以得到物体信息；
红外图中那个显示出不同物体的光谱信息，对于透光材料、反光材料以及吸光材料都能获取相应的材料的位置与轮廓信息。对于红外图中的物体，进行语义返稿和边缘提取，即根据之前训练好的语义信息如桌子、凳子等，判断红外图中的各物体，之后对于相应的物体进行初步分割并获取物体初步的位置信息，之后再通过红外图中的光谱信息对于各物体的边缘进行边缘提取以获取准确的物体的位置信息和边缘信息。
步骤S70，根据所述物体信息进行贝叶斯推断以获取推断结果，并将所述推断结果更新到所述环境地图中；
物体信息包括物体的位置信息以及边缘信息，即物体处于房间中的哪里以及物体的边缘中的边界点对应的位置信息，从而通过物体的位置信息以及边缘信息能够构建出物体的具体信息。这里的贝叶斯推断是利用已有的地图中各体素已有的概率信息，结合通过语义分割和边缘提取的物体信息，去更新地图中各体素的概率信息，从而能够更新地图并且能够将光学传感器难以测量的物体的相关信息加入到构建的地图中，从而保证最终构建的地图信息与实际环境信息的一致性。
在本实施例中，通过红外图提供更多环境中的物体信息，并通过贝叶斯推断方法对于地图中各体素的概率信息进行更新以获取与实际环境更加一致的地图。
进一步地，在本申请多数据源的SLAM方法上述实施例的基础上，提供多数据源的SLAM方法第七实施例，在第七实施例中，
步骤S50之后，还包括：
步骤D1，获取所述红外图中各点的置信度并通过置信度直方图显示所述置信度；
置信度直方图法为统计红外图中各点的置信度信息，并通过直方图将各点的置信度进行统计，根据直方图显示置信度的统计结果，可以获取红外图中整体置信度的分布情况。
步骤D2，删除所述红外图中置信度不满足预设条件的噪点；
通过直方图可以知道其中在不同的置信度值区间中存在的红外图中的点的数量，其中置信度值低于预设值且所处的置信度区间包含的点数目小于预设数目的区间即为不满足预设条件的点，也称为红外图中的噪点，并将噪点删除以提高之后构建地图过程中的准确性。同时还可以对于红外图进行降采样来减少之后扫地机器人的处理器的数据的压力。
在本实施例中，通过置信度直方图分析红外图中的点并删除其中不满足预设条件的噪点，从而提高构建地图时的精确度。
进一步地，在本申请多数据源的SLAM方法上述实施例的基础上，提供多数据源的SLAM方法第八实施例，在第八实施例中，
步骤S60之后，还包括：
步骤E1，基于所述环境地图，对所述推断结果进行重投影以获取重投影误差；
根据贝叶斯的推断结果，将深度相机获取的红外图中的点云信息重投影为以雷达为基准面的点云信息，判断两个点云信息间的误差作为重投影的误差。
步骤E2，基于所述重投影误差对所述物体信息进行贝叶斯推断；
因为重投影过程中存在坐标系的转换，因此导致在不同坐标系中的点云的位置存在一定的区别，因此在利用深度相机的红外图中的点云信息进行对于地图进行更新时，因为之前构建地图时是基于雷达作为基准面，因此根据获取的重投影误差将物体信息转换成雷达对应的平面的物体信息在进行贝叶斯推断以更新地图。
在本实施例中，根据不同的坐标系进行重投影并获取重投影误差，利用重投影误差进行贝叶斯推断从而保证构建的环境地图中各体素信息的一致性。
进一步地，在本申请多数据源的SLAM方法上述实施例的基础上，提供多数据源的SLAM方法第九实施例，在第九实施例中，
多数据源的SLAM方法还包括：
步骤F1，比较所述观测方程与所述运动方程的结果误差；
步骤F2，若所述结果误差大于预设阈值，则增加相应的出错次数；
步骤F3，若所述出错次数大于预设次数，则输出提示信息；
运动方程是基于运动约束构建的方程，保证最终获取的扫地机器人的状态信息的合理性，观测方程则是由传感器的观测数据建立的，因此如果当过程方程与运动方程的结果误差超过预设阈值时，说明传感器获取的过程数据存在较大的误差，即传感器的测量可能存在问题，同时当多次出现错误时，说明传感器获取的数据不准确，此时输出提示信息提示用户对于相应的传感器进行检测或更换。
在本实施例中，通过比较运动方程与观测方程的误差来检测传感器的准确度，当多次传感器的测量数据出现较大误差，输出提示信息以提示用户。
此外，参照图5，本申请实施例还提出一种多数据源的SLAM装置，所述多数据源的SLAM装置包括：
第一建立模块，用于根据运动约束建立扫地机器人的运动方程；
第二建立模块，用于根据获取的扫地机器人的各传感器的测量数据，对各传感器建立观测方程；
第一融合模块，用于利用贝叶斯递归估计算法对所述运动方程和所述观测方程进行融合，获取扫地机器人的融合后的状态信息；
构建模块，用于根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建环境地图。
在一实施例中，所述第一建立模块包括：
第一获取单元，用于获取扫地机器人的各传感器的测量数据；
划分单元，用于根据所述测量数据对应的传感器类型，将所述测量数据划分为快变数据和慢变数据；
建立单元，用于根据属于所述快变数据类型和所述慢变数据类型的所述测量数据，分别建立第一类型的观测方程和第二类型的观测方程。
在一实施例中，所述多数据源的SLAM装置还包括：
第二融合模块，用于将所述第一类型的观测方程进行融合以得到融合结果；
初值模块，用于将所述融合结果作为第二类型的观测方程的点云配准算法的初值；
配准模块，用于根据所述点云配准算法以及所述初值，得到配准结果；
第一推断模块，用于对所述融合结果与所述配准结果进行贝叶斯推断以获取扫地机器人的初步状态信息。
在一实施例中，所述第一融合模块包括：
第二获取单元，用于获取当前时刻由所述观测方程得到的初步状态信息；
融合单元，用于根据所述初步状态信息，结合运动方程，利用所述贝叶斯递归估计算法进行融合，得到当前时刻扫地机器人的融合后的状态信息。
在一实施例中，所述多数据源的SLAM装置还包括：
降采样模块，用于对扫地机器人获取的深度图进行降采样处理。
在一实施例中，所述构建模块包括：
确定单元，用于根据所述融合后的状态信息确定扫地机器人在地图中的位置信息；
第三获取单元，用于利用深度图投影的点云信息，获取各物体与所述位置信息的距离信息；
更新单元，用于根据所述距离信息，基于静态贝叶斯滤波算法更新地图中体素的概率信息以构建地图。
在一实施例中，所述多数据源的SLAM装置还包括：
第一获取模块，用于获取红外图；
得到模块，用于对所述红外图中的物体进行语义分割和边缘提取以得到物体信息；
更新模块，用于根据所述物体信息进行贝叶斯推断以获取推断结果，并将所述推断结果更新到所述环境地图中。
在一实施例中，所述多数据源的SLAM装置还包括：
第二获取模块，用于获取所述红外图中各点的置信度并通过置信度直方图显示所述置信度；
删除模块，用于删除所述红外图中置信度不满足预设条件的噪点。
在一实施例中，所述多数据源的SLAM装置还包括：
重投影模块，用于对所述推断结果进行重投影以获取重投影误差；
第二推断模块，用于对于基于所述重投影误差对所述物体信息进行贝叶斯推断。
在一实施例中，所述多数据源的SLAM装置还包括：
比较模块，用于比较所述观测方程与所述运动方程的结果误差；
增加模块，用于若所述结果误差大于预设阈值，则增加相应的出错次数；
输出模块，用于若所述出错次数大于预设次数，则输出提示信息。
本申请设备和可读存储介质（即计算机可读存储介质）的具体实施方式的拓展内容与上述扫地机器人的控制方法各实施例基本相同，在此不做赘述。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质（如ROM/RAM、磁碟、光盘）中，包括若干指令用以使得一台终端（可以是手机，计算机，服务器，空调器，或者网络设备等）执行本申请各个实施例所述的方法。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，这些均属于本申请的保护之内。

Claims (12)

  1. 一种多数据源的SLAM方法，其中，所述多数据源的SLAM方法包括以下步骤：
    根据运动约束建立扫地机器人的运动方程；
    根据获取的扫地机器人的各传感器的测量数据，对各传感器建立观测方程；
    利用贝叶斯递归估计算法对所述运动方程和所述观测方程进行融合，获取扫地机器人的融合后的状态信息；
    根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建环境地图。
  2. 如权利要求1所述的多数据源的SLAM方法，其中，所述根据获取的扫地机器人的各传感器的测量数据，对各传感器建立观测方程的步骤包括：
    获取扫地机器人的各传感器的测量数据；
    根据所述测量数据对应的传感器类型，将所述测量数据划分为快变数据和慢变数据；
    根据属于所述快变数据类型和所述慢变数据类型的所述测量数据，分别建立第一类型的观测方程和第二类型的观测方程。
  3. 如权利要求2所述的多数据源的SLAM方法，其中，所述利用贝叶斯递归估计算法对所述运动方程和所述观测方程进行融合的步骤之前，还包括：
    将所述第一类型的观测方程进行融合以得到融合结果；
    将所述融合结果作为第二类型的观测方程的点云配准算法的初值；
    根据所述点云配准算法以及所述初值，得到配准结果；
    对所述融合结果与所述配准结果进行贝叶斯推断以获取扫地机器人的初步状态信息。
  4. 如权利要求3所述的多数据源的SLAM方法，其中，所述利用贝叶斯递归估计算法对所述运动方程和所述观测方程进行融合，获取扫地机器人的融合后的状态信息的步骤包括：
    获取当前时刻由所述观测方程得到的初步状态信息；
    根据所述初步状态信息，结合运动方程，利用所述贝叶斯递归估计算法进行融合，得到当前时刻扫地机器人的融合后的状态信息。
  5. 如权利要求4所述的多数据源的SLAM方法，其中，所述根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建地图的步骤之前还包括：
    对扫地机器人获取的深度图进行降采样处理。
  6. 如权利要求4所述的多数据源的SLAM方法，其中，所述根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建地图的步骤包括：
    根据所述融合后的状态信息确定扫地机器人在地图中的位置信息；
    利用深度图投影的点云信息，获取各物体与所述位置信息的距离信息；
    根据所述距离信息，基于静态贝叶斯滤波算法更新地图中体素的概率信息以构建地图。
  7. 如权利要求1所述的多数据源的SLAM方法，其中，所述根据融合后的状态信息，利用扫地机器人获取的深度图，通过静态贝叶斯滤波算法更新地图中体素的概率信息以构建地图的步骤之后，还包括：
    获取红外图；
    对所述红外图中的物体进行语义分割和边缘提取以得到物体信息；
    根据所述物体信息进行贝叶斯推断以获取推断结果，并将所述推断结果更新到所述环境地图中。
  8. 如权利要求7所述的多数据源的SLAM方法，其中，所述获取红外图的步骤之后，还包括：
    获取所述红外图中各点的置信度并通过置信度直方图显示所述置信度；
    删除所述红外图中置信度不满足预设条件的噪点。
  9. 如权利要求7所述的多数据源的SLAM方法，其中，所述根据所述物体信息进行贝叶斯推断以获取推断结果的步骤之后，还包括：
    对所述推断结果进行重投影以获取重投影误差；
    基于所述重投影误差对所述物体信息进行贝叶斯推断。
  10. 如权利要求1所述的多数据源的SLAM方法，其中，所述多数据源的SLAM方法还包括：
    比较所述观测方程与所述运动方程的结果误差；
    若所述结果误差大于预设阈值，则增加相应的出错次数；
    若所述出错次数大于预设次数，则输出提示信息。
  11. 一种多数据源的SLAM设备，其中，所述多数据源的SLAM设备包括处理器、存储器、以及存储在所述存储器上并可被所述处理器执行的计算机程序，所述计算机程序被所述处理器执行时，实现如权利要求1至10中任一项所述的多数据源的SLAM方法的步骤。
  12. 一种计算机可读存储介质，其中，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时，实现如权利要求1至10中任一项所述的多数据源的SLAM方法的步骤。
PCT/CN2020/115672 2020-08-21 2020-09-16 多数据源的slam方法、设备及计算机可读存储介质 WO2022036792A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010853891.0 2020-08-21
CN202010853891.0A CN114077245A (zh) 2020-08-21 2020-08-21 多数据源的slam方法、装置、扫地机器人及可读介质

Publications (1)

Publication Number Publication Date
WO2022036792A1 true WO2022036792A1 (zh) 2022-02-24

Family

ID=80282769

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/115672 WO2022036792A1 (zh) 2020-08-21 2020-09-16 多数据源的slam方法、设备及计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN114077245A (zh)
WO (1) WO2022036792A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117292208A (zh) * 2023-11-24 2023-12-26 广州中医药大学(广州中医药研究院) 一种数据处理过程中错误图案的分类方法与系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115601432B (zh) * 2022-11-08 2023-05-30 肇庆学院 一种基于fpga的机器人位置最优估计方法及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100094460A1 (en) * 2008-10-09 2010-04-15 Samsung Electronics Co., Ltd. Method and apparatus for simultaneous localization and mapping of robot
CN104793182A (zh) * 2015-04-21 2015-07-22 东南大学 非高斯噪声条件下基于粒子滤波的室内定位方法
CN104807465A (zh) * 2015-04-27 2015-07-29 安徽工程大学 机器人同步定位与地图创建方法及装置
CN108387236A (zh) * 2018-02-08 2018-08-10 北方工业大学 一种基于扩展卡尔曼滤波的偏振光slam方法
CN109062230A (zh) * 2018-08-06 2018-12-21 江苏科技大学 水下辅助采油机器人控制系统及动力定位方法
CN110260866A (zh) * 2019-07-19 2019-09-20 闪电(昆山)智能科技有限公司 一种基于视觉传感器的机器人定位与避障方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100094460A1 (en) * 2008-10-09 2010-04-15 Samsung Electronics Co., Ltd. Method and apparatus for simultaneous localization and mapping of robot
CN104793182A (zh) * 2015-04-21 2015-07-22 东南大学 非高斯噪声条件下基于粒子滤波的室内定位方法
CN104807465A (zh) * 2015-04-27 2015-07-29 安徽工程大学 机器人同步定位与地图创建方法及装置
CN108387236A (zh) * 2018-02-08 2018-08-10 北方工业大学 一种基于扩展卡尔曼滤波的偏振光slam方法
CN109062230A (zh) * 2018-08-06 2018-12-21 江苏科技大学 水下辅助采油机器人控制系统及动力定位方法
CN110260866A (zh) * 2019-07-19 2019-09-20 闪电(昆山)智能科技有限公司 一种基于视觉传感器的机器人定位与避障方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LI XIUZHI, JIA SONG-MIN: "3D Map Building for Mobile Robot Based on Multi-Sensor Fusion Aided SLAM", BEIJING LIGONG DAXUE XUEBAO = TRANSACTION OF BEIJING INSTITUTE OF TECHNOLOGY, BEIJING LIGONG DAXUE, BEIJING, CN, vol. 35, no. 3, 31 March 2015 (2015-03-31), CN , pages 262 - 267, XP055902205, ISSN: 1001-0645, DOI: 10.15918/j.tbit1001-0645.2015.03.009 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117292208A (zh) * 2023-11-24 2023-12-26 广州中医药大学(广州中医药研究院) 一种数据处理过程中错误图案的分类方法与系统
CN117292208B (zh) * 2023-11-24 2024-02-23 广州中医药大学(广州中医药研究院) 一种数据处理过程中错误图案的分类方法与系统

Also Published As

Publication number Publication date
CN114077245A (zh) 2022-02-22

Similar Documents

Publication Publication Date Title
US11551422B2 (en) Floorplan generation based on room scanning
WO2020223974A1 (zh) 更新地图的方法及移动机器人
JP6978330B2 (ja) オブジェクト形状および設計からのずれの監視
CN107748569B (zh) 用于无人机的运动控制方法、装置及无人机系统
CN111402339B (zh) 一种实时定位方法、装置、系统及存储介质
CN111006676B (zh) 地图构建方法、装置及系统
US20170116353A1 (en) Methods, apparatuses and computer program products for automatic, non-parametric, non-iterative three dimensional geographic modeling
WO2022036792A1 (zh) 多数据源的slam方法、设备及计算机可读存储介质
CN110544294B (zh) 一种基于全景视频的稠密三维重构方法
WO2020063878A1 (zh) 一种处理数据的方法和装置
CN115578433B (zh) 图像处理方法、装置、电子设备及存储介质
JP2022050311A (ja) 車両の車線変更を検出するための方法、装置、電子機器、記憶媒体、路側機、クラウド制御プラットフォーム、及びコンピュータプログラム
KR20220025028A (ko) 시각적 비콘 기반의 비콘 맵 구축 방법, 장치
CN112083403A (zh) 用于虚拟场景的定位追踪误差校正方法及系统
CN109997123B (zh) 用于改进空间-时间数据管理的方法、系统和装置
JP7351892B2 (ja) 障害物検出方法、電子機器、路側機器、及びクラウド制御プラットフォーム
US11640692B1 (en) Excluding objects during 3D model generation
CN113160401B (zh) 一种面向物体的视觉slam轻量化语义地图创建方法
US11763478B1 (en) Scan-based measurements
CN113140032A (zh) 基于房间扫描的平面图生成
CN117232499A (zh) 多传感器融合的点云地图构建方法、装置、设备及介质
CN115578432B (zh) 图像处理方法、装置、电子设备及存储介质
CN114299192B (zh) 定位建图的方法、装置、设备和介质
CN111899277A (zh) 运动目标检测方法及装置、存储介质、电子装置
CN112904365B (zh) 地图的更新方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20949996

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20949996

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20949996

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 26-09-23)