WO2020168668A1 - 一种车辆的slam建图方法及系统 - Google Patents

一种车辆的slam建图方法及系统 Download PDF

Info

Publication number
WO2020168668A1
WO2020168668A1 PCT/CN2019/093537 CN2019093537W WO2020168668A1 WO 2020168668 A1 WO2020168668 A1 WO 2020168668A1 CN 2019093537 W CN2019093537 W CN 2019093537W WO 2020168668 A1 WO2020168668 A1 WO 2020168668A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
camera module
pose
map
image
Prior art date
Application number
PCT/CN2019/093537
Other languages
English (en)
French (fr)
Inventor
周建
李良
王磊刚
Original Assignee
广州小鹏汽车科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州小鹏汽车科技有限公司 filed Critical 广州小鹏汽车科技有限公司
Priority to EP19915759.5A priority Critical patent/EP3886053B1/en
Publication of WO2020168668A1 publication Critical patent/WO2020168668A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/579—Depth or shape recovery from multiple images from motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Definitions

  • the present invention relates to the technical field of automatic driving, in particular to a vehicle SLAM mapping method and system.
  • simultaneous monocular vision positioning and map creation means that autonomous vehicles use a single visual sensor (such as a camera) to create a map consistent with the real environment, and at the same time determine that they are on the map In the location.
  • the traditional monocular vision SLAM can only be initialized by a dynamic method, that is, it needs to use the images captured during the movement of the camera to be successfully initialized. Therefore, when drawing, the vehicle needs to be driven out for a certain distance, and after the initialization of the monocular vision SLAM, the drawing can be started. If the initialization is successful, the vehicle needs to drive a long distance to start building the map, which may result in a lower success rate of the drawing, and it may also easily lead to the lack of some environmental information in the constructed map.
  • the embodiment of the invention discloses a vehicle SLAM mapping method, which can complete initialization when the vehicle is stationary, so as to improve the success rate of mapping.
  • the first aspect of the embodiments of the present invention discloses a vehicle SLAM mapping method, the method includes:
  • the three-dimensional space positions of the mutually matched feature points are determined to obtain the
  • the map points corresponding to the matching feature points are constructed to construct the initial SLAM map to complete the initialization;
  • the acquisition of two initial frames respectively obtained by any two camera modules of the vehicle when the vehicle is stationary includes:
  • the first image and the second image are processed according to the image effect obtained by shooting with the same focal length, and the processed first image and the second image are used as two initial frames.
  • the monocular SLAM mapping is performed based on the initial SLAM map and combined with the image captured by the target camera module ,include:
  • the The current SLAM map is the initial SLAM map
  • the global SLAM map is constructed based on the values of the poses of each camera module included in the pose sequence of the target camera module when the projection error is the smallest and the values of the three-dimensional space positions of the map points in the current SLAM map.
  • the method further includes:
  • the coordinate origin of the global SLAM map coordinate system is the optical center position when the target camera module captures the initial frame
  • the vehicle is determined to be on the SLAM map according to the relocation pose of the target camera module in the SLAM map. After relocating the pose in, the method further includes:
  • the positioning pose of the current pose in the global SLAM map is determined as the positioning result of the vehicle at the current moment.
  • the second aspect of the embodiments of the present invention discloses a vehicle SLAM mapping system, including:
  • the initialization unit is used to obtain two initial frames respectively captured by any two camera modules of the vehicle when the vehicle is stationary; and, according to the pre-calibrated internal parameters and external parameters of the two camera modules, and mutual The disparity of the matched feature points in the two initial frames is determined to determine the three-dimensional space positions of the mutually matched feature points to obtain the map points corresponding to the mutually matched feature points, and the initial SLAM map is constructed to Initialization is completed; wherein the viewing ranges of the two camera modules at least partially overlap;
  • the mapping unit is configured to obtain an image captured by any one of the two camera modules of the target camera module after the initialization is successful; and, based on the initial SLAM map, combined with the target camera module
  • the images captured by the group are constructed for monocular SLAM.
  • the initialization unit is used to obtain two initial frames respectively captured by any two camera modules of the vehicle when the vehicle is stationary Specifically:
  • the initialization unit is used to obtain the first image and the second image obtained by shooting any two camera modules of the vehicle when the vehicle is stationary; An image and the second image are processed, and the processed first image and the second image are used as initial frames; wherein the focal lengths of the two camera modules are different.
  • the mapping unit includes:
  • the matching subunit is used to match the feature points in the image captured by the target camera module and the map points in the current SLAM map; wherein, when the image is the first image captured by the target camera module Frame images, the current SLAM map is the initial SLAM map;
  • the positioning subunit is used to determine the camera module pose corresponding to each frame of images captured by the target camera module according to the feature points and map points that are matched with each other, so as to obtain the pose sequence of the target camera module;
  • the mapping subunit is used to track the position of the feature points that are not matched to the map point in the image captured by the target camera module, and combine the camera model corresponding to the image where the feature points that are not matched to the map point are located Group poses, determine the three-dimensional space positions of the feature points that are not matched to the map points, so as to construct new map points to be added to the current SLAM map;
  • An obtaining subunit configured to obtain the first pose sequence of the vehicle corresponding to the pose sequence of the target camera module through the positioning module of the vehicle;
  • the optimization subunit is used to iteratively adjust the value of the pose of each camera module included in the pose sequence of the target camera module and the value of the three-dimensional space position of the map point in the current SLAM map until the The projection error between the pose sequence of the target camera module and the first pose sequence is the smallest;
  • the mapping subunit is further configured to use the values of the poses of each camera module included in the pose sequence of the target camera module when the projection error is the smallest and the three-dimensional values of the map points in the current SLAM map
  • the value of the spatial location constructs a global SLAM map.
  • system further includes:
  • the first acquiring unit is configured to acquire multiple frames of images captured by the target camera module and the global SLAM map during relocation;
  • the matching unit is configured to match the feature points in the frame image and the map points in the global SLAM map for each frame of the multi-frame image; and determine the frame according to the feature points and map points that are matched with each other
  • the camera module pose of the target camera module in the global SLAM map corresponding to the image to obtain the second camera module pose sequence of the target camera module;
  • the second acquiring unit is configured to acquire the second vehicle pose sequence of the vehicle corresponding to the second camera module pose sequence through the positioning module of the vehicle;
  • the relocation unit is configured to determine the relocation pose of the target camera module in the global SLAM map based on multiple frames of target images and the second camera module pose sequence, so that the target camera module is used
  • the relocation pose of the group converts the second vehicle pose sequence to the SLAM map coordinate system of the global SLAM map, the converted third vehicle pose sequence and the second camera module pose The sequence has the smallest error; and, according to the relocation pose of the target camera module in the global SLAM map, determining the relocation pose of the vehicle in the global SLAM map;
  • the coordinate origin of the global SLAM map coordinate system is the position of the optical center when the target camera module captures the initial frame.
  • system further includes:
  • the third acquiring unit is configured to acquire after the relocation unit determines the relocation pose of the vehicle in the SLAM map according to the relocation pose of the target camera module in the SLAM map The current pose of the vehicle measured by the positioning module of the vehicle at the current moment;
  • the positioning unit is configured to determine the positioning pose of the current pose in the global SLAM map as the current moment according to the current pose and the relocation pose of the vehicle in the global SLAM map The positioning result of the vehicle.
  • a third aspect of the embodiments of the present invention discloses a vehicle, and the vehicle includes any one of the vehicle SLAM mapping systems disclosed in the second aspect of the embodiments of the present invention.
  • the fourth aspect of the embodiments of the present invention discloses a vehicle SLAM mapping system, including:
  • a memory storing executable program codes
  • a processor coupled with the memory
  • the image captured by the camera module and the vehicle pose detected by the vehicle positioning module are transmitted to the processor, and the processor calls the executable program code stored in the memory to execute the implementation of the present invention Examples of any of the methods disclosed in the first aspect.
  • a fifth aspect of the present invention discloses a computer-readable storage medium that stores a computer program, wherein the computer program causes a computer to execute any method disclosed in the first aspect of the embodiments of the present invention.
  • the sixth aspect of the embodiments of the present invention discloses a computer program product.
  • the computer program product runs on a computer, the computer is caused to execute any one of the methods disclosed in the first aspect of the embodiments of the present invention.
  • the initial SLAM map is constructed by using the feature points matched with each other in the initial frames, so that the binocular camera module can be used Initialize; after the initialization is successful, use any one of the two camera modules to capture images for monocular SLAM mapping, which can integrate the advantages of binocular and monocular, and complete when the vehicle is stationary Initialization does not require the vehicle to move a certain distance, which can increase the success rate of mapping and reduce the lack of information in the map.
  • FIG. 1 is a schematic flowchart of a vehicle SLAM mapping method disclosed in an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of another vehicle SLAM mapping method disclosed in an embodiment of the present invention.
  • FIG. 3 is a schematic flowchart of another SLAM mapping method for vehicles disclosed in an embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of a vehicle SLAM mapping system disclosed in an embodiment of the present invention.
  • Figure 5 is a schematic structural diagram of another vehicle SLAM mapping system disclosed in an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of another vehicle SLAM mapping system disclosed in an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of another SLAM mapping system for a vehicle disclosed in an embodiment of the present invention.
  • the embodiment of the present invention discloses a vehicle SLAM mapping method and system, which can complete initialization when the vehicle is stationary, so as to improve the success rate of mapping. Detailed descriptions are given below.
  • FIG. 1 is a schematic flowchart of a vehicle SLAM mapping method disclosed in an embodiment of the present invention.
  • the SLAM mapping method for vehicles described in FIG. 1 is applicable to on-board electronic devices such as on-board computers and electronic control units (ECUs), which are not limited in the embodiment of the present invention.
  • ECUs electronice control units
  • the SLAM mapping method of the vehicle may include the following steps:
  • the vehicle-mounted electronic device acquires two initial frames respectively captured by any two camera modules of the vehicle when the vehicle is stationary.
  • the viewing ranges of the two camera modules at least partially overlap.
  • the above two camera modules may be any two camera modules in a multi-camera camera system.
  • the multi-view camera system can include three or more camera modules, and the distance between each camera module can be between 10cm-20cm, all facing the driving direction of the vehicle.
  • the multi-camera can be selected
  • the two camera modules with the farthest relative distance in the camera system (such as the camera modules located at both ends) take images, and the two images captured by the two camera modules with the farthest relative distance are taken as two initial frame.
  • the prerequisite for successful initialization is that there are matching feature points in the initial frame.
  • the matching feature points are the same object in the three-dimensional space after being photographed by two camera modules and projected to two The projection point obtained in the initial frame. Therefore, the viewing ranges of the two camera modules should at least partially overlap to ensure that there can be matching feature points in the initial frame.
  • the two camera modules can also be controlled to capture images at the same time to further ensure that there can be feature points that match each other in the initial frame.
  • the initial frame should meet certain requirements. For example, it can be set that the number of feature points in the initial frame should be greater than a preset threshold (such as 100).
  • the captured image is set as the initial Frame; otherwise, the two camera modules re-shoot the image.
  • the on-vehicle electronic equipment determines the three-dimensional space positions of the matching feature points according to the internal and external parameters pre-calibrated by the above two camera modules, and the disparity of the matching feature points in the two initial frames, to obtain
  • the map points corresponding to the matching feature points are constructed to construct the initial SLAM map to complete the initialization.
  • the respective internal parameters of the two camera modules such as focal length, distortion coefficient, etc.
  • the relative external parameters of the two camera modules such as baseline, right camera module, etc.
  • the translation and rotation relative to the left camera module are known, and the disparity of the matched feature points in the two initial frames can be combined to determine the three-dimensional spatial position of the matched feature points.
  • the internal parameters of the camera module can be used to characterize the mapping relationship between the three-dimensional space points and the image pixel points, so based on the internal parameters of the camera module and the location of the feature points in the image, the features can be restored The relative position of the point relative to the camera module, but lacks depth information.
  • the in-vehicle electronic device acquires an image captured by any one of the two camera modules.
  • the target camera module may be the above-mentioned camera module selected as the origin of the coordinate system.
  • the vehicle can start to drive, and the target camera module can continue to capture multiple frames of images while the vehicle is continuously moving.
  • the vehicle-mounted electronic equipment Based on the initial SLAM map, the vehicle-mounted electronic equipment combines the images captured by the target camera module to perform monocular vision SLAM mapping.
  • monocular vision SLAM that is, only one target camera module is used to capture images. It is understandable that the creation of monocular SLAM can specifically include the following steps:
  • the target camera module captures the first frame of image
  • the feature points in the first frame of image and the map points in the initial SLAM map are matched (ie 2D-3D matching);
  • the position of the camera (ie camera module) corresponding to the first frame of image can be determined according to the image position of the feature point in the first frame of image and the three-dimensional space position of the corresponding matched map point. posture;
  • two camera modules can be used to form a binocular camera system.
  • the binocular camera system works when the vehicle is stationary.
  • the initial frame with parallax can already be obtained, so there is no need for the vehicle to travel a certain distance for initialization.
  • the initialization can be completed when the vehicle is stationary, which can reduce the data loss during mapping; and after the initialization is completed, only one camera model is used.
  • only using a single camera module can reduce the amount of image data that needs to be processed.
  • the method described in Figure 1 can integrate the advantages of monocular and binocular, and can reduce the amount of data processing while improving the success rate of mapping.
  • the above steps 101 to 102 may be the steps executed when the mapping is initially started, or may be executed once the mapping failure is detected during the mapping process.
  • any two camera modules of the vehicle can be activated.
  • the target camera module can be included, and then images are taken through these two camera modules to be obtained when the vehicle is stationary To the initial frame.
  • the mapping failure may include a small number of feature points in the image captured by the target camera module, or a small number of feature points matching the map points in the image, which is not limited in the embodiment of the present invention. Therefore, performing the above steps 101 to 102 can also quickly re-initialize when the mapping fails, so that the mapping can be performed again. Compared with the monocular camera system that requires vehicle movement for initialization, it can reduce the time between two mappings. Data for is missing.
  • FIG. 2 is a schematic flowchart of another vehicle SLAM mapping method disclosed in an embodiment of the present invention.
  • the SLAM mapping method of the vehicle may include the following steps:
  • the vehicle-mounted electronic device acquires two initial frames respectively captured by any two camera modules of the vehicle when the vehicle is stationary.
  • the above-mentioned two camera modules may be any two camera modules in a multi-lens camera system, and the focal lengths and field angles of the two camera modules may be the same or different.
  • a camera module with a fixed focal length is generally selected.
  • different camera modules are generally set to different focal lengths to reduce the problem that a single camera module cannot switch the focal length.
  • the field of view angles of different camera modules may also be different. For example, three camera modules use field angles of 120°, 60°, and 30° respectively, so that different camera modules cover scenes with different angle ranges.
  • the accuracy of matching the feature points of the initial frame in the following step 202 is relatively high.
  • the focal length and the angle of view are different, the deformation degree of the object in the image captured by the two camera modules is different, which will reduce the accuracy of feature point matching. Therefore, when restoring the three-dimensional space position of the feature point based on binocular stereo vision, using two camera modules with the same focal length to take the initial frame, the accuracy of the restored three-dimensional space position may be higher than that of using two camera modules with different focal lengths to take the initial frame. frame.
  • the following steps may be performed:
  • the first image and the second image are processed according to the image effect obtained by shooting with the same focal length, and the processed first image and the second image are used as two initial frames.
  • the same focal length f AC mentioned above can satisfy the following conditions: f AC ⁇ [f A , f C ]. It is understandable that when calibrating the parameters of the camera module, the internal parameters of one camera module can be independently calibrated to obtain the aforementioned f A , and then the internal parameters of the other camera module can be independently calibrated to obtain the aforementioned f C , instead of calibrating the above two camera modules at the same time.
  • the images taken by two camera modules with different focal lengths can be adjusted to the image effect obtained by shooting at the same focal length through soft focusing, thereby reducing the accuracy of the three-dimensional space position calculation of camera modules with different focal lengths. , Thereby reducing the impact on the accuracy of mapping.
  • steps 202 to 203 are the same as steps 102 to 103, and the following content will not be repeated.
  • the vehicle-mounted electronic equipment matches the feature points in the image captured by the target camera module and the map points in the current SLAM map to obtain the first camera module pose sequence of the target camera module, and construct a new The map point is added to the current SLAM map.
  • the current SLAM map is the SLAM map that has been constructed at the current moment.
  • the current SLAM map is the initial SLAM map .
  • the camera module pose corresponding to each frame of images captured by the target camera module can be determined according to the feature points and map points that are matched with each other, so as to obtain the first camera module pose sequence of the target camera module; wherein, Each pose in the pose sequence of the first camera module can have a one-to-one correspondence with the image captured by the target camera module;
  • the location of the feature points that are not matched to the map points in the image captured by the target camera module can be tracked, and combined with the camera model corresponding to the image where the feature points that are not matched to the map point are located.
  • Group poses determine the three-dimensional space position of the feature points that are not matched to the map point, to construct a new map point to be added to the current SLAM map.
  • the vehicle-mounted electronic device obtains the first vehicle pose sequence of the vehicle corresponding to the first camera module pose sequence through the positioning module of the vehicle.
  • the vehicle may be equipped with sensors such as an inertial measurement unit (IMU) and wheel pulse counter. These sensors can be used as a positioning module of the vehicle (such as a body odometer) to calculate the driving distance of the vehicle.
  • the ground can also position the vehicle.
  • the first vehicle pose sequence may include multiple vehicle poses measured by the positioning module of the vehicle, and each vehicle pose corresponds to a certain camera module pose in the first camera module pose sequence.
  • the target camera module captures an image at a certain moment
  • the feature points in the image are matched with the map points to determine the pose of the camera module corresponding to the image; at the same time, the vehicle at that moment
  • the positioning module measures the pose of the vehicle, then the pose of the vehicle corresponds to the pose of the aforementioned camera module.
  • the vehicle-mounted electronic equipment iteratively adjusts the pose values of each camera module included in the pose sequence of the target camera module and the values of the three-dimensional space positions of the map points in the current SLAM map, until the pose of the first camera module The projection error between the sequence and the first vehicle pose sequence is the smallest.
  • the value of the pose of the camera module and the value of the three-dimensional space position of the map point should meet the constraint conditions required by the joint optimization (Bundle Adjustment).
  • the vehicle pose measured by the positioning module of the vehicle is relatively accurate. Therefore, based on the above-mentioned first vehicle pose sequence, assuming the value of the camera module pose and the three-dimensional space of the map point The value of the position is relatively accurate, so the gap between the pose sequence of the first camera module and the pose sequence of the first vehicle is small.
  • the camera module pose in the first camera module pose sequence may be a pose in the camera coordinate system
  • the vehicle pose in the first vehicle pose sequence may be a pose in the vehicle coordinate system.
  • Pose the camera coordinate system takes the optical center of the camera module as the coordinate origin
  • the vehicle coordinate system takes a certain part of the vehicle (such as the center point of the rear axle) as the coordinate origin
  • the conversion relationship between the camera coordinate system and the vehicle coordinate system can be Get pre-calibrated.
  • the gap between the pose sequence of the first camera module and the pose sequence of the first vehicle is relatively small. It can be that the gap between the pose sequence of the first camera module and the pose sequence of the first vehicle is the camera coordinate system and the vehicle coordinates.
  • the conversion relationship between the systems that is, if the pose sequence of the first camera module is converted to the vehicle coordinate system based on the conversion relationship between the camera coordinate system and the vehicle coordinate system, the converted pose sequence is the same as the first The vehicle pose sequence coincides.
  • the vehicle-mounted electronic device constructs a global SLAM map based on the values of the poses of each camera module included in the pose sequence of the target camera module when the projection error is the smallest and the values of the three-dimensional spatial positions of the map points in the current SLAM map.
  • the global SLAM map may be the SLAM map finally obtained when the vehicle-mounted electronic device stops executing the mapping program.
  • the implementation of the method shown in Figure 2 can make the camera system cover the observation scene as much as possible in a limited physical space through camera modules with different focal lengths, and at the same time, reduce the number of camera modes with different focal lengths through soft focusing.
  • it can be combined with the positioning module of the vehicle to further improve the accuracy of the constructed SLAM map through joint optimization.
  • FIG. 3 is a schematic flowchart of yet another vehicle SLAM mapping method disclosed in an embodiment of the present invention.
  • the SLAM mapping method of the vehicle may include the following steps:
  • Step 301 to step 304 are the same as step 101 to step 104, wherein the specific implementation of step 304 can be as shown in step 204 to step 206, and the following content will not be repeated.
  • the vehicle-mounted electronic device When the vehicle-mounted electronic device is repositioning, acquire multiple frames of target images captured by the target camera module and the aforementioned global SLAM map.
  • the global SLAM map is a digital description of the real environment.
  • the vehicle-mounted electronic equipment can reuse the global SLAM map. Similar to the way humans use maps for positioning, on-board electronic equipment can locate the position of the vehicle on the global SLAM map, thereby knowing the position of the vehicle in the real environment.
  • relocation can be triggered to determine the vehicle’s position in the global SLAM map again. position. Specifically, the following steps 306 to 309 can be executed for relocation.
  • the vehicle-mounted electronic device For each frame of the target image in the multi-frame target image, the vehicle-mounted electronic device matches the feature points in the frame of the target image and the map points in the global SLAM map, and determines the frame of the target image according to the matching feature points and map points. The corresponding camera module pose of the target camera module in the global SLAM map to obtain the second camera module pose sequence of the target camera module.
  • each frame of target image corresponds to a camera module pose in the second camera module pose sequence
  • the camera module pose in the second camera module pose sequence is the target The pose of the camera module in the camera coordinate system.
  • the vehicle-mounted electronic device obtains the second vehicle pose sequence of the vehicle corresponding to the second camera module pose sequence through the positioning module of the vehicle.
  • each vehicle pose included in the second vehicle pose sequence corresponds to the camera module pose included in the second camera module pose sequence
  • the vehicle-mounted electronic device can capture the target on the target camera module.
  • the vehicle pose is acquired through the positioning module to form a second vehicle pose sequence corresponding to the second camera module pose sequence, and the vehicle pose in the second vehicle pose sequence can be the vehicle coordinate system The next posture.
  • the vehicle-mounted electronic device determines the relocation pose of the target camera module in the global SLAM map based on the multi-frame target image and the second camera module pose sequence, so that the relocation pose of the target camera module is used to reposition the second camera module.
  • the vehicle pose sequence is converted to the SLAM map coordinate system of the global SLAM map, the error between the converted third vehicle pose sequence and the second camera module pose sequence is the smallest.
  • the relocation pose of the target camera module in the global SLAM map that is, the relative conversion relationship between the pose of the current target camera module and the origin of the global SLAM map coordinate system (translation amount and rotation amount) , That is, the conversion relationship between the camera coordinate system of the target camera module and the global SLAM map coordinate system; wherein, the origin of the global SLAM map coordinate system can be the optical center position when the target camera module shoots the aforementioned initial frame.
  • the second vehicle pose sequence in the vehicle coordinate system is converted to the global SLAM map, and the third vehicle pose sequence obtained by the conversion can be regarded as the moving track of the vehicle in the map.
  • the second camera module pose sequence can be considered as the movement track of the target camera module in the map. If the target camera module's repositioning pose is accurate, the movement track of the target camera module in the map and the vehicle in the map The gap between the movement trajectories is relatively small. Specifically, the gap between the third vehicle pose sequence and the second camera module pose sequence is the conversion relationship between the camera coordinate system and the vehicle coordinate system.
  • the second camera module can be continuously adjusted iteratively by jointly optimizing the constraint relationship between the defined target image, the pose of the camera module corresponding to the target image, and the three-dimensional space position of the map point.
  • the value of the pose of each camera module in the pose sequence take the value of the last camera module pose in the second camera module pose sequence as the relocation pose of the target camera module, or take the second camera
  • the average of the poses of each camera module in the module pose sequence is used as the relocation pose of the target camera module, and the second vehicle pose sequence is converted to the global SLAM map until the third vehicle pose sequence and the second
  • the error between camera module pose sequences is the smallest.
  • the value of the relocation pose of the target camera module when the error is the smallest is the final relocation pose.
  • the vehicle-mounted electronic device determines the relocation pose of the vehicle in the global SLAM map according to the relocation pose of the target camera module in the global SLAM map.
  • the vehicle coordinate system by using the conversion relationship between the vehicle coordinate system and the camera coordinate system of the target camera module, it can be determined that the vehicle is in the global SLAM map through the repositioning pose of the target camera module The repositioning pose.
  • the global SLAM map can be reused to locate the vehicle, and at the same time, the accuracy of relocation can be improved by means of multi-frame joint relocation.
  • the vehicle-mounted electronic device acquires the current pose of the vehicle measured by the positioning module of the vehicle at the current moment.
  • the on-board electronic device determines the current pose in the global SLAM map according to the current pose and the relocation pose of the vehicle in the global SLAM map as the positioning result of the vehicle at the current moment.
  • the current pose measured by the positioning module of the vehicle is the pose in the vehicle coordinate system
  • the relocation pose of the vehicle in the global SLAM map can be understood as the difference between the vehicle coordinate system and the global SLAM map coordinate system. Therefore, based on the repositioning pose of the vehicle in the global SLAM map, the current pose of the vehicle can be converted to the global SLAM map.
  • the visual SLAM system will not be able to perform positioning. Therefore, in this embodiment of the present invention, after successful relocation (that is, determining the relocation pose of the vehicle in the global SLAM map) of the on-board electronic device, the current pose of the vehicle measured by the positioning module of the vehicle shall prevail, and the The current pose is converted to the SLAM map as the final positioning result, which can reduce the dependence of vehicle positioning on visual information, increase the probability of successful vehicle positioning, greatly get rid of the unstable factors of the visual positioning process, and improve the system’s performance Stability under visual occlusion or loss of visual features.
  • the binocular camera system can be initialized, and the monocular camera module can be used to build the map after the initialization is successful, thereby fusing the advantages of monocular and binocular, and improving the success rate of mapping It can also reduce the amount of data processing.
  • the accuracy of the constructed SLAM map is further improved by combining the positioning module of the vehicle in a joint optimization manner.
  • multi-frame joint relocation is used to improve the accuracy of relocation.
  • the current pose measured by the positioning module of the vehicle is converted to the SLAM map as the positioning result to reduce vehicle positioning. The degree of dependence on visual information increases the probability of successful vehicle positioning.
  • FIG. 4 is a schematic structural diagram of a vehicle SLAM mapping system disclosed in an embodiment of the present invention. As shown in Figure 4, the system includes:
  • the initialization unit 401 is used to obtain two initial frames respectively captured by any two camera modules of the vehicle when the vehicle is stationary; and, according to the pre-calibrated internal parameters and external parameters of the two camera modules, and features that match each other
  • the disparity of the points in the two initial frames determines the three-dimensional space positions of the matching feature points to obtain the map points corresponding to the matching feature points, and construct the initial SLAM map to complete the initialization; among them, two camera modes There is at least a partial overlap in the viewing range of the group;
  • the initialization unit 401 can obtain the images taken by any two camera modules as the initial frame, and can also obtain the two camera modules with the farthest distance in the multi-camera system where the camera modules are located (such as The images captured by the camera modules located at both ends are used as the initial frame.
  • the camera module can be controlled to re-shoot the image until the number of feature points is greater than The image with the aforementioned preset threshold is used as the initial frame.
  • the mapping unit 402 is used to obtain an image captured by any one of the two camera modules by the target camera module after the initialization is successful; and, based on the initial SLAM map, combined with the image captured by the target camera module Perform monocular vision SLAM mapping.
  • the map creation unit 402 can specifically create a map through the following operations:
  • the mapping unit 402 is configured to match the feature points in the first frame of image captured by the target camera module and the map points in the initial SLAM map;
  • the mapping unit 402 can determine the camera corresponding to the first frame of image (ie, camera) according to the image position of the feature point in the first frame of image and the three-dimensional space position of the corresponding matched map point. Module) pose;
  • the mapping unit 402 For each frame of image captured by the target camera module after the first frame of image, the mapping unit 402 repeats the above operation, so that the camera pose corresponding to each frame of image can be determined; at the same time, for the image that does not match the map
  • the feature points of the points are tracked in the image captured by the target camera module. According to the image position in the image where these feature points are located, and the camera pose corresponding to the image where these feature points are located, the values of these feature points can be calculated Three-dimensional space position, and thus get a new map point to be added to the SLAM map.
  • the initialization unit 401 can be triggered to perform the above operations, so that it can quickly re-initialize when the mapping fails, so that the mapping can be performed again. For the monocular camera system that requires vehicle movement for initialization, data loss between two maps can be reduced.
  • the implementation of the system shown in Figure 4 can use two camera modules to form a binocular camera system during initialization, which eliminates the need for the vehicle to travel a certain distance for initialization.
  • the initialization can be completed when the vehicle is stationary; after the initialization is successful , Only use the images captured by one camera module to build maps to reduce the amount of image data that needs to be processed; therefore, the advantages of monocular and binocular can be combined, which can improve the success rate of mapping while reducing the amount of data processing .
  • FIG. 5 is a schematic structural diagram of another vehicle SLAM mapping system disclosed in an embodiment of the present invention. Among them, the SLAM mapping system of the vehicle shown in FIG. 5 is optimized by the SLAM mapping system of the vehicle shown in FIG. 4.
  • the focal lengths of the two camera modules are different; accordingly, the above-mentioned initialization unit 401 is used to obtain two initial frames respectively captured by any two camera modules of the vehicle when the vehicle is stationary.
  • the specific method can be:
  • the initialization unit 401 is used to obtain the first image and the second image obtained by shooting any two camera modules of the vehicle when the vehicle is stationary; and, according to the image effect obtained by shooting with the same focal length, the first image and the second image are The image is processed, and the processed first image and second image are used as two initial frames.
  • the initialization unit 401 can adjust the images taken by two camera modules with different focal lengths to the image effect obtained by shooting at the same focal length through soft focusing, thereby reducing the effect of camera modules with different focal lengths on the three-dimensional The influence of spatial position calculation accuracy, thereby reducing the influence on the accuracy of mapping.
  • the foregoing mapping unit 402 may include:
  • the matching subunit 4021 is used to match the feature points in the image captured by the target camera module and the map points in the current SLAM map; wherein, the current SLAM map is the SLAM map that has been constructed at the current moment, and when used to perform When the image matched by the feature point is the first frame image captured by the target camera module, the current SLAM map is the initial SLAM map;
  • the positioning subunit 4022 is used to determine the camera module pose corresponding to each frame of images captured by the target camera module according to the feature points and map points that are matched with each other, so as to obtain the pose sequence of the target camera module;
  • the mapping subunit 4023 is used to track the position of the feature points that are not matched to the map point in the image captured by the target camera module, and combine the pose of the camera module corresponding to the image where the feature points that are not matched to the map point are located , Determine the three-dimensional space position of the feature points that are not matched to the map point, to construct a new map point to be added to the current SLAM map;
  • the obtaining subunit 4024 is used to obtain the first pose sequence of the vehicle corresponding to the pose sequence of the target camera module through the positioning module of the vehicle; wherein, the positioning module may be an inertia measurement unit (IMU), Sensors such as wheel pulse counters can be used as a vehicle's positioning module (such as a body odometer) to calculate the distance traveled by the vehicle, and furthermore can position the vehicle.
  • IMU inertia measurement unit
  • Sensors such as wheel pulse counters can be used as a vehicle's positioning module (such as a body odometer) to calculate the distance traveled by the vehicle, and furthermore can position the vehicle.
  • the optimization subunit 4025 is used to iteratively adjust the pose values of each camera module included in the pose sequence of the target camera module and the value of the three-dimensional space position of the map point in the current SLAM map until the target camera module The projection error between the pose sequence and the first pose sequence is the smallest;
  • mapping subunit 4023 is used to calculate the values of the poses of each camera module included in the pose sequence of the target camera module when the projection error is the smallest and the three-dimensional spatial position of the map point in the current SLAM map Take the value to construct a global SLAM map.
  • the implementation of the system shown in Figure 5 enables the camera system to cover the observation scene as much as possible in a limited physical space by using camera modules with different focal lengths, and at the same time, it can reduce the pair of camera modules with different focal lengths through soft focusing.
  • it can be combined with the positioning module of the vehicle to further improve the accuracy of the constructed SLAM map through joint optimization.
  • FIG. 6 is a schematic structural diagram of another SLAM mapping system for a vehicle disclosed in an embodiment of the present invention.
  • the SLAM mapping system of the vehicle shown in FIG. 6 is optimized by the SLAM mapping system of the vehicle shown in FIG. 5.
  • it can also include:
  • the first acquiring unit 403 is configured to acquire multiple frames of images and a global SLAM map captured by the target camera module during relocation;
  • the first acquiring unit 403 can perform the above when the vehicle drives into the real environment indicated by the global SLAM map again, or when the vehicle is lost in the tracking of the feature points during the subsequent positioning process after the relocation is successful Operation
  • the matching unit 404 is configured to match the feature points in the frame image and the map points in the global SLAM map for each frame of the multi-frame image; and, according to the feature points and map points that are matched with each other, determine which frame image corresponds to The camera module pose of the target camera module in the global SLAM map to obtain the second camera module pose sequence of the target camera module;
  • the second acquiring unit 405 is configured to acquire the second vehicle pose sequence of the vehicle corresponding to the second camera module pose sequence through the positioning module of the vehicle;
  • the relocation unit 406 is used to determine the relocation pose of the target camera module in the global SLAM map based on the multi-frame target image and the second camera module pose sequence, so that the relocation pose of the target camera module can be used
  • the second vehicle pose sequence is converted to the SLAM map coordinate system of the global SLAM map, the error between the converted third vehicle pose sequence and the second camera module pose sequence is the smallest; and, according to the target camera module in the global SLAM
  • the relocation pose in the map to determine the relocation pose of the vehicle in the global SLAM map;
  • the coordinate origin of the global SLAM map coordinate system is the optical center position when the target camera module captures the initial frame.
  • system shown in FIG. 6 may also include:
  • the third acquisition unit 407 is used for acquiring the vehicle's positioning module measurement at the current moment after the relocation unit 405 determines the relocation pose of the vehicle in the SLAM map according to the relocation pose of the target camera module in the SLAM map The current pose of the vehicle;
  • the positioning unit 408 is configured to determine the positioning pose of the current pose in the global SLAM map as the positioning result of the vehicle at the current moment according to the measured current pose and the relocation pose of the vehicle in the global SLAM map.
  • the implementation of the system shown in Figure 6 can be initialized through the binocular camera system.
  • the monocular camera module is used to build the map, thus fusing the advantages of monocular and binocular, and improving the success rate of mapping at the same time It can also reduce the amount of data processing.
  • the accuracy of the constructed SLAM map is further improved by combining the positioning module of the vehicle in a joint optimization manner.
  • multi-frame joint relocation is used to improve the accuracy of relocation.
  • the current pose measured by the positioning module of the vehicle is converted to the SLAM map as the positioning result to reduce vehicle positioning. The degree of dependence on visual information increases the probability of successful vehicle positioning.
  • FIG. 7 is a schematic structural diagram of another SLAM mapping system for a vehicle disclosed in an embodiment of the present invention.
  • the system may include:
  • a memory 701 storing executable program codes
  • a processor 702 coupled with the memory 701;
  • Vehicle positioning module 704
  • the image taken by the camera module 703 and the vehicle pose detected by the vehicle positioning module 704 are transmitted to the processor 702, and the processor 702 calls the executable program code stored in the memory 701 to execute the steps shown in FIGS. 1 to 3 Any kind of SLAM mapping method for vehicles.
  • the embodiment of the present invention discloses a vehicle, including the SLAM mapping system of the vehicle shown in FIGS. 4-7.
  • the embodiment of the present invention discloses a computer-readable storage medium that stores a computer program, wherein the computer program causes a computer to execute any SLAM mapping method of a vehicle shown in FIGS. 1 to 3.
  • An embodiment of the present invention discloses a computer program product.
  • the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute any one of FIGS. 1 to 3 A SLAM mapping method for vehicles.
  • the units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the aforementioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory.
  • the essence of the technical solution of the present invention or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory.
  • a computer device which may be a personal computer, a server, or a network device, etc., specifically a processor in a computer device
  • the program can be stored in a computer-readable storage medium.
  • the storage medium includes read-only Memory (Read-Only Memory, ROM), Random Access Memory (RAM), Programmable Read-only Memory (PROM), Erasable Programmable Read Only Memory, EPROM), One-time Programmable Read-Only Memory (OTPROM), Electronically-Erasable Programmable Read-Only Memory (EEPROM), CD-ROM (Compact Disc) Read-Only Memory, CD-ROM) or other optical disk storage, magnetic disk storage, tape storage, or any other computer-readable medium that can be used to carry or store data.
  • Read-Only Memory ROM
  • RAM Random Access Memory
  • PROM Programmable Read-only Memory
  • EPROM Erasable Programmable Read Only Memory
  • OTPROM One-time Programmable Read-Only Memory
  • EEPROM Electronically-Erasable Programmable Read-Only Memory
  • CD-ROM Compact Disc

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Studio Devices (AREA)

Abstract

一种车辆的SLAM建图方法及系统，该方法包括：获取车辆的任意两个摄像模组分别拍摄得到的两个初始帧；其中，两个摄像模组的焦距和视场角可以相同，也可以不同；根据两个摄像模组的内部参数和外部参数以及相互匹配的特征点在两个初始帧中的视差，确定相互匹配的特征点的三维空间位置，以得到相互匹配的特征点对应的地图点，构建出初始的SLAM地图，以完成初始化；在初始化成功之后，获取两个摄像模组中的任意一个目标摄像模组拍摄到的图像；以初始的SLAM地图为基础，结合目标摄像模组拍摄到的图像进行单目视觉SLAM的建图，从而可以通过双目摄像头完成初始化，在初始化成功之后利用单目摄像头进行建图，降低了系统的运算负荷，以提高了建图的成功率。

Description

一种车辆的SLAM建图方法及系统 技术领域
本发明涉及自动驾驶技术领域，具体涉及一种车辆的SLAM建图方法及系统。
背景技术
在自动驾驶领域，单目视觉同步定位与地图创建(Simultaneous Localization and Mapping，SLAM)是指自动驾驶车辆利用单个视觉传感器(如摄像头)创建一个与真实环境相一致的地图，并同时确定自身在地图中的位置。
然而，传统的单目视觉SLAM只能通过动态方法进行初始化，即需要利用摄像头运动过程中拍摄到的图像才能成功初始化。因此，在制图时，需要先将车辆驶出一段距离，完成单目视觉SLAM的初始化之后才能开始进行建图。如果初始化成功的时机较晚，那么车辆需要驶出较长的距离才能开始建图，从而可能导致制图的成功率较低，而且也容易导致构建出的地图中缺少部分环境信息。
发明内容
本发明实施例公开了一种车辆的SLAM建图方法，能够在车辆静止时完成初始化，以提高建图的成功率。
本发明实施例第一方面公开一种车辆的SLAM建图方法，所述方法包括：
获取车辆的任意两个摄像模组在所述车辆静止时分别拍摄得到的两个初始帧；所述两个摄像模组的取景范围至少存在部分重叠；
根据所述两个摄像模组预先标定的内部参数和外部参数以及相互匹配的特征点在所述两个初始帧中的视差，确定所述相互匹配的特征点的三维空间位置，以得到所述相互匹配的特征点对应的地图点，构建出初始的SLAM地图，以完成初始化；
在初始化成功之后，获取所述两个摄像模组中的任意一个目标摄像模组拍摄到的图像；
以所述初始的SLAM地图为基础，结合所述目标摄像模组拍摄到的图像进行单目视觉SLAM的建图。
作为一种可选的实施方式，在本发明实施例第一方面中所述获取车辆的任意两个摄像模组在所述车辆静止时分别拍摄得到的两个初始帧，包括：
获取车辆的任意两个摄像模组在所述车辆静止时分别进行拍摄得到的第一图像和第二图像；其中，所述两个摄像模组的焦距不同；
按照使用相同焦距拍摄得到的图像效果对所述第一图像和所述第二图像进行处理，以处理后的所述第一图像和所述第二图像作为两个初始帧。
作为一种可选的实施方式，在本发明实施例第一方面中，所述以所述初始的SLAM地图为基础，结合所述目标摄像模组拍摄到的图像进行单目视觉SLAM的建图，包括：
对所述目标摄像模组拍摄到的图像中的特征点以及当前SLAM地图中的地图点进行匹配；其中，当所述图像为所述目标摄像模组拍摄到的第一帧图像时，所述当前SLAM地图为所述初始的SLAM地图；
根据相互匹配的特征点和地图点确定所述目标摄像模组拍摄到的每帧图像对应的摄像模组位姿，以得到所述目标摄像模组的第一摄像模组位姿序列；
跟踪未匹配到地图点的特征点在所述目标摄像模组拍摄到的图像中的位置，并结合所述未匹配到地图点的特征点所在的图像对应的摄像模组位姿，确定所述未匹配到地图点的特征点的三维空间位置，以构建出新的地图点加入至所述当前SLAM地图；
通过所述车辆的定位模块获取与所述第一摄像模组位姿序列相对应的所述车辆的第一车辆位姿序列；
迭代调整所述目标摄像模组的位姿序列包含的各个摄像模组位姿的取值以及所述当前SLAM地图中的地图点的三维空间位置的取值，直至所述第一摄像模组位姿序列与第一车辆位姿序列之间的投影误差最小；
以所述投影误差最小时所述目标摄像模组的位姿序列包含的各个摄像模组位姿的取值以及所述当前SLAM地图中的地图点的三维空间位置的取值构建全局SLAM地图。
作为一种可选的实施方式，在本发明实施例第一方面中，所述方法还包括：
在重定位时，获取所述目标摄像模组拍摄到的多帧目标图像以及所述全局SLAM地图；
针对所述多帧目标图像中的每帧目标图像，匹配该帧目标图像中的特征点以及所述全局SLAM地图中的地图点；
根据相互匹配的特征点和地图点确定该帧目标图像对应的所述目标摄像模组在所述全局SLAM地图中的摄像模组位姿，以得到所述目标摄像模组的第二摄像模组位姿序列；
通过所述车辆的定位模块获取与所述第二摄像模组位姿序列对应的所述车辆的第二车辆位姿序列；
基于多帧所述目标图像以及所述第二摄像模组位姿序列确定所述目标摄像模组在所述全局SLAM地图中的重定位位姿，以使利用所述目标摄像模组的所述重定位位姿将所述第二车辆位姿序列转换到所述全局SLAM地图的SLAM地图坐标系时，转换得到的第三车辆位姿序列与所述第二摄像模组位姿序列的误差最小；所述全局SLAM地图坐标系的坐标原点为所述目标摄像模组拍摄到所述初始帧时的光心位置；
根据所述目标摄像模组在所述全局SLAM地图中的重定位位姿，确定所述车辆在所述全局SLAM地图中的重定位位姿。
作为一种可选的实施方式，在本发明实施例第一方面中，在所述根据所述目标摄像模组在所述SLAM地图中的重定位位姿，确定所述车辆在所述SLAM地图中的重定位位姿之后，所述方法还包括：
获取当前时刻所述车辆的定位模块测量到的所述车辆的当前位姿；
根据所述当前位姿以及所述车辆在所述全局SLAM地图中的重定位位姿，确定所述当前位姿在所述全局SLAM地图中的定位位姿作为当前时刻所述车辆的定位结果。
本发明实施例第二方面公开一种车辆的SLAM建图系统，包括：
初始化单元，用于获取车辆的任意两个摄像模组在所述车辆静止时分别拍摄得到的两个初始帧；以及，根据所述两个摄像模组预先标定的内部参数和外部参数，以及相互匹配的特征点在所述两个初始帧中的视差，确定所述相互匹配的特征点的三维空间位置，以得到所述相互匹配的特征点对应的地图点，构建出初始的SLAM地图，以完成初始化；其中，所述两个摄像模组的取景范围至少存在部分重叠；
建图单元，用于在初始化成功之后，获取所述两个摄像模组中的任意一个目标摄像模组拍摄到的图像；以及，以所述初始的SLAM地图为基础，结合所述目标摄像模组拍摄到的图像进行单目视觉SLAM的建图。
作为一种可选的实施方式，在本发明实施例第二方面中，所述初始化单元用于获取车辆的任意两个摄像模组在所述车辆静止时分别拍摄得到的两个初始帧的方式具体包括：
所述初始化单元，用于获取车辆的任意两个摄像模组在所述车辆静止时分别进行拍摄得到的第一图像和第二图像；以及，按照使用相同焦距拍摄得到的图像效果对所述第一图像和所述第二图像进行处理，以处理后的所述第一图像和所述第二图像作为初始帧；其中，所述两个摄像模组的焦距不同。
作为一种可选的实施方式，在本发明实施例第二方面中，所述建图单元，包括：
匹配子单元，用于对所述目标摄像模组拍摄到的图像中的特征点以及当前SLAM地图中的地图点进行匹配；其中，当所述图像为所述目标摄像模组拍摄到的第一帧图像时，所述当前SLAM地图为所述初始的SLAM地图；
定位子单元，用于根据相互匹配的特征点和地图点确定所述目标摄像模组拍摄到的每帧图像对应的摄像模组位姿，以得到所述目标摄像模组的位姿序列；
建图子单元，用于跟踪未匹配到地图点的特征点在所述目标摄像模组拍摄到的图像中的位置，并结合所述未匹配到地图点的特征点所在的图像对应的摄像模组位姿，确定所述未匹配到地图点的特征点的三维空间位置，以构建出新的地图点加入至所述当前SLAM地图；
获取子单元，用于通过所述车辆的定位模块获取与所述目标摄像模组的位姿序列相对应的所述车辆的第一位姿序列；
优化子单元，用于迭代调整所述目标摄像模组的位姿序列包含的各个摄像模组位姿的取值以及所述当前SLAM地图中的地图点的三维空间位置的取值，直至所述目标摄像模组的位姿序列与所述第一位姿序列之间的投影误差最小；
所述建图子单元，还用于以所述投影误差最小时所述目标摄像模组的位姿序列包含的各个摄像模组位姿的取值以及所述当前SLAM地图中的地图点的三维空间位置的取值构建全局SLAM地图。
作为一种可选的实施方式，在本发明实施例第二方面中，所述系统还包括：
第一获取单元，用于在重定位时，获取所述目标摄像模组拍摄到的多帧图像以及所述全局SLAM地图；
匹配单元，用于针对所述多帧图像中的每帧图像，匹配该帧图像中的特征点以及所述全局SLAM地图中的地图点；以及，根据相互匹配的特征点和地图点确定该帧图像对应的所述目标摄像模组在所述全局SLAM地图中的摄像模组位姿，以得到所述目标摄像模组的第二摄像模组位姿序列；
第二获取单元，用于通过所述车辆的定位模块获取与所述第二摄像模组位姿序列对应的所述车辆的第二车辆位姿序列；
重定位单元，用于基于多帧目标图像以及所述第二摄像模组位姿序列确定所述目标摄像模组在所述全局SLAM地图中的重定位位姿，以使利用所述目标摄像模组的所述重定位位姿将所述第二车辆位姿序列转换到所述全局SLAM地图的SLAM地图坐标系时，转换得到的第三车辆位姿序列与所述第二摄像模组位姿序列的误差最小；以及，根据所述目标摄像模组在所述全局SLAM地图中的重定位位姿，确定所述车辆在所述全局SLAM地图中的重定位位姿；
其中，所述全局SLAM地图坐标系的坐标原点为所述目标摄像模组拍摄到所述初始帧时的光心位置。
作为一种可选的实施方式，在本发明实施例第二方面中，所述系统还包括：
第三获取单元，用于在所述重定位单元根据所述目标摄像模组在所述SLAM地图中的重定位位姿，确定所述车辆在所述SLAM地图中的重定位位姿之后，获取当前时刻所述车辆的定位模块测量到的所述车辆的当前位姿；
定位单元，用于根据所述当前位姿以及所述车辆在所述全局SLAM地图中的重定位位姿，确定所述当前位姿在所述全局SLAM地图中的定位位姿作为当前时刻所述车辆的定位结果。
本发明实施例第三方面公开一种车辆，所述车辆包括本发明实施例第二方面公开的任一项车辆的SLAM建图系统。
本发明实施例第四方面公开一种车辆的SLAM建图系统，包括：
存储有可执行程序代码的存储器；
与所述存储器耦合的处理器；
摄像模组及车辆定位模块；
所述摄像模组拍摄得到的图像以及所述车辆定位模块检测到的车辆位姿传输至所述处理器，所述处理器调用所述存储器中存储的所述可执行程序代码，执行本发明实施例第一方面公开的任一项方法。
本发明第五方面公开一种计算机可读存储介质，其存储计算机程序，其中，所述计算机程序使得计算机执行本发明实施例第一方面公开的任一项方法。
本发明实施例第六方面公开一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得所述计算机执行本发明实施例第一方面公开的任一项方法。
与现有技术相比，本发明实施例具有以下有益效果：
本发明实施例在车辆静止时，通过车辆的任意两个摄像模组分别拍摄得到两个初始帧；利用初始帧中相互匹配的特征点构建出初始的SLAM地图，从而可以通过双目摄像模组进行初始化；在初始化成功之后，利用两个摄像模组中的任意一个目标摄像模组拍摄图像，以进行单目视觉SLAM的建图，从而可以融合双目和单目的优点，在车辆静止时完成初始化，不需要车辆运动一定距离，进而可以提高建图的成功率，减少地图中的信息缺失。
附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例公开的一种车辆的SLAM建图方法的流程示意图；
图2是本发明实施例公开的另一种车辆的SLAM建图方法的流程示意图；
图3是本发明实施例公开的又一种车辆的SLAM建图方法的流程示意图；
图4是本发明实施例公开的一种车辆的SLAM建图系统的结构示意图；
图5是本发明实施例公开的另一种车辆的SLAM建图系统的结构示意图；
图6是本发明实施例公开的又一种车辆的SLAM建图系统的结构示意图；
图7是本发明实施例公开的再一种车辆的SLAM建图系统的结构示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
需要说明的是，本发明实施例及附图中的术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其它步骤或单元。
本发明实施例公开了一种车辆的SLAM建图方法及系统，能够在车辆静止时完成初始化，以提高建图的成功率。以下分别进行详细说明。
实施例一
请参阅图1，图1是本发明实施例公开的一种车辆的SLAM建图方法的流程示意图。其中，图1所描述的车辆的SLAM建图方法适用于车载电脑、电子控制单元(Electronic Control Unit，ECU)等车 载电子设备，本发明实施例不做限定。如图1所示，该车辆的SLAM建图方法可以包括以下步骤：
101、车载电子设备获取车辆的任意两个摄像模组在车辆静止时分别拍摄得到的两个初始帧。
本发明实施例中，两个摄像模组的取景范围至少存在部分重叠。作为一种可选的实施方式，上述的两个摄像模组可以为多目摄像系统中的任意两个摄像模组。多目摄像系统中可以包含三个或以上的摄像模组，每个摄像模组之间的设置距离在可以在10cm-20cm之间，均朝向车辆的行驶方向设置。
作为另一种可选的实施方式，由于多目摄像系统中相邻的摄像模组之间的相对距离并不远，为了使得特征点在两个初始帧中的视差足够大，可以选取多目摄像系统中相对距离最远的两个摄像模组(如分别位于两端的摄像模组)拍摄图像，将上述的相对距离最远的两个摄像模组分别拍摄得到的两张图像作为两个初始帧。
可以理解的是，初始化成功的前提条件是初始帧中存在相互匹配的特征点，其中，相互匹配的特征点为三维空间中同一个物体分别被两个摄像模组拍摄之后，分别投影到两个初始帧中得到的投影点。因此，两个摄像模组的取景范围应该至少存在部分重叠，以保证初始帧中可以存在相互匹配的特征点。优选的，还可以控制两个摄像模组同时拍摄图像，进一步保证初始帧中可以存在相互匹配的特征点。此外，初始帧应该满足一定要求。比如，可以设定初始帧中的特征点数目应该大于预设阈值(如100)。也就是说，上述的两个摄像模组拍摄到的图像中，如果摄像模组取景范围的重叠部分对应的图像中的特征点数目大于上述的预设阈值，那么将拍摄到的图像设置为初始帧；否则，两个摄像模组重新拍摄图像。
102、车载电子设备根据上述的两个摄像模组预先标定的内部参数和外部参数，以及相互匹配的特征点在两个初始帧中的视差，确定相互匹配的特征点的三维空间位置，以得到相互匹配的特征点对应的地图点，构建出初始的SLAM地图，以完成初始化。
本发明实施例中，基于立体视觉的原理，当两个摄像模组的各自的内部参数(如焦距、畸变系数等)以及两个摄像模组相对的外部参数(如基线、右侧摄像模组相对于左侧摄像模组的平移量和旋转量等)已知，结合相互匹配的特征点在两个初始帧中的视差，即可确定出上述的相互匹配的特征点的三维空间位置。
可以理解的是，摄像模组的内部参数可用于表征三维空间点与图像像素点之间的映射关系，因此基于摄像模组的内部参数、以及特征点在图像中的位置，即可恢复出特征点相对于摄像模组的相对位置，但缺少深度信息。进一步地，特征点的深度信息d的求取方式可以通过以下简化的数学模型表示：d＝b*f/p；b为基线(即两个摄像模组的光心之间的距离)，f为焦距，p为相互匹配的特征点在两帧图像中的视差。
假设以两个摄像模组中的任意一个摄像模组在拍摄到初始帧时的位置作为坐标系原点，通过上述的实施方式，可以确定出重复出现在两个初始帧中的特征点相对于坐标系原点的三维空间位置，从而得到对应的地图点，这些地图点构成了初始的SLAM地图，SLAM系统的初始化完成。
103、在初始化成功之后，车载电子设备获取两个摄像模组中的任意一个目标摄像模组拍摄到的图像。
本发明实施例中，为了方便计算，目标摄像模组可以为上述被选取为坐标系原点的摄像模组。作为一种可选的实施方式，在初始化成功之后，车辆可以开始行驶，目标摄像模组可以在车辆不断行驶的过程中，持续拍摄多帧图像。
104、车载电子设备以初始的SLAM地图为基础，结合目标摄像模组拍摄到的图像进行单目视觉SLAM的建图。
本发明实施例中，在初始化成功之后，即可转换为单目视觉SLAM，也就是说，只使用一个目标摄 像模组拍摄图像。可以理解的是，单目SLAM的建图具体可以包括以下步骤：
当目标摄像模组拍摄到第一帧图像时，对第一帧图像中的特征点以及初始的SLAM地图中的地图点进行匹配(即2D-3D的匹配)；
针对相互匹配的特征点和地图点，根据特征点在第一帧图像中的图像位置以及对应匹配的地图点的三维空间位置，可以确定出第一帧图像对应的相机(即摄像模组)位姿；
针对目标摄像模组在第一帧图像之后拍摄到的每一帧图像，重复上述的步骤，从而可以确定出每帧图像对应的相机位姿；同时，针对图像中未匹配到地图点的特征点，在目标摄像模组拍摄到的图像中跟踪这些特征点，根据这些特征点所在图像中的图像位置，以及这些特征点所在图像对应的相机位姿，可以计算出这些特征点的三维空间位置，从而得到新的地图点加入至SLAM地图中。
综上，通过不断添加地图点，可以构建出车辆途经环境的地图。
可见，在图1所描述的方法中，在初始化时，可以利用两个摄像模组构成双目摄像系统，与只有单个摄像模组的单目摄像系统相比，双目摄像系统在车辆静止时已经可以获取到存在视差的初始帧，因此无需车辆行驶一段距离以进行初始化，在车辆静止时即可完成初始化，从而可以减少建图时的数据丢失；而在初始化完成之后，只利用一个摄像模组拍摄得到的图像进行建图，与利用双目摄像系统进行建图的方案相比，只使用单个摄像模组可以减少需要处理的图像数据量。也就是说，图1所描述的方法可以融合单目和双目的优点，在提高建图成功率的同时还可以减少数据处理量。
此外，上述的步骤101～步骤102可以为初始启动建图时执行的步骤，也可以为建图过程中一旦检测到建图失败时执行的步骤。也就是说，当检测到建图失败时，可以启动车辆的任意两个摄像模组，优选的，可以包含目标摄像模组，然后通过这两个摄像模组拍摄图像，以作为车辆静止时获取到的初始帧。其中，建图失败可以包括目标摄像模组拍摄到的图像中存在的特征点数量较少，或者图像中与地图点相互匹配的特征点较少等情况，本发明实施例不做限定。因此，执行上述的步骤101～步骤102还可以在建图失败时快速进行再次初始化，从而再次进行建图，相较于需要车辆移动进行初始化的单目摄像系统，可以减少两次建图之间的数据缺失。
实施例二
请参阅图2，图2是本发明实施例公开的另一种车辆的SLAM建图方法的流程示意图。如图2所示，该车辆的SLAM建图方法可以包括以下步骤：
201、车载电子设备获取车辆的任意两个摄像模组在车辆静止时分别拍摄得到的两个初始帧。
本发明实施例中，上述的两个摄像模组可以为多目摄像系统中的任意两个摄像模组，两个摄像模组的焦距和视场角可以相同，也可以不同。然而，多目摄像系统由于摄像模组数量较多，为了减少图像数据处理量，加快处理时间，一般会选择采用固定焦距的摄像模组。进一步地，为了扩大能够覆盖的观察距离，不同的摄像模组一般会设定为不同的焦距，以减少单个摄像模组无法切换焦距带来的问题。此外，不同的摄像模组的视场角也可能不同，比如三个摄像模组分别采用120°、60°和30°的视场角，以使不同的摄像模组覆盖不同角度范围的场景。
当焦距和视场角相同时，两个摄像模组拍摄得到的图像效果相同，因此在执行下述的步骤202对初始帧进行特征点匹配的准确率比较高。当焦距和视场角不同时，两个摄像模组拍摄得到的图像中物体的形变程度不同，从而会降低特征点匹配的准确率。因此，基于双目立体视觉恢复特征点三维空间位置时，使用焦距相同的两个摄像模组拍摄初始帧，恢复出的三维空间位置的精度可能高于使用不同焦距的两个摄像模组拍摄初始帧。在本发明实施例中，在采用不同焦距的摄像模组的同时，为了减少不同焦距的摄像模组对三维空间位置计算精度的影响，可以执行以下步骤：
获取车辆的任意两个摄像模组在车辆静止时分别进行拍摄得到的第一图像和第二图像；
按照使用相同焦距拍摄得到的图像效果对第一图像和第二图像进行处理，以处理后的第一图像和第二图像作为两个初始帧。
其中，假设两个摄像模组中一个摄像模组的焦距为f A，另一个摄像模组的焦距为f C，那么上述的相同焦距f AC可以满足以下条件：f AC∈[f A，f C]。可以理解的是，在标定摄像模组的参数时，可以先独立标定其中一个摄像模组的内部参数，以得到上述的f A，再独立标定另一个摄像模组的内部参数，以得到上述的f C，而非同时标定上述的两个摄像模组。
执行上述的步骤，可以通过软调焦的方式使得两个不同焦距的摄像模组拍摄得到的图像调整为相同焦距拍摄得到的图像效果，从而可以减少不同焦距的摄像模组对三维空间位置计算精度的影响，从而减少对建图精度的影响。
此外，下述的步骤202～步骤203与步骤102～步骤103相同，以下内容不再赘述。
204、车载电子设备对目标摄像模组拍摄到的图像中的特征点以及当前SLAM地图中的地图点进行匹配，以获得目标摄像模组的第一摄像模组位姿序列，并构建出新的地图点加入至当前SLAM地图。
本发明实施例中，当前SLAM地图为当前时刻已经构建出的SLAM地图，当用于进行特征点匹配的图像为目标摄像模组拍摄到的第一帧图像时，当前SLAM地图为初始的SLAM地图。
具体地，可以根据相互匹配的特征点和地图点确定目标摄像模组拍摄到的每帧图像对应的摄像模组位姿，以得到目标摄像模组的第一摄像模组位姿序列；其中，第一摄像模组位姿序列中的各个位姿可以与目标摄像模组拍摄到的图像一一对应；
针对未匹配到地图点的特征点，可以跟踪未匹配到地图点的特征点在目标摄像模组拍摄到的图像中的位置，并结合未匹配到地图点的特征点所在的图像对应的摄像模组位姿，确定未匹配到地图点的特征点的三维空间位置，以构建出新的地图点加入至当前SLAM地图。
205、车载电子设备通过车辆的定位模块获取与第一摄像模组位姿序列相对应的车辆的第一车辆位姿序列。
本发明实施例中，车辆上可以装设有惯性测量单元(Inertial measurement unit，IMU)、轮脉冲计数器等传感器，这些传感器可以作为车辆的定位模块(如车身里程计)计算车辆的行驶距离，进一步地还可以对车辆进行位置定位。第一车辆位姿序列中可以包括多个由车辆的定位模块测量出的车辆位姿，并且每个车辆位姿与第一摄像模组位姿序列中的某一个摄像模组位姿对应。可以理解的是，假设某一时刻目标摄像模组拍摄到一张图像，根据该图像中的特征点与地图点进行匹配，确定出该图像对应的摄像模组位姿；同时，在该时刻车辆的定位模块测量出车辆位姿，那么该车辆位姿与上述的摄像模组位姿对应。
206、车载电子设备迭代调整目标摄像模组的位姿序列包含的各个摄像模组位姿的取值以及当前SLAM地图中的地图点的三维空间位置的取值，直至第一摄像模组位姿序列与第一车辆位姿序列之间的投影误差最小。
本发明实施例中，在迭代调整的过程中，摄像模组位姿的取值、地图点的三维空间位置的取值应该满足联合优化(Bundle Adjustment)所需的约束条件。本发明实施例中，可以认为车辆的定位模块测量出的车辆位姿相对准确，因此，以上述的第一车辆位姿序列为准，假设摄像模组位姿的取值以及地图点的三维空间位置的取值相对准确，那么第一摄像模组位姿序列与第一车辆位姿序列之间的差距较小。其中，可以理解的是，第一摄像模组位姿序列中的摄像模组位姿可以为相机坐标系下的位姿，第一车辆位姿序列中的车辆位姿可以为车辆坐标系下的位姿；相机坐标系以摄像模组的光心为坐标原点，车辆坐标系以车辆的某一部位(如后轴中心点)为坐标原点；相机坐标系与车辆坐标系之间的转 换关系可以预先标定得到。第一摄像模组位姿序列与第一车辆位姿序列之间的差距较小，可以为第一摄像模组位姿序列与第一车辆位姿序列之间的差距为相机坐标系与车辆坐标系之间的转换关系，也就是说，如果基于相机坐标系与车辆坐标系之间的转换关系将第一摄像模组位姿序列转换至车辆坐标系下，转换得到的位姿序列与第一车辆位姿序列重合。
207、车载电子设备以投影误差最小时目标摄像模组的位姿序列包含的各个摄像模组位姿的取值以及当前SLAM地图中的地图点的三维空间位置的取值构建全局SLAM地图。
本发明实施例中，全局SLAM地图可以为车载电子设备停止执行建图程序时最终得到的SLAM地图。
可见，实施如图2所示的方法，可以通过不同焦距的摄像模组在有限的物理空间内使得摄像系统尽可能多地覆盖观察场景，同时通过软调焦的方式可以减少不同焦距的摄像模组对三维空间位置计算精度的影响，从而减少对建图精度的影响。此外，还可以结合车辆的定位模块，通过联合优化的方式进一步地提高构建出的SLAM地图的精度。
实施例三
请参阅图3，图3是本发明实施例公开的又一种车辆的SLAM建图方法的流程示意图。如图3所示，该车辆的SLAM建图方法可以包括以下步骤：
步骤301～步骤304与步骤101～步骤104相同，其中，步骤304的具体实现方式可以如步骤204～步骤206所示，以下内容不再赘述。
305、车载电子设备在重定位时，获取目标摄像模组拍摄到的多帧目标图像以及上述的全局SLAM地图。
本发明实施例中，全局SLAM地图为真实环境的数字化描述，在构建出全局SLAM地图之后，车载电子设备可以重用全局SLAM地图。类似于人类利用地图进行定位的方式，车载电子设备可以定位车辆在全局SLAM地图中的位置，从而获知车辆在真实环境中的位置。在车辆再次驶入全局SLAM地图所指示的真实环境，或者车辆在重定位成功之后的后续定位过程中发生特征点跟踪丢失的情况，都可以触发重定位，以再次确定车辆在全局SLAM地图中的位置。具体地，可以执行下述的步骤306～步骤309进行重定位。
306、车载电子设备针对多帧目标图像中的每帧目标图像，匹配该帧目标图像中的特征点以及全局SLAM地图中的地图点，并根据相互匹配的特征点和地图点确定该帧目标图像对应的目标摄像模组在全局SLAM地图中的摄像模组位姿，以得到目标摄像模组的第二摄像模组位姿序列。
本发明实施例中，可以理解的是，每帧目标图像对应第二摄像模组位姿序列中的一个摄像模组位姿，第二摄像模组位姿序列中的摄像模组位姿为目标摄像模组的相机坐标系下的位姿。
307、车载电子设备通过车辆的定位模块获取与第二摄像模组位姿序列对应的车辆的第二车辆位姿序列。
本发明实施例中，第二车辆位姿序列中包含的各个车辆位姿与第二摄像模组位姿序列中包含的摄像模组位姿对应，车载电子设备可以在目标摄像模组拍摄到目标图像的同一时刻，通过定位模块获取车辆位姿，从而构成与第二摄像模组位姿序列对应的第二车辆位姿序列，并且第二车辆位姿序列中的车辆位姿可以为车辆坐标系下的位姿。
308、车载电子设备基于多帧目标图像以及第二摄像模组位姿序列确定目标摄像模组在全局SLAM地图中的重定位位姿，以使利用目标摄像模组的重定位位姿将第二车辆位姿序列转换到全局SLAM地图的SLAM地图坐标系时，转换得到的第三车辆位姿序列与第二摄像模组位姿序列的误差最小。
本发明实施例中，目标摄像模组在全局SLAM地图中的重定位位姿，即当前目标摄像模组的位姿与 全局SLAM地图坐标系原点之间的相对转换关系(平移量和旋转量)，也就是目标摄像模组的相机坐标系与全局SLAM地图坐标系之间的转换关系；其中，全局SLAM地图坐标系原点可以为目标摄像模组拍摄上述的初始帧时的光心位置。如果目标摄像模组的重定位位姿已知，基于车辆坐标系与目标摄像模组的相机坐标系之间的转换关系，以及相机坐标系与全局SLAM地图坐标系之间的转换关系，可以将车辆坐标系下的第二车辆位姿序列转换至全局SLAM地图中，转换得到的第三车辆位姿序列可以认为是车辆在地图中的移动轨迹。而第二摄像模组位姿序列可以认为是目标摄像模组在地图中的移动轨迹，如果目标摄像模组的重定位位姿准确，目标摄像模组在地图中的移动轨迹与车辆在地图中的移动轨迹之间的差距比较小，具体可以为第三车辆位姿序列与第二摄像模组位姿序列之间的差距为相机坐标系与车辆坐标系之间的转换关系。
作为一种可选的实施方式，可以再次通过联合优化所限定的目标图像、目标图像对应的摄像模组位姿以及地图点的三维空间位置之间的约束关系，不断迭代调整第二摄像模组位姿序列中各个摄像模组位姿的取值，以第二摄像模组位姿序列中最后一个摄像模组位姿的取值作为目标摄像模组的重定位位姿，或者以第二摄像模组位姿序列中各个摄像模组位姿的平均值作为目标摄像模组的重定位位姿，将第二车辆位姿序列转换至全局SLAM地图中，直至第三车辆位姿序列与第二摄像模组位姿序列之间的误差最小。误差最小时目标摄像模组的重定位位姿的取值即为最终的重定位位姿。
309、车载电子设备根据目标摄像模组在全局SLAM地图中的重定位位姿，确定车辆在全局SLAM地图中的重定位位姿。
本发明实施例中，可以理解的是，利用车辆坐标系与目标摄像模组的相机坐标系之间的转换关系，即可通过目标摄像模组的重定位位姿确定出车辆在全局SLAM地图中的重定位位姿。
通过执行上述的步骤305～步骤309，可以重用全局SLAM地图，以对车辆进行定位，同时，还可以通过多帧联合重定位的方式，提高重定位的精度。
310、车载电子设备获取当前时刻车辆的定位模块测量到的车辆的当前位姿。
311、车载电子设备根据当前位姿以及车辆在全局SLAM地图中的重定位位姿，确定当前位姿在全局SLAM地图中的定位位姿作为当前时刻车辆的定位结果。
本发明实施例中，车辆的定位模块测量到的当前位姿为车辆坐标系下的位姿，车辆在全局SLAM地图中的重定位位姿可以理解为车辆坐标系与全局SLAM地图坐标系之间的转换关系，因此，基于车辆在全局SLAM地图中的重定位位姿，可以将车辆的当前位姿转换至全局SLAM地图中。
由于基于视觉SLAM的车辆定位依赖于特征点匹配，一旦摄像模组被遮挡或者摄像模组拍摄到白墙等特征点较少的图像，视觉SLAM系统将无法进行定位。因此在本发明实施例中，车载电子设备在重定位成功(即确定出车辆在全局SLAM地图中的重定位位姿)之后，以车辆的定位模块测量到的车辆的当前位姿为准，将当前位姿转换到SLAM地图中，作为最终的定位结果，从而可以降低车辆定位对视觉信息的依赖程度，提高车辆定位成功的概率，大幅度摆脱了视觉定位过程的不稳定因素，提高了系统在视觉遮挡或视觉特征丢失等情况下的稳定性。
可见，在图3所描述的方法中，可以通过双目摄像系统进行初始化，在初始化成功之后通过单目摄像模组进行建图，从而融合单目和双目的优点，在提高建图成功率的同时还可以减少数据处理量。进一步地，在建图过程中，通过结合车辆的定位模块，以联合优化的方式进一步地提高构建出的SLAM地图的精度。在重定位时，通过多帧联合重定位的方式，提高重定位的精度，在重定位之后，将车辆的定位模块测量得到的当前位姿转换至SLAM地图中作为定位结果，以降低车辆定位对视觉信息的依赖程度，提高车辆定位成功的概率。
实施例四
请参阅图4，图4是本发明实施例公开的一种车辆的SLAM建图系统的结构示意图。如图4所示，该系统包括：
初始化单元401，用于获取车辆的任意两个摄像模组在车辆静止时分别拍摄得到的两个初始帧；以及，根据两个摄像模组预先标定的内部参数和外部参数，以及相互匹配的特征点在两个初始帧中的视差，确定相互匹配的特征点的三维空间位置，以得到相互匹配的特征点对应的地图点，构建出初始的SLAM地图，以完成初始化；其中，两个摄像模组的取景范围至少存在部分重叠；
本发明实施例中，初始化单元401可以获取任意两个摄像模组拍摄得到的图像作为初始帧，也可以获取摄像模组所在的多目摄像系统中相对距离最远的两个摄像模组(如分别位于两端的摄像模组)拍摄到的图像作为初始帧。
此外，如果初始化单元401判断出摄像模组取景范围的重叠部分对应的图像中的特征点数目未大于上述的预设阈值，可以控制摄像模组重新拍摄图像，直至获取到上述的特征点数目大于上述的预设阈值的图像作为初始帧。
建图单元402，用于在初始化成功之后，获取两个摄像模组中的任意一个目标摄像模组拍摄到的图像；以及，以初始的SLAM地图为基础，结合目标摄像模组拍摄到的图像进行单目视觉SLAM的建图。
本发明实施例中，可以理解的是，建图单元402具体可以通过以下操作进行建图：
建图单元402用于对目标摄像模组拍摄到的第一帧图像中的特征点以及初始的SLAM地图中的地图点进行匹配；
针对相互匹配的特征点和地图点，建图单元402根据特征点在第一帧图像中的图像位置以及对应匹配的地图点的三维空间位置，可以确定出第一帧图像对应的相机(即摄像模组)位姿；
针对目标摄像模组在第一帧图像之后拍摄到的每一帧图像，建图单元402重复上述的操作，从而可以确定出每帧图像对应的相机位姿；同时，针对图像中未匹配到地图点的特征点，在目标摄像模组拍摄到的图像中跟踪这些特征点，根据这些特征点所在图像中的图像位置，以及这些特征点所在图像对应的相机位姿，可以计算出这些特征点的三维空间位置，从而得到新的地图点加入至SLAM地图中。
此外，建图单元402在建图的过程中一旦检测到建图失败，即可触发初始化单元401执行上述的操作，从而可以在建图失败时快速进行再次初始化，从而再次进行建图，相较于需要车辆移动进行初始化的单目摄像系统，可以减少两次建图之间的数据缺失。
实施如图4所示的系统，可以在初始化时，利用两个摄像模组构成双目摄像系统，从而无需车辆行驶一段距离以进行初始化，能够在车辆静止时即可完成初始化；在初始化成功之后，只利用一个摄像模组拍摄得到的图像进行建图，以减少需要处理的图像数据量；因此，可以融合单目和双目的优点，在提高建图成功率的同时还可以减少数据处理量。此外，还可以在建图失败时快速进行再次初始化，以减少两次建图之间的数据缺失。
实施例五
请参阅图5，图5是本发明实施例公开的另一种车辆的SLAM建图系统的结构示意图。其中，图5所示的车辆的SLAM建图系统是由图4所示的车辆的SLAM建图系统进行优化得到的。
其中，图5所示的系统中，两个摄像模组的焦距不同；相应地，上述的初始化单元401用于获取车辆的任意两个摄像模组在车辆静止时分别拍摄得到的两个初始帧的方式具体可以为：
初始化单元401，用于获取车辆的任意两个摄像模组在车辆静止时分别进行拍摄得到的第一图像和第二图像；以及，按照使用相同焦距拍摄得到的图像效果对第一图像和第二图像进行处理，以处理 后的第一图像和第二图像作为两个初始帧。
其中，假设两个摄像模组中一个摄像模组的焦距为f A，另一个摄像模组的焦距为f C，那么上述的相同焦距f AC可以满足以下条件：f AC∈[f A，f C]。通过上述的实施方式，初始化单元401可以通过软调焦的方式使得两个不同焦距的摄像模组拍摄得到的图像调整为相同焦距拍摄得到的图像效果，从而可以减少不同焦距的摄像模组对三维空间位置计算精度的影响，从而减少对建图精度的影响。
可选的，在图5所示的系统中，上述的建图单元402，可以包括：
匹配子单元4021，用于对目标摄像模组拍摄到的图像中的特征点以及当前SLAM地图中的地图点进行匹配；其中，当前SLAM地图为当前时刻已经构建出的SLAM地图，当用于进行特征点匹配的图像为目标摄像模组拍摄到的第一帧图像时，当前SLAM地图为初始的SLAM地图；
定位子单元4022，用于根据相互匹配的特征点和地图点确定目标摄像模组拍摄到的每帧图像对应的摄像模组位姿，以得到目标摄像模组的位姿序列；
建图子单元4023，用于跟踪未匹配到地图点的特征点在目标摄像模组拍摄到的图像中的位置，并结合未匹配到地图点的特征点所在的图像对应的摄像模组位姿，确定未匹配到地图点的特征点的三维空间位置，以构建出新的地图点加入至当前SLAM地图；
获取子单元4024，用于通过车辆的定位模块获取与目标摄像模组的位姿序列相对应的车辆的第一位姿序列；其中，定位模块可以为惯性测量单元(Inertiai measurement unit，IMU)、轮脉冲计数器等传感器，这些传感器可以作为车辆的定位模块(如车身里程计)计算车辆的行驶距离，进一步地还可以对车辆进行位置定位。
优化子单元4025，用于迭代调整目标摄像模组的位姿序列包含的各个摄像模组位姿的取值以及当前SLAM地图中的地图点的三维空间位置的取值，直至目标摄像模组的位姿序列与第一位姿序列之间的投影误差最小；
相应地，上述的建图子单元4023，用于以投影误差最小时目标摄像模组的位姿序列包含的各个摄像模组位姿的取值以及当前SLAM地图中的地图点的三维空间位置的取值构建全局SLAM地图。
实施如图5所示的系统，可以通过不同焦距的摄像模组在有限的物理空间内使得摄像系统尽可能多地覆盖观察场景，同时通过软调焦的方式可以减少不同焦距的摄像模组对三维空间位置计算精度的影响，从而减少对建图精度的影响。此外，还可以结合车辆的定位模块，通过联合优化的方式进一步地提高构建出的SLAM地图的精度。
实施例六
请参阅图6，图6是本发明实施例公开的又一种车辆的SLAM建图系统的结构示意图。其中，图6所示的车辆的SLAM建图系统是由图5所示的车辆的SLAM建图系统进行优化得到的。如图6所示，还可以包括：
第一获取单元403，用于在重定位时，获取目标摄像模组拍摄到的多帧图像以及全局SLAM地图；
本发明实施例中，第一获取单元403在车辆再次驶入全局SLAM地图所指示的真实环境，或者车辆在重定位成功之后的后续定位过程中发生特征点跟踪丢失的情况时，都可以执行上述的操作；
匹配单元404，用于针对多帧图像中的每帧图像，匹配该帧图像中的特征点以及全局SLAM地图中的地图点；以及，根据相互匹配的特征点和地图点确定该帧图像对应的目标摄像模组在全局SLAM地图中的摄像模组位姿，以得到目标摄像模组的第二摄像模组位姿序列；
第二获取单元405，用于通过车辆的定位模块获取与第二摄像模组位姿序列对应的车辆的第二车辆位姿序列；
重定位单元406，用于基于多帧目标图像以及第二摄像模组位姿序列确定目标摄像模组在全局SLAM地图中的重定位位姿，以使利用目标摄像模组的重定位位姿将第二车辆位姿序列转换到全局SLAM地图的SLAM地图坐标系时，转换得到的第三车辆位姿序列与第二摄像模组位姿序列的误差最小；以及，根据目标摄像模组在全局SLAM地图中的重定位位姿，确定车辆在全局SLAM地图中的重定位位姿；
其中，全局SLAM地图坐标系的坐标原点为目标摄像模组拍摄到初始帧时的光心位置。
可选的，图6所示的系统还可以包括：
第三获取单元407，用于在重定位单元405根据目标摄像模组在SLAM地图中的重定位位姿，确定车辆在SLAM地图中的重定位位姿之后，获取当前时刻车辆的定位模块测量到的车辆的当前位姿；
定位单元408，用于根据测量出的当前位姿以及车辆在全局SLAM地图中的重定位位姿，确定当前位姿在全局SLAM地图中的定位位姿作为当前时刻车辆的定位结果。
实施如图6所示的系统，可以通过双目摄像系统进行初始化，在初始化成功之后通过单目摄像模组进行建图，从而融合单目和双目的优点，在提高建图成功率的同时还可以减少数据处理量。进一步地，在建图过程中，通过结合车辆的定位模块，以联合优化的方式进一步地提高构建出的SLAM地图的精度。在重定位时，通过多帧联合重定位的方式，提高重定位的精度，在重定位之后，将车辆的定位模块测量得到的当前位姿转换至SLAM地图中作为定位结果，以降低车辆定位对视觉信息的依赖程度，提高车辆定位成功的概率。
实施例七
请参阅图7，图7是本发明实施例公开的再一种车辆的SLAM建图系统的结构示意图。如图7所示，该系统可以包括：
存储有可执行程序代码的存储器701；
与存储器701耦合的处理器702；
摄像模组703；
车辆定位模块704；
其中，摄像模组703拍摄得到的图像以及车辆定位模块704检测到的车辆位姿传输至处理器702，处理器702调用存储器701中存储的可执行程序代码，执行图1～图3所示的任一种车辆的SLAM建图方法。
本发明实施例公开一种车辆，包括图4～图7所示的车辆的SLAM建图系统。
本发明实施例公开一种计算机可读存储介质，其存储计算机程序，其中，该计算机程序使得计算机执行图1～图3所示的任一种车辆的SLAM建图方法。
本发明实施例公开一种计算机程序产品，该计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，且该计算机程序可操作来使计算机执行图1～图3所示的任一种车辆的SLAM建图方法。
应理解，说明书通篇中提到的“一个实施例”或“一实施例”意味着与实施例有关的特定特征、结构或特性包括在本发明的至少一个实施例中。因此，在整个说明书各处出现的“在一个实施例中”或“在一实施例中”未必一定指相同的实施例。此外，这些特定特征、结构或特性可以以任意适合的方式结合在一个或多个实施例中。本领域技术人员也应该知悉，说明书中所描述的实施例均属于可选实施例，所涉及的动作和模块并不一定是本发明所必须的。
在本发明的各种实施例中，应理解，上述各过程的序号的大小并不意味着执行顺序的必然先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本发明实施例的实施过程构成任何限定。
上述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物单元，即可位于一个地方，或者也可以分布到多个网络单元上。可根据实际的需要选择其中的部分或全部单元来实现本实施例方案的目的。
另外，在本发明各实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
上述集成的单元若以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可获取的存储器中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或者部分，可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储器中，包括若干请求用以使得一台计算机设备(可以为个人计算机、服务器或者网络设备等，具体可以是计算机设备中的处理器)执行本发明的各个实施例上述方法的部分或全部步骤。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质包括只读存储器(Read-Only Memory，ROM)、随机存储器(Random Access Memory，RAM)、可编程只读存储器(Programmable Read-only Memory，PROM)、可擦除可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)、一次可编程只读存储器(One-time Programmable Read-Only Memory，OTPROM)、电子抹除式可复写只读存储器(Electrically-Erasable Programmable Read-Only Memory，EEPROM)、只读光盘(Compact Disc Read-Only Memory，CD-ROM)或其他光盘存储器、磁盘存储器、磁带存储器、或者能够用于携带或存储数据的计算机可读的任何其他介质。
以上对本发明实施例公开的一种车辆的SLAM建图方法及系统进行了详细介绍，本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想。同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本发明的限制。

Claims (11)

  1. 一种车辆的SLAM建图方法，其特征在于，包括：
    获取车辆的任意两个摄像模组在所述车辆静止时分别拍摄得到的两个初始帧；所述两个摄像模组的取景范围至少存在部分重叠；
    根据所述两个摄像模组预先标定的内部参数和外部参数以及相互匹配的特征点在所述两个初始帧中的视差，确定所述相互匹配的特征点的三维空间位置，以得到所述相互匹配的特征点对应的地图点，构建出初始的SLAM地图，以完成初始化；
    在初始化成功之后，获取所述两个摄像模组中的任意一个目标摄像模组拍摄到的图像；
    以所述初始的SLAM地图为基础，结合所述目标摄像模组拍摄到的图像进行单目视觉SLAM的建图。
  2. 根据权利要求1所述的方法，其特征在于，所述获取车辆的任意两个摄像模组在所述车辆静止时分别拍摄得到的两个初始帧，包括：
    获取车辆的任意两个摄像模组在所述车辆静止时分别进行拍摄得到的第一图像和第二图像；其中，所述两个摄像模组的焦距不同；
    按照使用相同焦距拍摄得到的图像效果对所述第一图像和所述第二图像进行处理，以处理后的所述第一图像和所述第二图像作为两个初始帧。
  3. 根据权利要求1或2所述的方法，其特征在于，所述以所述初始的SLAM地图为基础，结合所述目标摄像模组拍摄到的图像进行单目视觉SLAM的建图，包括：
    对所述目标摄像模组拍摄到的图像中的特征点以及当前SLAM地图中的地图点进行匹配；其中，当所述图像为所述目标摄像模组拍摄到的第一帧图像时，所述当前SLAM地图为所述初始的SLAM地图；
    根据相互匹配的特征点和地图点确定所述目标摄像模组拍摄到的每帧图像对应的摄像模组位姿，以得到所述目标摄像模组的第一摄像模组位姿序列；
    跟踪未匹配到地图点的特征点在所述目标摄像模组拍摄到的图像中的位置，并结合所述未匹配到地图点的特征点所在的图像对应的摄像模组位姿，确定所述未匹配到地图点的特征点的三维空间位置，以构建出新的地图点加入至所述当前SLAM地图；
    通过所述车辆的定位模块获取与所述第一摄像模组位姿序列相对应的所述车辆的第一车辆位姿序列；
    迭代调整所述目标摄像模组的位姿序列包含的各个摄像模组位姿的取值以及所述当前SLAM地图中的地图点的三维空间位置的取值，直至所述第一摄像模组位姿序列与第一车辆位姿序列之间的投影误差最小；
    以所述投影误差最小时所述目标摄像模组的位姿序列包含的各个摄像模组位姿的取值以及所述当前SLAM地图中的地图点的三维空间位置的取值构建全局SLAM地图。
  4. 根据权利要求3所述的方法，其特征在于，所述方法还包括：
    在重定位时，获取所述目标摄像模组拍摄到的多帧目标图像以及所述全局SLAM地图；
    针对所述多帧目标图像中的每帧目标图像，匹配该帧目标图像中的特征点以及所述全局SLAM地图中的地图点；
    根据相互匹配的特征点和地图点确定该帧目标图像对应的所述目标摄像模组在所述全局SLAM地图中的摄像模组位姿，以得到所述目标摄像模组的第二摄像模组位姿序列；
    通过所述车辆的定位模块获取与所述第二摄像模组位姿序列对应的所述车辆的第二车辆位姿序列；
    基于多帧所述目标图像以及所述第二摄像模组位姿序列确定所述目标摄像模组在所述全局SLAM地图中的重定位位姿，以使利用所述目标摄像模组的所述重定位位姿将所述第二车辆位姿序列转换到所述全局SLAM地图的SLAM地图坐标系时，转换得到的第三车辆位姿序列与所述第二摄像模组位姿序 列的误差最小；所述全局SLAM地图坐标系的坐标原点为所述目标摄像模组拍摄到所述初始帧时的光心位置；
    根据所述目标摄像模组在所述全局SLAM地图中的重定位位姿，确定所述车辆在所述全局SLAM地图中的重定位位姿。
  5. 根据权利要求4所述的方法，其特征在于，在所述根据所述目标摄像模组在所述SLAM地图中的重定位位姿，确定所述车辆在所述SLAM地图中的重定位位姿之后，所述方法还包括：
    获取当前时刻所述车辆的定位模块测量到的所述车辆的当前位姿；
    根据所述当前位姿以及所述车辆在所述全局SLAM地图中的重定位位姿，确定所述当前位姿在所述全局SLAM地图中的定位位姿作为当前时刻所述车辆的定位结果。
  6. 一种车辆的SLAM建图系统，其特征在于，包括：
    初始化单元，用于获取车辆的任意两个摄像模组在所述车辆静止时分别拍摄得到的两个初始帧；以及，根据所述两个摄像模组预先标定的内部参数和外部参数，以及相互匹配的特征点在所述两个初始帧中的视差，确定所述相互匹配的特征点的三维空间位置，以得到所述相互匹配的特征点对应的地图点，构建出初始的SLAM地图，以完成初始化；其中，所述两个摄像模组的取景范围至少存在部分重叠；
    建图单元，用于在初始化成功之后，获取所述两个摄像模组中的任意一个目标摄像模组拍摄到的图像；以及，以所述初始的SLAM地图为基础，结合所述目标摄像模组拍摄到的图像进行单目视觉SLAM的建图。
  7. 根据权利要求6所述的系统，其特征在于，所述初始化单元用于获取车辆的任意两个摄像模组在所述车辆静止时分别拍摄得到的两个初始帧的方式具体包括：
    所述初始化单元，用于获取车辆的任意两个摄像模组在所述车辆静止时分别进行拍摄得到的第一图像和第二图像；以及，按照使用相同焦距拍摄得到的图像效果对所述第一图像和所述第二图像进行处理，以处理后的所述第一图像和所述第二图像作为初始帧；其中，所述两个摄像模组的焦距不同。
  8. 根据权利要求6或7所述的系统，其特征在于，所述建图单元，包括：
    匹配子单元，用于对所述目标摄像模组拍摄到的图像中的特征点以及当前SLAM地图中的地图点进行匹配；其中，当所述图像为所述目标摄像模组拍摄到的第一帧图像时，所述当前SLAM地图为所述初始的SLAM地图；
    定位子单元，用于根据相互匹配的特征点和地图点确定所述目标摄像模组拍摄到的每帧图像对应的摄像模组位姿，以得到所述目标摄像模组的位姿序列；
    建图子单元，用于跟踪未匹配到地图点的特征点在所述目标摄像模组拍摄到的图像中的位置，并结合所述未匹配到地图点的特征点所在的图像对应的摄像模组位姿，确定所述未匹配到地图点的特征点的三维空间位置，以构建出新的地图点加入至所述当前SLAM地图；
    获取子单元，用于通过所述车辆的定位模块获取与所述目标摄像模组的位姿序列相对应的所述车辆的第一位姿序列；
    优化子单元，用于迭代调整所述目标摄像模组的位姿序列包含的各个摄像模组位姿的取值以及所述当前SLAM地图中的地图点的三维空间位置的取值，直至所述目标摄像模组的位姿序列与所述第一位姿序列之间的投影误差最小；
    所述建图子单元，还用于以所述投影误差最小时所述目标摄像模组的位姿序列包含的各个摄像模组位姿的取值以及所述当前SLAM地图中的地图点的三维空间位置的取值构建全局SLAM地图。
  9. 根据权利要求8所述的系统，其特征在于，所述系统还包括：
    第一获取单元，用于在重定位时，获取所述目标摄像模组拍摄到的多帧图像以及所述全局SLAM地图；
    匹配单元，用于针对所述多帧图像中的每帧图像，匹配该帧图像中的特征点以及所述全局SLAM地图中的地图点；以及，根据相互匹配的特征点和地图点确定该帧图像对应的所述目标摄像模组在所述全局SLAM地图中的摄像模组位姿，以得到所述目标摄像模组的第二摄像模组位姿序列；
    第二获取单元，用于通过所述车辆的定位模块获取与所述第二摄像模组位姿序列对应的所述车辆的第二车辆位姿序列；
    重定位单元，用于基于多帧目标图像以及所述第二摄像模组位姿序列确定所述目标摄像模组在所述全局SLAM地图中的重定位位姿，以使利用所述目标摄像模组的所述重定位位姿将所述第二车辆位姿序列转换到所述全局SLAM地图的SLAM地图坐标系时，转换得到的第三车辆位姿序列与所述第二摄像模组位姿序列的误差最小；以及，根据所述目标摄像模组在所述全局SLAM地图中的重定位位姿，确定所述车辆在所述全局SLAM地图中的重定位位姿；
    其中，所述全局SLAM地图坐标系的坐标原点为所述目标摄像模组拍摄到所述初始帧时的光心位置。
  10. 根据权利要求9所述的系统，其特征在于，所述系统还包括：
    第三获取单元，用于在所述重定位单元根据所述目标摄像模组在所述SLAM地图中的重定位位姿，确定所述车辆在所述SLAM地图中的重定位位姿之后，获取当前时刻所述车辆的定位模块测量到的所述车辆的当前位姿；
    定位单元，用于根据所述当前位姿以及所述车辆在所述全局SLAM地图中的重定位位姿，确定所述当前位姿在所述全局SLAM地图中的定位位姿作为当前时刻所述车辆的定位结果。
  11. 一种车辆，其特征在于，所述车辆包括权利要求6-10任一项所述的车辆的SLAM建图系统。
PCT/CN2019/093537 2019-02-22 2019-06-28 一种车辆的slam建图方法及系统 WO2020168668A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP19915759.5A EP3886053B1 (en) 2019-02-22 2019-06-28 Slam mapping method and system for vehicle

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910136692.5A CN109887087B (zh) 2019-02-22 2019-02-22 一种车辆的slam建图方法及系统
CN201910136692.5 2019-02-22

Publications (1)

Publication Number Publication Date
WO2020168668A1 true WO2020168668A1 (zh) 2020-08-27

Family

ID=66929173

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/093537 WO2020168668A1 (zh) 2019-02-22 2019-06-28 一种车辆的slam建图方法及系统

Country Status (3)

Country Link
EP (1) EP3886053B1 (zh)
CN (1) CN109887087B (zh)
WO (1) WO2020168668A1 (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111950524A (zh) * 2020-08-28 2020-11-17 广东省现代农业装备研究所 一种基于双目视觉和rtk的果园局部稀疏建图方法和系统
CN112734841A (zh) * 2020-12-31 2021-04-30 华南理工大学 一种用轮式里程计-imu和单目相机实现定位的方法
CN113095184A (zh) * 2021-03-31 2021-07-09 上海商汤临港智能科技有限公司 定位方法、行驶控制方法、装置、计算机设备及存储介质
CN113435392A (zh) * 2021-07-09 2021-09-24 阿波罗智能技术(北京)有限公司 应用于自动泊车的车辆定位方法、装置及车辆
CN113610932A (zh) * 2021-08-20 2021-11-05 苏州智加科技有限公司 双目相机外参标定方法和装置
CN113723568A (zh) * 2021-09-30 2021-11-30 中国电子科技集团公司第五十四研究所 基于多传感器和海平面的遥感图像特征点高程获取方法
CN114399547A (zh) * 2021-12-09 2022-04-26 中国人民解放军国防科技大学 一种基于多帧的单目slam鲁棒初始化方法

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109887087B (zh) * 2019-02-22 2021-02-19 广州小鹏汽车科技有限公司 一种车辆的slam建图方法及系统
CN112097742B (zh) * 2019-06-17 2022-08-30 北京地平线机器人技术研发有限公司 一种位姿确定方法及装置
CN112261351A (zh) * 2019-07-22 2021-01-22 比亚迪股份有限公司 车载景观系统及车辆
CN112530270B (zh) * 2019-09-17 2023-03-14 北京初速度科技有限公司 一种基于区域分配的建图方法及装置
CN111091621A (zh) * 2019-12-11 2020-05-01 东南数字经济发展研究院 双目视觉的同步定位与构图方法、装置、设备及存储介质
CN111240321B (zh) * 2020-01-08 2023-05-12 广州小鹏汽车科技有限公司 基于slam地图的高频定位方法及车辆控制系统
CN112597787B (zh) * 2020-08-27 2021-10-15 禾多科技（北京）有限公司 用于融合局部图像的方法、装置、服务器和介质
CN112767480A (zh) * 2021-01-19 2021-05-07 中国科学技术大学 一种基于深度学习的单目视觉slam定位方法
JP2022137532A (ja) * 2021-03-09 2022-09-22 本田技研工業株式会社 地図生成装置および位置認識装置
CN113030960B (zh) * 2021-04-06 2023-07-04 陕西国防工业职业技术学院 一种基于单目视觉slam的车辆定位方法
CN112801077B (zh) * 2021-04-15 2021-11-05 智道网联科技(北京)有限公司 用于自动驾驶车辆的slam初始化的方法及相关装置
CN113516692B (zh) * 2021-05-18 2024-07-19 上海汽车集团股份有限公司 一种多传感器融合的slam方法和装置
CN113793417A (zh) * 2021-09-24 2021-12-14 东北林业大学 一种可创建大规模地图的单眼slam方法
WO2023070441A1 (zh) * 2021-10-28 2023-05-04 深圳市大疆创新科技有限公司 可移动平台的定位方法和装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017080451A1 (en) * 2015-11-11 2017-05-18 Zhejiang Dahua Technology Co., Ltd. Methods and systems for binocular stereo vision
CN107167826A (zh) * 2017-03-31 2017-09-15 武汉光庭科技有限公司 一种自动驾驶中基于可变网格的图像特征检测的车辆纵向定位系统及方法
CN107990899A (zh) * 2017-11-22 2018-05-04 驭势科技（北京）有限公司 一种基于slam的定位方法和系统
CN108401461A (zh) * 2017-12-29 2018-08-14 深圳前海达闼云端智能科技有限公司 三维建图方法、装置、系统、云端平台、电子设备和计算机程序产品
CN109887087A (zh) * 2019-02-22 2019-06-14 广州小鹏汽车科技有限公司 一种车辆的slam建图方法及系统

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013099169A1 (ja) * 2011-12-27 2013-07-04 パナソニック株式会社 ステレオ撮影装置
WO2015165523A1 (en) * 2014-04-30 2015-11-05 Longsand Limited Simultaneous localization and mapping initialization
CN104217439B (zh) * 2014-09-26 2017-04-19 南京工程学院 一种室内视觉定位系统及方法
CN105869136A (zh) * 2015-01-22 2016-08-17 北京雷动云合智能技术有限公司 一种基于多摄像机的协作式视觉slam方法
CN105469405B (zh) * 2015-11-26 2018-08-03 清华大学 基于视觉测程的同时定位与地图构建方法
US9807365B2 (en) * 2015-12-08 2017-10-31 Mitsubishi Electric Research Laboratories, Inc. System and method for hybrid simultaneous localization and mapping of 2D and 3D data acquired by sensors from a 3D scene
CN108307179A (zh) * 2016-08-30 2018-07-20 姜汉龙 一种3d立体成像的方法
CN106384382A (zh) * 2016-09-05 2017-02-08 山东省科学院海洋仪器仪表研究所 一种基于双目立体视觉的三维重建系统及其方法
CN106570913B (zh) * 2016-11-04 2019-12-13 上海玄彩美科网络科技有限公司 基于特征的单目slam快速初始化方法
US20180190014A1 (en) * 2017-01-03 2018-07-05 Honeywell International Inc. Collaborative multi sensor system for site exploitation
CN108803591A (zh) * 2017-05-02 2018-11-13 北京米文动力科技有限公司 一种地图生成方法及机器人
CN108257161B (zh) * 2018-01-16 2021-09-10 重庆邮电大学 基于多相机的车辆环境三维重构和运动估计系统及方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017080451A1 (en) * 2015-11-11 2017-05-18 Zhejiang Dahua Technology Co., Ltd. Methods and systems for binocular stereo vision
CN107167826A (zh) * 2017-03-31 2017-09-15 武汉光庭科技有限公司 一种自动驾驶中基于可变网格的图像特征检测的车辆纵向定位系统及方法
CN107990899A (zh) * 2017-11-22 2018-05-04 驭势科技（北京）有限公司 一种基于slam的定位方法和系统
CN108401461A (zh) * 2017-12-29 2018-08-14 深圳前海达闼云端智能科技有限公司 三维建图方法、装置、系统、云端平台、电子设备和计算机程序产品
CN109887087A (zh) * 2019-02-22 2019-06-14 广州小鹏汽车科技有限公司 一种车辆的slam建图方法及系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3886053A4 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111950524A (zh) * 2020-08-28 2020-11-17 广东省现代农业装备研究所 一种基于双目视觉和rtk的果园局部稀疏建图方法和系统
CN111950524B (zh) * 2020-08-28 2024-03-29 广东省现代农业装备研究所 一种基于双目视觉和rtk的果园局部稀疏建图方法和系统
CN112734841A (zh) * 2020-12-31 2021-04-30 华南理工大学 一种用轮式里程计-imu和单目相机实现定位的方法
CN113095184A (zh) * 2021-03-31 2021-07-09 上海商汤临港智能科技有限公司 定位方法、行驶控制方法、装置、计算机设备及存储介质
CN113435392A (zh) * 2021-07-09 2021-09-24 阿波罗智能技术(北京)有限公司 应用于自动泊车的车辆定位方法、装置及车辆
CN113610932A (zh) * 2021-08-20 2021-11-05 苏州智加科技有限公司 双目相机外参标定方法和装置
CN113610932B (zh) * 2021-08-20 2024-06-04 苏州智加科技有限公司 双目相机外参标定方法和装置
CN113723568A (zh) * 2021-09-30 2021-11-30 中国电子科技集团公司第五十四研究所 基于多传感器和海平面的遥感图像特征点高程获取方法
CN114399547A (zh) * 2021-12-09 2022-04-26 中国人民解放军国防科技大学 一种基于多帧的单目slam鲁棒初始化方法
CN114399547B (zh) * 2021-12-09 2024-01-02 中国人民解放军国防科技大学 一种基于多帧的单目slam鲁棒初始化方法

Also Published As

Publication number Publication date
EP3886053A1 (en) 2021-09-29
CN109887087A (zh) 2019-06-14
CN109887087B (zh) 2021-02-19
EP3886053A4 (en) 2022-07-13
EP3886053B1 (en) 2024-09-11

Similar Documents

Publication Publication Date Title
WO2020168668A1 (zh) 一种车辆的slam建图方法及系统
CN109887032B (zh) 一种基于单目视觉slam的车辆定位方法及系统
CN106780601B (zh) 一种空间位置追踪方法、装置及智能设备
KR102516326B1 (ko) 이미지 라인들로부터의 카메라 외부 파라미터들 추정
JP2019536170A (ja) 仮想的に拡張された視覚的同時位置特定及びマッピングのシステム及び方法
JP6522630B2 (ja) 車両の周辺部を表示するための方法と装置、並びに、ドライバー・アシスタント・システム
US11788845B2 (en) Systems and methods for robust self-relocalization in a visual map
US11216979B2 (en) Dual model for fisheye lens distortion and an algorithm for calibrating model parameters
TW202036480A (zh) 影像定位方法及其系統
WO2022141294A1 (zh) 仿真测试方法、系统、仿真器、存储介质及程序产品
CN110458104B (zh) 人眼视线检测系统的人眼视线方向确定方法和系统
CN110119189B (zh) Slam系统的初始化、ar控制方法、装置和系统
CN110120098A (zh) 场景尺度估计及增强现实控制方法、装置和电子设备
CN114638897A (zh) 基于无重叠视域的多相机系统的初始化方法、系统及装置
CN114529800A (zh) 一种旋翼无人机避障方法、系统、装置及介质
CN111862225A (zh) 图像标定方法、标定系统以及具有该系统的车辆
WO2020014941A1 (zh) 一种建立地图的方法、定位方法、装置、终端及存储介质
US10891805B2 (en) 3D model establishing device and calibration method applying to the same
CN117115271A (zh) 无人机飞行过程中的双目相机外参数自标定方法及系统
CN111089579B (zh) 异构双目slam方法、装置及电子设备
US20210256732A1 (en) Image processing method and unmanned aerial vehicle
WO2021217444A1 (zh) 深度图生成方法、电子设备、计算处理设备及存储介质
KR102298047B1 (ko) 디지털 콘텐츠를 녹화하여 3d 영상을 생성하는 방법 및 장치
CN112399068B (zh) 一种图像处理系统和图像处理方法
WO2023188046A1 (ja) 画像処理装置、画像処理方法、及び画像処理プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19915759

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019915759

Country of ref document: EP

Effective date: 20210623

NENP Non-entry into the national phase

Ref country code: DE