WO2022077296A1 - Procédé de reconstruction tridimensionnelle, charge de cardan, plate-forme amovible et support de stockage lisible par ordinateur - Google Patents

Procédé de reconstruction tridimensionnelle, charge de cardan, plate-forme amovible et support de stockage lisible par ordinateur Download PDF

Info

Publication number
WO2022077296A1
WO2022077296A1 PCT/CN2020/120978 CN2020120978W WO2022077296A1 WO 2022077296 A1 WO2022077296 A1 WO 2022077296A1 CN 2020120978 W CN2020120978 W CN 2020120978W WO 2022077296 A1 WO2022077296 A1 WO 2022077296A1
Authority
WO
WIPO (PCT)
Prior art keywords
pose
dimensional point
image
acquisition device
point clouds
Prior art date
Application number
PCT/CN2020/120978
Other languages
English (en)
Chinese (zh)
Inventor
徐骥飞
杜劼熹
上官政和
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2020/120978 priority Critical patent/WO2022077296A1/fr
Publication of WO2022077296A1 publication Critical patent/WO2022077296A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05—Geographic models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/38—Registration of image sequences

Definitions

  • the present application relates to the technical field of three-dimensional reconstruction, and in particular, to a three-dimensional reconstruction method, a PTZ load, a movable platform, and a computer-readable storage medium.
  • Simultaneous Localization and Mapping is a technology for robots to estimate their own motion and build a map of the surrounding environment in an unknown environment. It has a wide range of applications in drones, autonomous driving, mobile robot navigation, virtual reality and augmented reality.
  • the positioning under the global map can be achieved through high-precision GPS signals and prior maps.
  • the SLAM method is usually implemented based on the camera. This method uses images for 3D reconstruction, and uses the 3D reconstructed map to perceive its own motion and the surrounding environment, but from a single image Therefore, this method has the problems of low positioning accuracy and low quality of 3D model construction.
  • one of the objectives of the present application is to provide a three-dimensional reconstruction method, a pan-tilt load, a movable platform, and a computer-readable storage medium.
  • an embodiment of the present application provides a three-dimensional reconstruction method, which is applied to a pan-tilt load, and the pan-tilt payload is provided with a first image acquisition device and a second image acquisition device, and the method includes:
  • the motion data includes: during the acquisition of the three-dimensional point cloud and image sequence of the target scene, the pose information of the load on the PTZ;
  • the three-dimensional point cloud is registered according to the motion data to obtain the relative pose between the three-dimensional point clouds; and the image sequence is registered to obtain the relative pose between the images;
  • the relative pose between the three-dimensional point clouds, and the relative pose between the images obtain the pose and/or the first image acquisition device when acquiring the three-dimensional point cloud. 2. the pose when the image acquisition device acquires the image;
  • a 3D model of the target scene is generated according to the 3D point cloud, the sequence of images, and the pose when the first image acquisition device acquires the 3D point cloud and/or the pose when the second image acquisition device acquires the image .
  • an embodiment of the present application provides a pan-tilt load, including a first image acquisition device, a second image acquisition device, an inertial measurement unit, a memory for storing executable instructions, and a processor;
  • the first image acquisition device is used to acquire a three-dimensional point cloud of the target scene
  • the second image acquisition device is used for acquiring the image sequence of the target scene
  • the inertial measurement unit is used to obtain motion data of the gimbal load;
  • the motion data includes the pose information of the gimbal load;
  • the processor executes the executable instructions, it is configured to:
  • the three-dimensional point cloud is registered according to the motion data to obtain the relative pose between the three-dimensional point clouds; and the image sequence is registered to obtain the relative pose between the images;
  • the relative pose between the three-dimensional point clouds, and the relative pose between the images obtain the pose and/or the first image acquisition device when acquiring the three-dimensional point cloud. 2. the pose when the image acquisition device acquires the image;
  • a 3D model of the target scene is generated according to the 3D point cloud, the sequence of images, and the pose when the first image acquisition device acquires the 3D point cloud and/or the pose when the second image acquisition device acquires the image .
  • an embodiment of the present application provides a movable platform, including the PTZ load described in the second aspect.
  • an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the implementation is as described in the first aspect Methods.
  • a three-dimensional reconstruction method, a pan-tilt load, a movable platform, and a computer-readable storage medium acquire a three-dimensional point cloud and an image sequence of a target scene by adopting a multi-sensor method;
  • the motion data of the gimbal load is obtained, and the motion data corresponds to the pose information of the gimbal load;
  • the three-dimensional point cloud is registered according to the motion data , obtain the relative pose between the three-dimensional point clouds; and, register the image sequence to obtain the relative pose between the images; then play according to the motion data, the relative position between the three-dimensional point clouds pose and the relative pose between the images, obtain the pose when the first image acquisition device acquires the three-dimensional point cloud and/or the pose when the second image acquisition device acquires the image; finally, according to the three-dimensional point cloud
  • the three-dimensional point cloud can indicate the three-dimensional information of the target scene
  • the The motion data includes the pose information of the pan-tilt load
  • the first image acquisition device and the second image acquisition device are arranged on the pan-tilt payload, so the motion data can indirectly indicate the first image acquisition
  • the pose information of the device and the second image acquisition device is obtained, thereby obtaining high-precision pose information, so that the three-dimensional model generated based on the high-precision pose information has higher positioning accuracy and robustness.
  • FIG. 1 is a schematic structural diagram of a pan-tilt load provided by an embodiment of the present application.
  • FIG. 2 is an application scenario diagram provided by an embodiment of the present application
  • FIG. 3 is a schematic flowchart of a three-dimensional reconstruction method provided by an embodiment of the present application.
  • FIGS. 4 , 5 , 6 and 7 are schematic diagrams of different structures of a pose graph model provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of another pan-tilt load provided by an embodiment of the present application.
  • Simultaneous Localization and Mapping is a technology for robots to estimate their own motion and build a map of the surrounding environment in an unknown environment. It has a wide range of applications in drones, autonomous driving, mobile robot navigation, virtual reality and augmented reality.
  • the positioning under the global map can be achieved through high-precision GPS signals and prior maps.
  • the SLAM method is usually implemented based on the camera. This method uses images for 3D reconstruction, and uses the 3D reconstructed map to perceive its own motion and the surrounding environment, but from a single image The information obtained is limited, therefore, this method has the problems of low positioning accuracy and low quality of 3D model construction.
  • the embodiments of the present application provide a three-dimensional reconstruction method and a gimbal load.
  • the gimbal load integrates a plurality of different sensors, and obtains a three-dimensional point cloud and an image sequence of a target scene by adopting a multi-sensor method; and , during the acquisition of the three-dimensional point cloud and the image sequence, acquire motion data of the gimbal load, where the motion data includes the pose information of the gimbal load;
  • the point clouds are registered to obtain the relative pose between the three-dimensional point clouds; and the image sequence is registered to obtain the relative pose between the images; then according to the motion data, the three-dimensional point cloud and the relative pose between the images, obtain the pose when the first image acquisition device acquires the three-dimensional point cloud and/or the second image acquisition device when the image is acquired; finally
  • a 3D model of the target scene is generated according to the 3D point cloud, the sequence of images, and the pose when the first image acquisition device acquires the 3D point cloud and/or the pose when the second image
  • the three-dimensional reconstruction method can be applied to a pan-tilt load, where a plurality of different sensors are installed.
  • FIG. 1 which provides a schematic structural diagram of a pan-tilt load.
  • the first image acquisition device 21 is used to acquire the three-dimensional point cloud of the target scene in real time
  • the second image acquisition device 22 is used for to acquire the image sequence of the target scene in real time
  • the inertial measurement unit 23 is used to acquire the motion data of the PTZ load in real time
  • the PTZ load can be based on the real-time acquisition of the three-dimensional point cloud, the image
  • the sequence and the motion data generate a three-dimensional model in real time, so that the real-time requirements can be met while the accuracy requirements are met, so that subsequent positioning or measurement can be performed directly based on the real-time generated three-dimensional model.
  • the inertial measurement unit is used to measure the three-axis attitude angle (or angular rate) and acceleration of the gimbal load.
  • an inertial measurement unit includes three single-axis accelerometers and three single-axis gyroscopes.
  • the accelerometer detects the acceleration signal of the gimbal load
  • the gyro detects the angular velocity signal of the gimbal load, and measures the angular velocity signal of the gimbal load.
  • the angular velocity and acceleration of the gimbal load in the three-dimensional space are calculated, and the attitude of the gimbal load is calculated.
  • the first image acquisition device may include at least one or more of the following: lidar, binocular vision sensor, and structured light depth camera.
  • the lidar is used to transmit a laser pulse sequence to the target scene, then receive the laser pulse sequence reflected from the target, and generate a three-dimensional point cloud according to the reflected laser pulse sequence.
  • the lidar can determine the reception time of the reflected laser pulse sequence, eg, by detecting the rising edge time and/or the falling edge time of the electrical signal pulse to determine the reception time of the laser pulse sequence.
  • the laser radar can calculate TOF (Time of flight, time of flight) by using the receiving time information and the transmitting time of the laser pulse sequence, so as to determine the distance from the detected object to the laser radar.
  • the lidar is a self-illuminating sensor, does not depend on light source illumination, is less disturbed by ambient light, and can work normally even in a closed environment without light, so that a high-precision three-dimensional model can be generated later, and it has a wide range of applicability.
  • the binocular vision sensor obtains two images of the target scene from different positions based on the principle of parallax, and obtains three-dimensional geometric information by calculating the positional deviation between corresponding points of the two images, thereby generating a three-dimensional point cloud.
  • the binocular vision sensor has low hardware requirements, and correspondingly, it can also reduce the cost. It only needs to be an ordinary CMOS (Complementary Metal Oxide Semiconductor) camera. As long as the light is suitable, both indoor and outdoor environments can be used. use, so it also has certain applicability.
  • CMOS Complementary Metal Oxide Semiconductor
  • the structured light depth camera projects light with certain structural characteristics into the target scene and then collects it. This kind of light with certain structure will collect different image phase information due to different depth regions of the subject, and then It is converted into depth information to obtain a 3D point cloud.
  • Structured light depth camera is also a self-illuminating sensor, does not depend on light source illumination, and is less disturbed by ambient light, and can work normally even in a closed environment without light, so as to generate high-precision 3D models later, which has a wide range of applicability .
  • the second image acquisition device can acquire color images, grayscale images, infrared images, and the like.
  • the second image acquisition device includes at least one or more of the following: a visible light camera, a grayscale camera, and an infrared camera.
  • the second image acquisition device may capture a sequence of images at a specified frame rate.
  • the sequence of images may be captured at a standard frame rate such as about 24p, 25p, 30p, 48p, 50p, 60p, 72p, 90p, 100p, 120p, 300p, 50i or 60i.
  • Capture image sequences at frame rates of one image per second, 2 seconds, 5 seconds, or 10 seconds.
  • the second image capture device may have adjustable capture parameters. Under different capture parameters, the second image capture device may capture different images despite being subjected to exactly the same external conditions (eg location, lighting). Capture parameters may include exposure (eg, exposure time, shutter speed, aperture, film speed), gain, gamma, region of interest, binning/subsampling, pixel clock, offset, trigger, ISO, and the like. Exposure-related parameters may control the amount of light reaching the image sensor in the second image capture device. For example, shutter speed can control the amount of time light reaches the image sensor and aperture can control the amount of light that reaches the image sensor in a given time. A gain-related parameter can control the amplification of the signal from the optical sensor. ISO controls the level of sensitivity of the camera to the available light.
  • Exposure-related parameters may control the amount of light reaching the image sensor in the second image capture device. For example, shutter speed can control the amount of time light reaches the image sensor and aperture can control the amount of light that reaches the image sensor in a given time.
  • a lidar, a visible light camera and an inertial measurement unit are installed on the pan/tilt load.
  • the lidar, the visible light camera, and the inertial measurement unit may operate at the same frame rate.
  • the lidar, the visible light camera and the inertial measurement unit may also work at different frame rates, the frame rates of the lidar and the visible light camera and the inertial measurement unit.
  • the pan-tilt payload can be mounted on a movable platform, and the pan-tilt payload can be carried by the movable platform to move in a target scene, so as to perform three-dimensional reconstruction for the target scene, which is convenient for follow-up.
  • the three-dimensional model obtained by the three-dimensional reconstruction can be used for positioning tasks, mapping tasks, or navigation tasks, and the like.
  • the movable platform includes, but is not limited to, an unmanned aerial vehicle, an unmanned vehicle, or a mobile robot.
  • the PTZ load can be connected to the remote control terminal in communication, and the user can control the PTZ load through the remote control terminal to collect data in real time and perform 3D reconstruction in real time, and transmit the 3D model obtained by the 3D reconstruction to the remote control terminal in real time.
  • the remote control terminal is displayed on the remote control terminal, so that the user can know the progress of the three-dimensional reconstruction in real time, which is convenient for the user to use; further, the user can also input feedback information for the three-dimensional model displayed on the remote control terminal.
  • the feedback information is used to indicate the position where the first image acquisition device missed the acquisition, and the remote control terminal transmits the feedback information to the PTZ load, so that the PTZ load can control the first image acquisition device
  • the 3D point cloud at the position indicated by the feedback information in the target scene is collected to realize the online monitoring process, to ensure accurate and effective acquisition of 3D point cloud data, and to improve the efficiency of 3D reconstruction.
  • the gimbal load 20 can be mounted on the unmanned aerial vehicle 10, and the gimbal load 20 and the unmanned aerial vehicle 10 are connected in communication with the remote control terminal 30, The user can control the UAV 10 and the gimbal load 20 through the remote control terminal 30.
  • the remote control terminal 30 can respond to the user's trigger operation and send a three-dimensional reconstruction instruction to the gimbal load 20, so the
  • the pan-tilt load 20 is installed with a first image acquisition device, a second image acquisition device and an inertial measurement unit; the first image acquisition device acquires the three-dimensional point cloud of the target scene in real time in response to the three-dimensional reconstruction instruction;
  • the second image acquisition device acquires the image sequence of the target scene in real time in response to the three-dimensional reconstruction instruction;
  • the inertial measurement unit acquires the motion data of the pan-tilt load 20 in real time in response to the three-dimensional reconstruction instruction, so the
  • the motion data includes: during the acquisition of the three-dimensional point cloud and image sequence of the target scene, the pose information of the gimbal load; then the gimbal load 20 can be based on the real-time acquisition of the three-dimensional point cloud, the image
  • the sequence and the motion data generate a three-dimensional model in real time, so as to meet the real-time
  • the three-dimensional model is described so that the user can know the progress of the three-dimensional reconstruction in real time, which is convenient for the user to use; the user can also control the flight of the unmanned aerial vehicle 10 through the remote control terminal 30, so that the unmanned aerial vehicle 10 can carry the gimbal load 20 and fly to the desired location.
  • different positions of the target scene so that the pan-tilt load 20 can perform three-dimensional reconstruction on different positions of the target scene, and obtain a complete three-dimensional map in the target scene, so that the unmanned aerial vehicle can be based on the 3D map for positioning tasks, measurement tasks or cruise tasks, etc.
  • FIG. 3 is a schematic flowchart of a three-dimensional reconstruction method provided by an embodiment of the present application.
  • the method is applied to a PTZ load, and the method includes:
  • step S101 a 3D point cloud of a target scene, an image sequence, and motion data loaded by the gimbal are acquired; wherein, the 3D point cloud is acquired by the first image acquisition device, and the image sequence is acquired by the first image acquisition device.
  • the motion data includes: during the acquisition of a three-dimensional point cloud and an image sequence of a target scene, the pose information of the PTZ load.
  • step S102 the three-dimensional point cloud is registered according to the motion data to obtain the relative pose between the three-dimensional point clouds; and the image sequence is registered to obtain the relative pose between the images .
  • step S103 according to the motion data, the relative pose between the three-dimensional point clouds, and the relative pose between the images, the pose and the pose when the first image acquisition device acquires the three-dimensional point cloud is acquired /or the pose when the second image acquisition device acquires an image.
  • step S104 according to the three-dimensional point cloud, the image sequence, and the pose when the first image acquisition device acquires the three-dimensional point cloud and/or the pose when the second image acquisition device acquires the image, generate 3D model of the target scene.
  • the target scene may be a scene where the satellite positioning signal is lower than a predetermined intensity
  • the target scene may be an indoor scene or an outdoor non-open scene (such as underwater, a mine, etc.), in this kind of scene, you can use
  • the three-dimensional reconstruction method of this embodiment generates a high-precision three-dimensional model.
  • the satellite positioning signals include, but are not limited to, GPS signals, signals from the Galileo Satellite Navigation System (GALILEO), or signals from the Beidou Satellite Navigation System (BDS).
  • GALILEO Galileo Satellite Navigation System
  • BDS Beidou Satellite Navigation System
  • the strength of the satellite positioning signal may be determined by at least one of the following parameters: the signal-to-noise ratio of the satellite positioning signal, the cold start time or the warm start time of the satellite positioning signal; in an exemplary embodiment , if the signal-to-noise ratio of the satellite positioning signal is lower than the preset threshold, it indicates that the strength of the satellite positioning signal is lower than the preset strength; in another exemplary embodiment, if the signal-to-noise ratio of the satellite positioning signal If the ratio is lower than the preset threshold and the cold start time of the satellite positioning signal does not meet the preset time condition, it indicates that the strength of the satellite positioning signal is lower than the preset strength.
  • the pan-tilt load can use a depth camera to acquire a 3D point cloud of the target scene in real time, and use a second image acquisition device to acquire an image sequence of the target scene in real time; and during acquisition of the 3D point cloud and the image sequence, Using an inertial measurement unit to collect motion data of the gimbal load in real time, the motion data corresponds to the pose information of the gimbal load.
  • This embodiment uses the three-dimensional point cloud, the image sequence, and the motion data acquired in real time to perform real-time three-dimensional reconstruction.
  • the motion data can indicate The pose information of the pan-tilt load, the first image acquisition device and the second image acquisition device are set on the pan-tilt payload, so the motion data can indirectly indicate the first image acquisition device and the The pose information of the second image acquisition device is obtained to obtain high-precision pose information, so that the three-dimensional model generated based on the high-precision pose information has higher positioning accuracy and robustness; and real-time acquisition and real-time three-dimensional reconstruction The process is also enough to meet the needs of the scene with real-time requirements, so that the subsequent positioning or measurement can be directly based on the real-time generated 3D model.
  • the pan-tilt load can be connected to the remote control terminal in communication, and the pan-tilt load can generate a collection trajectory according to the accuracy requirements and/or density requirements of the three-dimensional model input by the user on the remote control terminal. and acquisition time, and then control the first image acquisition device, the second image acquisition device and the inertial measurement unit to acquire data according to the acquisition track and the acquisition time.
  • This embodiment implements automatic planning of the acquisition scheme based on the accuracy requirements and/or density requirements of the three-dimensional model, and controls the first image acquisition device, the second image acquisition device, and the inertial measurement unit to acquire data. There is no need to participate in the process, which can effectively prevent human errors introduced in the acquisition process and improve the accuracy of the generated 3D model.
  • the PTZ load After acquiring the 3D point cloud, the image sequence and the motion data, the PTZ load registers the 3D point cloud according to the motion data, and obtains the relative pose between the 3D point clouds, The relative poses between the three-dimensional point clouds are used to calibrate the three-dimensional point clouds, thereby improving the accuracy of subsequent three-dimensional reconstruction results; The relative pose between the images is used to determine the pose of the second image acquisition device, thereby improving the accuracy of subsequent three-dimensional reconstruction results.
  • each point in the 3D point cloud has an independent coordinate system, so as to ensure the accuracy of the 3D coordinates of the point in the corresponding coordinate system.
  • the 3D point cloud when used for 3D reconstruction in the future, the points in the 3D point cloud need to be in the same coordinate system, so the 3D point cloud needs to be reprojected, the motion of the lidar during the acquisition process is calculated, and the This amount of movement is compensated on the corresponding laser point.
  • the three-dimensional point cloud is transformed by the motion data obtained by the inertial measurement unit installed on the same pan/tilt load as the first image acquisition device, and the motion data obtained by the inertial measurement unit corresponds to the cloud
  • the motion data acquired by the inertial measurement unit also indicates the motion process of the first image acquisition device, so the motion data measured by the inertial measurement unit can be used to accurately determine the transformation between points in different coordinate systems
  • the transformation relationship between the points can be obtained by using the motion data and the external parameter transformation relationship between the inertial measurement unit and the first image acquisition device.
  • the PTZ load obtains the transformation relationship between points according to the motion data corresponding to each point in the group of 3D point clouds, and then according to the relationship between the points The transformation relationship reprojects the points in the set of 3D point clouds to the same coordinate system.
  • a group of three-dimensional point clouds collected in a continuous time series are transformed into the same coordinate system, so as to realize the integration of three-dimensional point cloud data, which is beneficial to improve the efficiency of three-dimensional reconstruction.
  • the PTZ load acquires the motion data corresponding to the point according to the acquisition time of each point;
  • the first image acquisition device is a laser radar, and the acquisition time of the point is based on the reflected laser pulses Determined by the receiving time of the sequence, the acquisition time of the point is the acquisition time of the corresponding motion data, and the gimbal load acquires the motion data from the inertial measurement unit at the same moment when the point is acquired.
  • the first image acquisition device will acquire each point at a preset time interval, and when acquiring the transformation relationship between points, the corresponding motion of each point in the set of point clouds can be obtained.
  • the data determines the relative pose of the inertial measurement unit within the preset time interval, wherein the preset time interval is the time interval for acquiring adjacent points; then the gimbal load is based on the inertial measurement unit at the time interval.
  • the relative pose within a preset time interval, and the extrinsic parameter transformation relationship between the first image acquisition device and the inertial measurement unit obtain the transformation relationship between the points.
  • the motion data acquired by the inertial measurement unit also indicates the movement process of the first image acquisition device, so it can be Using the motion data measured by the inertial measurement unit, the transformation relationship between points in different coordinate systems can be accurately determined, so that a set of three-dimensional point clouds collected in a continuous time series can be transformed based on the transformation relationship between the points. Under the same coordinate system, the integration of 3D point cloud data is realized, which is beneficial to improve the efficiency of 3D reconstruction.
  • the gimbal load can obtain the relative poses between the adjacent two groups of 3D point clouds, and the relative poses between the adjacent two groups of 3D point clouds are used to
  • the two sets of 3D point clouds are reprojected to the same coordinate system to realize the matching and integration of 3D point cloud data, thus facilitating the subsequent 3D reconstruction process.
  • the adjacent two sets of three-dimensional point clouds include points collected for the same object, so as to ensure the smooth matching of the two adjacent three-dimensional point clouds.
  • the PTZ load acquires the surface features and/or corner features of each group of 3D point clouds. Then, according to the surface features and/or corner features of the adjacent two groups of three-dimensional point clouds, the relative poses between the two adjacent groups of three-dimensional point clouds are obtained; the relative poses between the two adjacent groups of three-dimensional point clouds are used for Reproject two adjacent sets of 3D point clouds to the same coordinate system.
  • This embodiment uses the surface features and/or corner features in the 3D point cloud acquisition space to perform high-speed matching, simplifies the matching process, helps improve the efficiency of acquiring the relative poses between two adjacent groups of 3D point clouds, and shortens the matching time. .
  • ICP Iterative Closest Point/Plane
  • the surface features and/or corner features of the three-dimensional point clouds of the group may be determined according to the curvature information between the midpoints of the three-dimensional point clouds of each group.
  • the surface features of each group of 3D point clouds are determined according to curvature information whose curvature is greater than a preset threshold; the corner features of each group of 3D point clouds are determined according to curvature information whose curvature is less than or equal to a preset threshold determined.
  • the matching process between points is converted into a matching process between surface features and/or corner features of a three-dimensional point cloud in space, which is beneficial to simplify the matching process and realize the two adjacent sets of three-dimensional point clouds.
  • the high-speed matching of features and/or corner features is further conducive to improving the efficiency of obtaining relative poses between two adjacent sets of 3D point clouds, shortening the matching time, and meeting the needs of real-time generation of 3D models.
  • the surface feature includes at least one three-dimensional coordinate information indicating a plane and a normal vector of the plane; and/or the corner feature includes at least one three-dimensional coordinate information indicating an edge and a vector indicating the edge.
  • the pan-tilt load may be based on the adjacent two groups of three-dimensional point clouds.
  • the more similar the features are, and/or the more similar the corner features of the adjacent two sets of 3D point clouds are, indicating that the points in the adjacent two sets of 3D point clouds indicated by the surface feature and/or the corner feature may be matched.
  • the relative poses between the adjacent two groups of three-dimensional point clouds are obtained through the similarity between the surface features and/or the similarity between the corner features, so as to ensure the accuracy of the obtained results.
  • the similarity between the surface features of the adjacent two groups of three-dimensional point clouds is determined based on the distance between the surface features of the two adjacent groups of three-dimensional point clouds; and/or, the The similarity between the corner features of the adjacent two groups of three-dimensional point clouds is determined based on the distance between the corner features of the two adjacent groups of three-dimensional point clouds. Wherein, and/or represent both or one of the two.
  • the optimization algorithm can be used to solve: in, represents the distance difference between the surface features of the adjacent two groups of three-dimensional point clouds, Represents the distance difference between the corner features of the adjacent two groups of three-dimensional point clouds.
  • the relative poses between two adjacent groups of 3D point clouds can be obtained through the transformation and expression of LieAlgebra algorithm.
  • This embodiment uses the surface feature and the corner feature to perform high-speed matching, which is beneficial to improve the matching efficiency and meet the requirement of generating a three-dimensional model in real time.
  • the image sequence includes a plurality of images acquired in a continuous time sequence.
  • the PTZ payload can extract feature points of each of the images, and then perform registration based on the feature points of the images to obtain the relative poses between the images.
  • the PTZ load may use a bundle adjustment (Bundle Adjustment, BA) algorithm to register the feature points of the images, and obtain the relative poses between the images.
  • BA Bunle Adjustment
  • the depth map obtained by the back-projection of the 3D point cloud can be used to assist the registration of the image process to improve image registration accuracy.
  • the PTZ load can reproject the 3D point cloud by using the motion data, and generate a depth map corresponding to the image according to the reprojected 3D point cloud and the motion data; It is mentioned that each point in the 3D point cloud has independent coordinates.
  • each point in the 3D point cloud needs to be in the same coordinate system, so the PTZ load uses The motion data reprojects the 3D point cloud, so that each point in the reprojected 3D point cloud is in the same coordinate system; further, the motion data obtained by the inertial measurement unit corresponds to the PTZ
  • the pose information of the load since the inertial measurement unit, the second image acquisition device and the first image acquisition device are all installed on the PTZ load, the second image acquisition device and the first image acquisition device are The motion process is the same as or similar to the motion process of the PTZ load, so the motion data acquired by the inertial measurement unit can also indicate the motion process of the second image acquisition device and the second image acquisition device, Then, a depth map corresponding to the image may be generated using the reprojected 3D point cloud and the motion data.
  • a polyhedron may be generated based on the reprojected 3D point cloud, and then projected onto the camera plane according to the multi-facet map and the motion data to obtain the image corresponding to the depth map.
  • the PTZ load also extracts the feature points in the image, and obtains the depth information of the feature points from the depth map corresponding to the image; and then according to the feature points of the image and the The depth information is registered to obtain the relative pose between the images.
  • the depth map obtained by the back-projection of the three-dimensional point cloud is used to assist the registration process of the image, which is beneficial to improve the accuracy of image registration, thereby improving the accuracy of the subsequently generated three-dimensional model.
  • the pan/tilt load in the embodiment of the present application uses the motion data to reproject the three-dimensional point cloud , so that each point in the reprojected three-dimensional point cloud is in the same coordinate system, and considering that the inertial measurement unit, the second image acquisition device and the first image acquisition device are all installed on the PTZ load,
  • the motion data obtained by the inertial measurement unit can also indicate the motion process of the second image acquisition device and the second image acquisition device, and then can generate and match the motion data according to the re-projected three-dimensional point cloud and the motion data.
  • the depth map corresponding to the images in the image sequence is obtained; finally, registration is performed according to the feature points extracted from the depth map to obtain the relative pose between the images.
  • the depth map obtained by the back-projection of the three-dimensional point cloud is used to perform registration instead of the image, so as to ensure that an accurate image registration result can be obtained even in the absence of light, thereby improving the accuracy of the subsequently generated three-dimensional model.
  • a distributed accelerator in order to meet the requirement of generating a 3D model online in real time, can be used to obtain the relative posture between the 3D point clouds and the relative posture between the images, so as to speed up the matching process and improve data acquisition. efficiency.
  • the PTZ load can be based on the three-dimensional point cloud, the image sequence, the motion data, the three-dimensional point
  • the relative poses between the clouds and the relative poses between the images generate a three-dimensional model of the target scene in real time, so as to meet the requirement of generating a three-dimensional model online in real time.
  • the PTZ load can transmit the 3D model to the remote control terminal in real time after generating the 3D model, so that the 3D model can be displayed on the remote control terminal in real time, so that the user can know the progress of the 3D reconstruction in real time, which is convenient.
  • the remote control terminal uses the user; if the user finds a missing location after seeing the 3D model, he can also input feedback information for the 3D model displayed on the remote control terminal, where the feedback information is used to instruct the first image acquisition device
  • the remote control terminal transmits the feedback information to the PTZ load, so that the PTZ load can control the first image acquisition device to collect the information indicated by the feedback information in the target scene.
  • the 3D point cloud at the location realizes the online monitoring process, ensures the accurate and effective acquisition of 3D point cloud data, and is conducive to improving the efficiency of 3D reconstruction.
  • the PTZ load is obtained according to the motion data, the relative pose between the 3D point clouds and the relative pose between the images.
  • the pose when the acquisition device acquires the three-dimensional point cloud and/or the pose when the second image acquisition device acquires the image generates a three-dimensional model in real time.
  • This embodiment realizes the generation of a high-precision three-dimensional model based on the fusion of various data.
  • the gimbal load first establishes a pose graph model, and the pose graph model includes but is not limited to a sliding window pose graph model. Then, the pose graph model is optimized by using the motion data, the relative poses between the three-dimensional point clouds, and the relative poses between the images, and the first image acquisition device acquires the three-dimensional point cloud. The pose and/or the pose when the second image acquisition device acquires the image.
  • the motion data obtained by the inertial measurement unit corresponds to the pose information of the gimbal load, since the inertial measurement unit, the second image acquisition device and the first image acquisition device are all installed on the gimbal load,
  • the motion process of the second image acquisition device and the first image acquisition device is the same or similar to the motion process of the gimbal load, so the motion data acquired by the inertial measurement unit can also indicate the first image acquisition device.
  • the motion process of the second image acquisition device and the second image acquisition device therefore, the motion data can be used as one of the optimization factors for optimizing the pose graph model.
  • motion data includes acceleration, which is integrated to obtain velocity, and velocity is integrated to obtain relative displacement. The relative displacement can constrain the displacement between the two positions of the gimbal load.
  • the motion data includes the angular velocity, and the angle is obtained after the angular velocity is integrated, and then the relative attitude information is obtained.
  • the relative attitude can constrain the rotation between the two attitudes of the gimbal load.
  • each vertex is configured to indicate the pose when the first image acquisition device acquires the three-dimensional point cloud and/or the pose when the second image acquisition device acquires an image; and , the edges between the vertices are configured to indicate the relative poses between the three-dimensional point clouds, the relative poses between the images, or the motion data, and each edge constitutes the constraint relationship of each vertex, through The relative pose between the three-dimensional point clouds indicated by each edge, the relative pose between the images, or the motion data are used to optimize the position of the first image acquisition device in each vertex when the three-dimensional point cloud is collected. pose and/or the pose when the second image acquisition device acquires the image.
  • the pose graph model includes a sliding window pose graph model, and in each iterative optimization process of the window, the pose indicated by the ith (i is an integer greater than 1) vertex is selected and the pose indicated by at least one reference vertex (for example, the reference vertex can be the i-jth vertex, and j is an integer greater than 0) as the pose to be iteratively optimized in the window, using the three-dimensional point indicated by the edge between the vertices
  • the relative pose between clouds, the relative pose between the images, or the motion data is used for pose optimization.
  • the window is slid to select the pose indicated by the next vertex for optimization.
  • each vertex instructs the first image acquisition device to acquire a three-dimensional point cloud n is an integer greater than 0, and each edge indicates the relative pose between the three-dimensional point clouds and the relative pose between the images or the motion data; in another example, please refer to FIG.
  • n is an integer greater than 0
  • each edge indicates the relative pose between the three-dimensional point clouds, all The relative pose between the images or the motion data.
  • the PTZ load may also be equipped with a locator, which is used to obtain positioning information such as GPS information and/or RTK information, the locator collects positioning information during the process of collecting the three-dimensional point cloud by the first image collecting device and acquiring the image sequence by the second image collecting device, please refer to FIG.
  • the edges between the vertices can also be configured to indicate GPS information and/or RTK information, so as to obtain high-precision pose information and improve the positioning accuracy of the generated three-dimensional model.
  • the pan-tilt load can utilize the The pose of the three-dimensional point cloud reprojects the three-dimensional point cloud to the target coordinate system, for example, the target coordinate system can be the world coordinate system, and then generates an initial three-dimensional model according to the re-projected three-dimensional point cloud, and then according to the The pose when the second image acquisition device acquires the image and the multiple images in the image sequence perform texture mapping on the initial three-dimensional model to obtain the final three-dimensional model.
  • the PTZ load may add a tag index to the 3D model, and then store the 3D model in the storage location pointed to by the tag index, so that the tag index can be used in subsequent use based on the tag index.
  • the three-dimensional model is found by index; and/or, the three-dimensional model is transmitted to the remote control terminal in real time, so that the three-dimensional model is displayed on the remote control terminal in real time, so that the progress of the three-dimensional reconstruction can be displayed to the user in real time, which is convenient for the user to use.
  • the PTZ load can also perform loop closure detection according to the 3D point cloud or the image sequence, and optimize the pose graph model according to the results of the loop closure detection, thereby optimizing all
  • the 3D model is described to ensure the accuracy and global uniformity of the optimized 3D model.
  • a bag-of-words model may be constructed according to the image sequence or the depth map obtained by re-projection of the three-dimensional point cloud, so that the bag-of-words model can measure the similarity or all of the images in the image sequence.
  • the similarity between the multiple depth images, the pan-tilt load can perform loop closure detection according to the similarity between the multiple images in the image sequence, and determine the image forming a loop; or, according to the three-dimensional point cloud
  • the similarity between the multiple depth maps obtained by reprojection is subjected to loop closure detection to determine the depth map forming a loop closure, and then to determine the image corresponding to the depth map;
  • the pose graph model is globally optimized, that is, the relative poses between the looped images are formed to globally optimize the pose graph model, and the optimized pose information is obtained, that is, the optimized first image acquisition device
  • the pose when collecting the three-dimensional point cloud and/or the pose when the second image acquisition device acquires the image, and then update the three-dimensional model according to the globally optimized pose graph model, that is, update according to the optimized pose information
  • the three-dimensional model ensures the accuracy and global uniformity of the optimized three-dimensional model.
  • the down-sampled global model and the three-dimensional model stored in the first image acquisition device may be used to perform loop closure detection, to determine the relative poses between the three-dimensional point clouds forming loop closures, and then to perform loop closure detection.
  • the gimbal load can globally optimize the pose graph model by using the result of the loopback detection, that is, the relative poses between the three-dimensional point clouds forming the loopback can be used to globally optimize the pose graph model, and the optimized pose graph model can be obtained.
  • the pose information that is, the optimized pose when the first image acquisition device collects the 3D point cloud and/or the pose when the second image acquisition device acquires the image, and then updated according to the globally optimized pose graph model
  • the three-dimensional model is to update the three-dimensional model according to the optimized pose information, so as to ensure the accuracy and global unity of the optimized three-dimensional model.
  • the edges in the pose graph model are also configured to indicate the relative poses between the images forming the loops or the relative poses between the three-dimensional point clouds forming the loops, thereby
  • the pose graph model is optimized, and the three-dimensional model is updated according to the globally optimized pose graph model.
  • the PTZ load can store the updated 3D model in the storage location pointed to by the corresponding label index; and/or, transmit the updated 3D model to the remote control in real time terminal to display the 3D model in real time on the remote control terminal, so that the user can know the progress of the 3D reconstruction in real time, which is convenient for the user to use; if the user finds a missing location after seeing the 3D model, he can also target the display on the remote control terminal.
  • Feedback information is input to the 3D model on the PTZ, and the feedback information is used to indicate the position where the first image acquisition device is missing, and the remote control terminal transmits the feedback information to the PTZ load, so that the PTZ The load can control the first image acquisition device to collect the three-dimensional point cloud at the position indicated by the feedback information in the target scene, realize the online monitoring process, ensure the accurate and effective acquisition of the three-dimensional point cloud data, and help improve the three-dimensional reconstruction. efficiency.
  • an embodiment of the present application further provides a pan-tilt load 20 , including a first image acquisition device 21 , a second image acquisition device 22 , an inertial measurement unit 23 , and a memory for storing executable instructions 24 and processor 25.
  • the first image acquisition device 21 is used to acquire a three-dimensional point cloud of the target scene.
  • the second image acquisition device 22 is configured to acquire an image sequence of the target scene.
  • the inertial measurement unit 23 is configured to acquire motion data of the gimbal load; the motion data includes the pose information of the gimbal load.
  • processor 25 executes the executable instructions, it is configured to:
  • the three-dimensional point cloud is registered according to the motion data to obtain the relative pose between the three-dimensional point clouds; and the image sequence is registered to obtain the relative pose between the images;
  • the relative pose between the three-dimensional point clouds, and the relative pose between the images obtain the pose and/or the first image acquisition device when acquiring the three-dimensional point cloud. 2. the pose when the image acquisition device acquires the image;
  • a 3D model of the target scene is generated according to the 3D point cloud, the sequence of images, and the pose when the first image acquisition device acquires the 3D point cloud and/or the pose when the second image acquisition device acquires the image .
  • the processor 25 executes the executable instructions included in the memory 24, and the processor 25 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processors) Processor, DSP), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • CPU Central Processing Unit
  • DSP Digital Signal Processors
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the memory 24 stores executable instructions for the three-dimensional reconstruction method, and the memory 24 may include at least one type of storage medium, including flash memory, hard disk, multimedia card, card-type memory (eg, SD or DX memory, etc.) , Random Access Memory (RAM), Static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory, Disk, CD and so on. Also, the pan-tilt load 20 may cooperate with a network storage device that performs the storage function of the memory through a network connection.
  • the memory 24 may be an internal storage unit of the gimbal load 20 , such as a hard disk or a memory of the gimbal load 20 .
  • the memory 24 can also be an external storage device of the gimbal load 20, such as a pluggable hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, and a flash memory card equipped on the gimbal load 20. (Flash Card) etc. Further, the memory 24 may also include both an internal storage unit of the pan-tilt load 20 and an external storage device. Memory 24 is used to store executable instructions and other programs and data required by the device. The memory 24 may also be used to temporarily store data that has been or will be output.
  • an external storage device of the gimbal load 20 such as a pluggable hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, and a flash memory card equipped on the gimbal load 20. (Flash Card) etc.
  • the memory 24 may also include both an internal storage unit of the pan-tilt load 20 and an external storage device. Memory 24 is used to store
  • the various embodiments described herein can be implemented using computer readable media such as computer software, hardware, or any combination thereof.
  • the embodiments described herein can be implemented using application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays ( FPGA), processors, controllers, microcontrollers, microprocessors, electronic units designed to perform the functions described herein are implemented.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGA field programmable gate arrays
  • processors controllers, microcontrollers, microprocessors, electronic units designed to perform the functions described herein are implemented.
  • embodiments such as procedures or functions may be implemented with separate software modules that allow the performance of at least one function or operation.
  • the software codes may be implemented by a software application (or program) written in any suitable programming language, which may be stored in
  • FIG. 8 is only an example of the pan-tilt load 20, and does not constitute a limitation to the pan-tilt payload 20. It may include more or less components than the one shown, or combine some components, or different Components, such as devices, may also include input and output devices, network access devices, buses, and the like.
  • the first image acquisition device 21 is specifically configured to acquire the 3D point cloud of the target scene in real time; the second image acquisition device 22 is specifically configured to acquire the image sequence of the target scene in real time; and the The inertial measurement unit 23 is specifically configured to acquire motion data of the PTZ load in real time.
  • the target scene includes an indoor scene or an outdoor non-empty scene, and/or a scene where the satellite positioning signal is lower than a predetermined intensity.
  • the first image acquisition device 21 includes at least one or more of the following: lidar, binocular vision sensor, and structured light depth camera;
  • the second image acquisition device 22 includes at least one or more of the following: a visible light camera, a grayscale camera, and an infrared camera.
  • the processor 25 is specifically configured to: acquire the pose and/or the second image according to the three-dimensional point cloud, the image sequence, and the first image acquisition device when acquiring the three-dimensional point cloud
  • the acquisition device acquires the pose of the image, and generates a three-dimensional model of the target scene in real time.
  • the processor 25 when acquiring the relative pose between the three-dimensional point clouds, is further configured to: for a group of three-dimensional point clouds collected in a continuous time series, according to the The transformation relationship between the motion data acquisition points corresponding to each point;
  • the points in the set of three-dimensional point clouds are reprojected to the same coordinate system according to the transformation relationship between the points.
  • the processor 25 is further configured to: determine the relative pose of the inertial measurement unit 23 within a preset time interval according to the motion data corresponding to each point in the set of three-dimensional point clouds; Let the time interval be the time interval for obtaining adjacent points;
  • the motion data corresponding to each point is determined according to the acquisition time of the point.
  • the processor 25 when acquiring the relative pose between the three-dimensional point clouds, the processor 25 is further configured to:
  • For two adjacent groups of 3D point clouds obtain the surface features and/or corner features of each group of 3D point clouds; wherein, the adjacent two groups of 3D point clouds both include points collected for the same object;
  • the relative poses between the two adjacent groups of three-dimensional point clouds are obtained; the relative poses between the two adjacent groups of three-dimensional point clouds are used to The adjacent two sets of 3D point clouds are reprojected to the same coordinate system.
  • the surface features and/or corner features of each group of three-dimensional point clouds are determined according to curvature information between three-dimensional points in the group of three-dimensional point clouds.
  • the surface features of each group of three-dimensional point clouds are determined according to curvature information whose curvature is greater than a preset threshold;
  • the corner features of each group of three-dimensional point clouds are determined according to curvature information whose curvature is less than or equal to a preset threshold.
  • the processor 25 is further configured to: according to the similarity between the surface features of the adjacent two groups of three-dimensional point clouds, and/or the similarity between the corner features of the adjacent two groups of three-dimensional point clouds degrees to obtain the relative pose between 3D point clouds.
  • the similarity between the surface features of the adjacent two groups of three-dimensional point clouds is determined based on the distance between the surface features of the two adjacent groups of three-dimensional point clouds; and/or,
  • the similarity between the corner features of the adjacent two groups of three-dimensional point clouds is determined based on the distance between the corner features of the two adjacent groups of three-dimensional point clouds.
  • the surface feature includes at least one three-dimensional coordinate information indicating a plane and a normal vector of the plane; and/or,
  • the corner feature includes at least one three-dimensional coordinate information indicating an edge and a vector indicating the edge.
  • the sequence of images includes a plurality of images acquired in a continuous time sequence
  • the processor 25 is further configured to extract feature points in the images, and perform registration based on the feature points of the images to obtain the relative poses between the images .
  • the sequence of images includes a plurality of images acquired in a continuous time sequence
  • the processor 25 is further configured to:
  • the registration is performed according to the feature points of the images and the depth information of the feature points, and the relative poses between the images are acquired.
  • the processor 25 when acquiring the relative pose between the images, is further configured to: in the absence of light, use the motion data to reproject the three-dimensional point cloud, and re-project the three-dimensional point cloud according to the re-projection.
  • the projected three-dimensional point cloud and the motion data generate a depth map corresponding to the images in the image sequence; perform registration according to the feature points extracted from the depth map to obtain the relative poses between the images.
  • the processor 25 is further configured to:
  • the pose graph model is optimized by using the motion data, the relative poses between the three-dimensional point clouds, and the relative poses between the images, and obtains the data obtained when the first image acquisition device 21 obtains the three-dimensional point cloud.
  • each vertex is configured to instruct the first image acquisition device 21 to acquire a pose when acquiring a three-dimensional point cloud and/or the second image acquisition device 22 to acquire an image. posture when
  • edges between the vertices are configured to indicate relative poses between the three-dimensional point clouds, relative poses between the images, or the motion data.
  • the edges between the various vertices are further configured to indicate GPS information and/or RTK information.
  • the pose graph model includes a sliding window pose graph model.
  • the processor 25 is further configured to: after adding a label index to the 3D model, store the 3D model in the storage location pointed to by the label index; and/or, store the 3D model Real-time transmission to the remote control terminal to display the three-dimensional model in real time on the remote control terminal.
  • the processor 25 is further configured to:
  • Loop closure detection is performed according to the similarity between multiple images in the image sequence; or, loop closure detection is performed according to the similarity between multiple depth maps obtained by re-projection of the 3D point cloud;
  • the pose graph model is globally optimized using the result of loop closure detection; and the three-dimensional model is updated according to the globally optimized pose graph model.
  • the processor 25 is further configured to: perform global optimization on the pose graph model by using the relative pose between the images forming the loop or the relative pose between the points forming the loop.
  • a bag-of-words model is used to measure the similarity between multiple images in the image sequence or the similarity between the multiple depth images.
  • the processor 25 is further configured to:
  • the first image acquisition device 21 , the second image acquisition device 22 and the inertial measurement unit 23 are controlled to acquire data according to the acquisition track and acquisition time.
  • the processor 25 is further configured to:
  • Receive feedback information transmitted by the remote control terminal the feedback information is used to indicate the position of the missed acquisition by the first image acquisition device 21, and the feedback information is input by the user with respect to the three-dimensional model displayed on the remote control terminal;
  • the first image acquisition device 21 is controlled to acquire the three-dimensional point cloud at the position indicated by the feedback information in the target scene.
  • an embodiment of the present application also provides a movable platform, including the above-mentioned PTZ load.
  • the movable platform includes at least an unmanned aerial vehicle, an unmanned vehicle or a mobile robot.
  • non-transitory computer-readable storage medium such as a memory including instructions, executable by a processor of an apparatus to perform the above-described method.
  • the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.
  • a non-transitory computer-readable storage medium when the instructions in the storage medium are executed by the processor of the terminal, enable the terminal to execute the above method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Remote Sensing (AREA)
  • Computer Graphics (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Processing Or Creating Images (AREA)

Abstract

L'invention concerne un procédé de reconstruction tridimensionnelle, une charge de cardan (20), une plate-forme amovible et un support de stockage lisible par ordinateur. Le procédé consiste à : acquérir des nuages de points tridimensionnels d'une scène cible, une séquence d'images et des données de mouvement d'une charge de cardan (20), les nuages de points tridimensionnels étant acquis par un premier appareil de collecte d'images (21), la séquence d'images étant acquise par un second appareil de collecte d'images (22), et les données de mouvement comprenant des informations de pose de la charge de cardan (20) pendant l'acquisition des nuages de points tridimensionnels de la scène cible et de la séquence d'images ; enregistrer les nuages de points tridimensionnels en fonction des données de mouvement, de façon à acquérir une pose relative entre les nuages de points tridimensionnels, et enregistrer la séquence d'images, de façon à acquérir une pose relative entre les images ; et générer un modèle tridimensionnel de la scène cible selon les nuages de points tridimensionnels, la séquence d'images, les données de mouvement, la pose relative entre les nuages de points tridimensionnels et la pose relative entre les images. Un modèle tridimensionnel de haute précision est acquis.
PCT/CN2020/120978 2020-10-14 2020-10-14 Procédé de reconstruction tridimensionnelle, charge de cardan, plate-forme amovible et support de stockage lisible par ordinateur WO2022077296A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/120978 WO2022077296A1 (fr) 2020-10-14 2020-10-14 Procédé de reconstruction tridimensionnelle, charge de cardan, plate-forme amovible et support de stockage lisible par ordinateur

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/120978 WO2022077296A1 (fr) 2020-10-14 2020-10-14 Procédé de reconstruction tridimensionnelle, charge de cardan, plate-forme amovible et support de stockage lisible par ordinateur

Publications (1)

Publication Number Publication Date
WO2022077296A1 true WO2022077296A1 (fr) 2022-04-21

Family

ID=81207575

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/120978 WO2022077296A1 (fr) 2020-10-14 2020-10-14 Procédé de reconstruction tridimensionnelle, charge de cardan, plate-forme amovible et support de stockage lisible par ordinateur

Country Status (1)

Country Link
WO (1) WO2022077296A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114529652A (zh) * 2022-04-24 2022-05-24 深圳思谋信息科技有限公司 点云补偿方法、装置、设备、存储介质和计算机程序产品
CN116844074A (zh) * 2023-07-25 2023-10-03 北京爱科农科技有限公司 一种果园三维场景及重点区域的全景展示联动的方法
CN117168441A (zh) * 2023-11-02 2023-12-05 西安因诺航空科技有限公司 一种多传感器融合的slam定位和重建方法及系统
WO2023237065A1 (fr) * 2022-06-08 2023-12-14 先临三维科技股份有限公司 Procédé et appareil de détection de fermeture de boucle, dispositif électronique et support

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104236548A (zh) * 2014-09-12 2014-12-24 清华大学 一种微型无人机室内自主导航方法
CN106780729A (zh) * 2016-11-10 2017-05-31 中国人民解放军理工大学 一种无人机序列影像批处理三维重建方法
CN110060332A (zh) * 2019-04-09 2019-07-26 上海科技大学 基于机载采集设备的高精度三维建图与建模系统
US20190325089A1 (en) * 2018-04-18 2019-10-24 Reconstruct Inc. Computation of point clouds and joint display of point clouds and building information models with project schedules for monitoring construction progress, productivity, and risk for delays

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104236548A (zh) * 2014-09-12 2014-12-24 清华大学 一种微型无人机室内自主导航方法
CN106780729A (zh) * 2016-11-10 2017-05-31 中国人民解放军理工大学 一种无人机序列影像批处理三维重建方法
US20190325089A1 (en) * 2018-04-18 2019-10-24 Reconstruct Inc. Computation of point clouds and joint display of point clouds and building information models with project schedules for monitoring construction progress, productivity, and risk for delays
CN110060332A (zh) * 2019-04-09 2019-07-26 上海科技大学 基于机载采集设备的高精度三维建图与建模系统

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114529652A (zh) * 2022-04-24 2022-05-24 深圳思谋信息科技有限公司 点云补偿方法、装置、设备、存储介质和计算机程序产品
CN114529652B (zh) * 2022-04-24 2022-07-19 深圳思谋信息科技有限公司 点云补偿方法、装置、设备和存储介质
WO2023237065A1 (fr) * 2022-06-08 2023-12-14 先临三维科技股份有限公司 Procédé et appareil de détection de fermeture de boucle, dispositif électronique et support
CN116844074A (zh) * 2023-07-25 2023-10-03 北京爱科农科技有限公司 一种果园三维场景及重点区域的全景展示联动的方法
CN117168441A (zh) * 2023-11-02 2023-12-05 西安因诺航空科技有限公司 一种多传感器融合的slam定位和重建方法及系统
CN117168441B (zh) * 2023-11-02 2024-02-20 西安因诺航空科技有限公司 一种多传感器融合的slam定位和重建方法及系统

Similar Documents

Publication Publication Date Title
WO2022077296A1 (fr) Procédé de reconstruction tridimensionnelle, charge de cardan, plate-forme amovible et support de stockage lisible par ordinateur
US11263761B2 (en) Systems and methods for visual target tracking
CN112567201B (zh) 距离测量方法以及设备
CN111156998B (zh) 一种基于rgb-d相机与imu信息融合的移动机器人定位方法
CN109887057B (zh) 生成高精度地图的方法和装置
US20210141378A1 (en) Imaging method and device, and unmanned aerial vehicle
KR102191445B1 (ko) 필수 이미지 및 위치 캡처 시스템 및 방법
WO2022036980A1 (fr) Procédé et appareil de détermination de pose, dispositif électronique, support de stockage et programme
US11906983B2 (en) System and method for tracking targets
CN108810473B (zh) 一种在移动平台上实现gps映射摄像机画面坐标的方法及系统
CN111936821A (zh) 用于定位的系统和方法
JP2015535980A (ja) 特に装置に係る視覚ベースの位置決めに使用される画像処理方法
WO2018193574A1 (fr) Procédé de production de trajectoire de vol, dispositif de traitement d'informations, système de production de trajectoire de vol, programme et support d'enregistrement
CN113587934A (zh) 一种机器人、室内定位方法、装置和可读存储介质
US11460302B2 (en) Terrestrial observation device having location determination functionality
US20210229810A1 (en) Information processing device, flight control method, and flight control system
WO2020019175A1 (fr) Procédé et dispositif de traitement d'image et dispositif photographique et véhicule aérien sans pilote
CA3069813A1 (fr) Capture, connexion et utilisation de donnees d'interieur de batiment a partir de dispositifs mobiles
US20210233271A1 (en) Information processing apparatus, server, movable object device, information processing method, and program
KR102402949B1 (ko) 정밀도가 향상된 영상 정보의 획득 방법
CN116027351A (zh) 一种手持/背包式slam装置及定位方法
US20210256732A1 (en) Image processing method and unmanned aerial vehicle
CN111581322B (zh) 视频中兴趣区域在地图窗口内显示的方法和装置及设备
TWI738315B (zh) 基於光標籤的自動追蹤拍攝系統
Zhao et al. 2D monocular visual odometry using mobile-phone sensors

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20957076

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20957076

Country of ref document: EP

Kind code of ref document: A1