WO2021115071A1 - Procédé et appareil de reconstruction tridimensionnelle pour image d'endoscope monoculaire, et dispositif terminal - Google Patents

Procédé et appareil de reconstruction tridimensionnelle pour image d'endoscope monoculaire, et dispositif terminal Download PDF

Info

Publication number
WO2021115071A1
WO2021115071A1 PCT/CN2020/129546 CN2020129546W WO2021115071A1 WO 2021115071 A1 WO2021115071 A1 WO 2021115071A1 CN 2020129546 W CN2020129546 W CN 2020129546W WO 2021115071 A1 WO2021115071 A1 WO 2021115071A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
key frame
pixel
coordinates
distortion
Prior art date
Application number
PCT/CN2020/129546
Other languages
English (en)
Chinese (zh)
Inventor
廖祥云
孙寅紫
王琼
王平安
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2021115071A1 publication Critical patent/WO2021115071A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10068—Endoscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker
    • G06T2207/30208—Marker matrix
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • This application belongs to the field of image processing technology, and in particular relates to a method, device and terminal device for three-dimensional reconstruction of monocular endoscopic images.
  • Three-dimensional reconstruction is one of the research hotspots in computer vision. Its main purpose is to restore the three-dimensional structure of objects from two-dimensional images. It is widely used in augmented reality, virtual navigation and medical fields.
  • the three-dimensional information of the image mainly depends on the visual real-time positioning and map construction (Simultaneous Localization and Mapping, SLAM) technology.
  • the distortion of the monocular endoscope imaging will cause the increase of the pose error, and the endoscope is usually used with a cold light source, and its imaging will also be disturbed by the light, which may affect the feature matching result in the SLAM process. It is usually difficult to use a monocular endoscope to provide accurate samples for training. Combining the SLAM scheme and the depth prediction scheme can the two-dimensional image sequence be densely reconstructed. However, due to the above-mentioned pose and depth map errors and other factors, The accuracy and effect of 3D reconstruction are deteriorated.
  • the embodiments of the present application provide a method and device for three-dimensional reconstruction of a monocular endoscope, which can solve the problem of reducing errors caused by imaging distortion caused by the inherent parameters of the monocular endoscope, and the accuracy of the three-dimensional reconstruction of a two-dimensional image sequence is not high and The problem of poor results.
  • an embodiment of the present application provides a three-dimensional reconstruction method of a monocular endoscopic image, including:
  • the image reconstruction based on the pose parameters of the key frame and the depth map of the key frame to obtain a three-dimensional point cloud includes:
  • the acquiring the distortion images of a plurality of checkerboard calibration boards taken by a monocular endoscope, and correcting the distortion images of the checkerboard calibration boards to obtain an image sequence includes:
  • the performing distortion correction on the image to be corrected based on the camera coordinate system to obtain the image sequence includes:
  • the pixel coordinates of the preset coordinates in the pixel coordinate system are mapped to the camera coordinate system to obtain the image sequence.
  • the obtaining the pixel coordinates of the key frame includes:
  • the determining a key frame from the image sequence includes:
  • the first image is used as a key frame, where the first image and the second image are Any two adjacent frames of images in the image sequence.
  • the obtaining the pose parameters of the key frame includes:
  • the estimating the depth map of the key frame includes:
  • a reference frame image from the key frames, where the reference frame image is any frame image or multiple frames of images in the key frame;
  • an embodiment of the present application provides a three-dimensional reconstruction device for monocular endoscopic images, including:
  • the acquisition module is used to acquire the distortion images of a plurality of checkerboard calibration boards taken by a monocular endoscope, and perform distortion correction on the distortion images of the checkerboard calibration boards to obtain an image sequence;
  • a calculation module for obtaining the pose parameters of the key frame and estimating the depth map of the key frame
  • the generating module is used for image reconstruction based on the pose parameters of the key frame and the depth map of the key frame to obtain a three-dimensional point cloud.
  • an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and running on the processor.
  • the processor executes the computer program, Realize the above-mentioned three-dimensional reconstruction method.
  • an embodiment of the present application provides a computer-readable storage medium that stores a computer program that implements the above-mentioned three-dimensional reconstruction method when the computer program is executed by a processor.
  • the embodiment of the present application has the following beneficial effects: by obtaining the distortion images of multiple checkerboard calibration boards taken by a monocular endoscope, and performing distortion correction on the distortion images of the multiple checkerboard calibration boards to obtain an image sequence, Determine the key frame from the image sequence, obtain the pose parameters of the key frame, estimate the depth map of the key frame, and reconstruct the image based on the pose parameters of the key frame and the depth map of the key frame to obtain a three-dimensional point cloud.
  • the above method uses the checkerboard calibration board image to achieve the calibration and distortion correction of the monocular endoscope to obtain the image sequence, which effectively reduces the imaging distortion error caused by the monocular endoscope itself, and determines multiple images that meet the requirements from the image sequence
  • a key frame and determine the pose parameters of the key frame, it can avoid the interference of external factors such as light changes, can accurately estimate the pose parameters and depth map, and perform image reconstruction according to the position parameters and depth map of the key frame to get more
  • the fine three-dimensional point cloud also improves the display effect of the image.
  • FIG. 1 is a schematic flowchart of a method for three-dimensional reconstruction of monocular endoscopic images provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of the image distortion correction process provided by an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a three-dimensional reconstruction device for monocular endoscopic images provided by an embodiment of the present application
  • Fig. 4 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • the term “if” can be construed as “when” or “once” or “in response to determination” or “in response to detecting “.
  • the phrase “if determined” or “if detected [described condition or event]” can be interpreted as meaning “once determined” or “in response to determination” or “once detected [described condition or event]” depending on the context ]” or “in response to detection of [condition or event described]”.
  • FIG. 1 shows a schematic flow chart of a three-dimensional reconstruction method of a monocular endoscopic image provided by the first application of the present application.
  • the three-dimensional reconstruction method of a monocular endoscope image provided by the present application includes S101 To S104, the details are as follows:
  • S101 Obtain distortion images of multiple checkerboard calibration boards taken by a monocular endoscope, and perform distortion correction on the distortion images of the multiple checkerboard calibration boards to obtain an image sequence;
  • the distortion image of the checkerboard calibration board can be used for distortion correction of the monocular endoscope.
  • the checkerboard calibration board is a binary image with two rows of black and white stripe intervals.
  • the monocular endoscope can observe the calibration from different angles.
  • the board obtains multiple monocular endoscope distortion images.
  • the imaging process of the camera mainly involves the transformation between the image pixel coordinate system, the image physical coordinate system, the camera coordinate system and the world coordinate system. Due to the lens imaging principle, the camera imaging distortion occurs, and the distortion correction is to find the corresponding relationship between the point positions before and after the distortion. .
  • the imaging model of the monocular endoscope is different from the small hole imaging model, but is closer to the fisheye camera model.
  • the checkerboard calibration board is a black and white grid arranged at intervals, also called a checkerboard calibration board.
  • the calibration target is used in machine vision, image measurement, photogrammetry, three-dimensional reconstruction and other applications to correct lens distortion and determine the physical size and pixels.
  • the conversion relationship between the two, and to determine the relationship between the three-dimensional geometric position of a point on the surface of a space object and its corresponding point in the image requires the establishment of a geometric model of camera imaging.
  • the camera's geometric model can be obtained by the camera with a fixed-pitch pattern array plate and calculation by the calibration algorithm, thereby obtaining high-precision measurement and reconstruction results.
  • the flat plate with a fixed-pitch pattern array is the calibration plate.
  • the camera calibration of the monocular endoscope can be realized, and the distorted image can be corrected to obtain the image according to the calibration monocular endoscope.
  • Sequence that is, real image, can reduce the error caused by image distortion to image recognition.
  • Figure 2 shows a flow chart of the implementation of distortion correction provided by the present application. As shown in Figure 2, the acquisition of the distortion images of the multiple checkerboard calibration boards taken by the monocular endoscope, and the distortion images of the multiple checkerboard calibration boards Perform correction to obtain an image sequence, including S1011 to S1013:
  • 20 images with a checkerboard calibration board taken with a monocular endoscope at about different angles are acquired, the corner points of the checkerboard in the image are extracted, and the distorted image that meets the fitting conditions is selected.
  • the Canny corner operator can be used to detect the distortion images obtained from all monocular endoscopes observing the checkerboard calibration board, and count the number of corner points in all the distortion images.
  • the distortion image that meets the fitting conditions is preferably the number of corner points detected in the image No less than 6 meshes. Among them, the number of corner points can be selected according to actual conditions, and is not specifically limited here.
  • the parameters of the ellipse equation are obtained by fitting the selected distortion image and the detected corner points.
  • the ellipse equation can be a standard equation including 6 parameters. According to the corner points in the detected distortion image, the least square method is used to obtain the parameters.
  • the parameters of the ellipse equation are obtained from the curved surface projection parameters, and the parameter fitting results of the ellipse equations of multiple distorted images are obtained by means of filtering.
  • fx and fy are the focal lengths of the endoscope in pixels
  • cx and cy are the principal point positions in pixels (that is, the center pixel position of the imaging).
  • the chessboard is a calibration board composed of black and white squares as the calibration object for camera calibration (mapped from the real world to the object in the digital image). Compared with a three-dimensional object, a two-dimensional object lacks some information.
  • the checkerboard is used as a calibration object because the plane checkerboard mode is easier to handle. After changing the position of the checkerboard many times to capture the image, it can obtain richer coordinate information.
  • S1012 Determine an image to be corrected from the distorted image according to the camera parameter and the distortion parameter;
  • calibrating the monocular endoscope can determine the pose of the camera to obtain the camera parameters and distortion parameters of the monocular endoscope.
  • the camera parameters and the distortion parameters are calculated Whether the image has not been distorted to the distorted image to be corrected, that is, it can judge whether the captured multiple images are distorted, or set a preset threshold, and compare the calculation result with the preset threshold to obtain a comparison result, where the comparison result is The ones with greater difference in the comparison result are regarded as distortion and those with little difference in the comparison result are regarded as no distortion, and vice versa.
  • various distortions are often produced in the process of image acquisition or display.
  • the common ones are geometric shape distortion, grayscale distortion, and color distortion.
  • the causes of image distortion are the aberration and distortion of the imaging system. , Limited bandwidth, shooting status, scanning nonlinearity, relative motion, etc., non-uniform lighting conditions or point light source lighting, etc.
  • the image to be corrected is determined from the multiple images taken, which is convenient to eliminate the error of distortion in image recognition and processing, and improves the accuracy of image processing to a certain extent.
  • S1013 Perform distortion correction on the image to be corrected based on the camera coordinate system to obtain the image sequence.
  • the straight line in the distortion space is generally no longer a straight line in the image space, but only the straight line passing through the center of symmetry is an exception.
  • the center of symmetry can be found, and then the general geometric distortion correction process can be performed. .
  • the general steps of distortion correction are to first find the symmetry center of the distortion map, convert the address space relationship represented by the distortion map into a space coordinate system with the center of symmetry as the origin, and then space transformation, rearrange the pixels on the input image, that is, the distortion map to restore The original spatial relationship, that is, use the address mapping relationship to find their corresponding point in the distortion map space for each point in the correction map space, and finally the grayscale difference is to assign the corresponding grayscale value to the pixel after the space transformation to restore the original The gray value of the location.
  • the correction of geometric distortion requires the use of coordinate transformation, including simple transformations such as parallel movement, rotation, enlargement and reduction.
  • the process of distortion correction can be understood as processing a distorted image into an undistorted image, that is, a real image.
  • Different camera models display different images when taking pictures, and may or may not be distorted.
  • the process of using distortion correction can be the same or different.
  • Image distortion mainly includes radial distortion and tangential distortion.
  • Radial distortion refers to the smallest distortion at the center position. As the radius increases, the distortion increases. Radial distortion can be divided into pincushion distortion and barrel distortion. Tangential distortion refers to when the lens is not parallel to the imaging plane, similar to perspective transformation.
  • the performing distortion correction on the image to be corrected based on the camera coordinate system to obtain the image sequence includes steps A1 to A3:
  • Step A1 Obtain the preset coordinates of each pixel of the image to be corrected in the camera coordinates
  • the camera coordinate system can be obtained by calibrating the monocular endoscope.
  • the world coordinate system and the camera coordinate system, the camera coordinate system and the image coordinate system, and the image coordinates can be realized.
  • System to pixel coordinate system conversion The conversion between the world coordinate system and the camera coordinate system is from one three-dimensional coordinate to another three-dimensional coordinate system.
  • the pose parameters of the camera can be obtained through the rotation matrix and the translation vector, that is, the camera coordinate system.
  • From the camera coordinate system to the image coordinate system a three-dimensional coordinate is projected on a two-dimensional plane, and it is estimated based on the distance between the two coordinate systems, that is, the focal length of the camera.
  • the preset coordinates in the camera coordinate system are corrected to obtain the coordinates in the undistorted camera coordinate system, and the coordinates in the undistorted camera coordinate system are mapped to the pixel coordinate system to obtain an undistorted image sequence.
  • Step A2 Project the camera coordinate system onto the plane where each pixel of the image to be corrected is located, and obtain the pixel coordinates of the preset coordinates in the pixel coordinate system;
  • the coordinates of the pixel (u', v') of the image taken by the monocular endoscope in the camera coordinate system are (x, y, z), and the pixel point in the camera coordinate system is The coordinates are projected to the plane where the image is located, that is, the image coordinate system.
  • the positional relationship of the origin of the image coordinate system relative to the origin of the pixel coordinate system it can be regarded as projecting the coordinates of the pixel point in the camera coordinate system to the pixel coordinate system, which can be expressed as follows :
  • ⁇ ′ ⁇ (1+k 1 ⁇ 2 +k 2 ⁇ 4 +k 3 ⁇ 6 +k 4 ⁇ 8 )
  • (x', y') are the coordinates projected on the plane
  • r represents the distance between the point and the center on the projection plane (projection radius)
  • ⁇ represents the angle of incidence.
  • Step A3 Map the pixel coordinates of the preset coordinates in the pixel coordinate system to the camera coordinate system to obtain the image sequence.
  • N undistorted images there are a total of 4 internal parameters + 6N external parameters to calibrate.
  • 10 or 20 images can be generally used to obtain a more accurate solution by using the least square method.
  • the distortion-related parameters can be obtained according to the remaining point coordinates.
  • a remapping process can be used to convert the pixel coordinates of the distorted endoscopic image into the coordinates of the distorted camera coordinate system.
  • the distorted camera coordinate system coordinates are transformed into the undistorted camera coordinate system coordinates, and finally the undistorted camera coordinate system coordinates are transformed into the pixel coordinates of the undistorted image, so that the corrected image sequence can be obtained, and the corresponding image can also be obtained.
  • the pixel coordinates of it is convenient to determine the key frame and the pose parameters of the key frame later.
  • ORB_SLAM2 is an embedded position recognition model, which has the characteristics of relocation, preventing tracking failure (such as occlusion), reinitialization of the mapped scene, loop detection, etc., using the same ORB features for tracking, mapping, and location recognition tasks.
  • Features have good robustness in rotation and scale, and have good invariance to the camera's automatic gain, automatic exposure, and illumination changes. It can also quickly extract features and match features to meet the needs of real-time operation.
  • This application uses ORB_SLAM2 to determine the key frame and pose estimation of monocular endoscopic images.
  • ORB You can use ORB to extract features of the image sequence, estimate the initial pose of the camera through the previous image frame, and initialize the position through global relocation.
  • the four processes of posture, tracking the local map and the judgment standard of the new key frame are used to determine the key frame and the posture parameter of the key frame more accurately.
  • the key frame can be used as a mark of the image sequence and has a guiding effect.
  • the distortion-corrected images in the image sequence are arranged in a preset order, and they can be arranged in sequence according to the shooting time order, which is convenient for feature extraction of each image. Processing to improve the efficiency of monocular endoscope image processing.
  • the determining the key frame from the image sequence includes steps B1 to B2, which are specifically as follows:
  • Step B1 Obtain local features of each image in the image sequence, and perform feature point matching on each image in the image sequence based on the local feature of each image to obtain a matching result;
  • extract the local features of each image in the image sequence and perform feature point matching on each image in the image sequence with the local features of each image to extract the regions corresponding to the coordinates of each image for feature matching, or Extract all pixels in the rich area of the image, and match the feature points of the two images before and after in the preset order, that is, the number of feature points that are successfully matched with the same ORB feature in the two frames of the image is used as the matching result, and set the number of feature points that are successfully matched
  • the threshold is between 50-100.
  • the peripheral edge area of the image imaged by the monocular endoscope is a black area with no information, and useful feature information cannot be extracted. Therefore, the information-rich area in the image is selected and the area can be defined as a region of interest.
  • ORB Oriented FAST and Rotated Brief
  • the ORB algorithm includes feature point extraction and feature point description.
  • the ORB algorithm has the characteristics of fast calculation speed and uses FAST detection
  • the feature point again, is to use the BRIEF algorithm to calculate the descriptor.
  • the unique binary string representation of the descriptor not only saves storage space, but also greatly shortens the matching time.
  • the key frame can be used as a marker to quickly process the image sequence, which can improve the efficiency of monocular endoscope image processing.
  • Step B2 When the matching result is that the number of feature points matched by the first image and the second image is greater than or equal to a preset threshold, use the first image as a key frame, wherein the first image and the first image The two images are any two adjacent frames of images in the image sequence.
  • the threshold for the number of feature points that are successfully matched is set to be between 50 and 100. When the number of feature points matched by the first image and the second image exceeds the threshold, it is determined that the first and the next two frames of images are matched successfully .
  • the constant motion rate model can be used to predict the current camera position (that is, the camera is considered to be moving at a constant speed), and then the corresponding cloud of the feature points in the previous frame image can be searched for in the map
  • the matching point between the point and the current frame image is finally used to further optimize the pose of the current camera by using the searched matching point, so as to obtain the image in the image sequence that meets the requirements to improve the accuracy of determining the key frame.
  • ORB_SLAM2 based on the feature point method can obtain the pose parameters.
  • the pose parameters describe the two images corresponding to the camera.
  • the depth map refers to the number of bits used to store each pixel, which is used to measure the color resolution of the image.
  • obtaining the pose parameters of the key frame includes:
  • the pose of the first image that is, the image of the previous frame
  • the key frame contains
  • extract the ORB feature of each frame of the key frame according to the pose initialization of the first image perform feature matching with the previous frame, and estimate its pose parameters (rotation matrix Ri, translation vector ti )
  • take the image of successful pose estimation as the key frame obtain the pose parameter corresponding to the key frame, and store the key frame and the pose parameter corresponding to the key frame together, so as to perform depth estimation on all the key frames later.
  • the estimating the depth map of the key frame includes:
  • a reference frame image from the key frames, where the reference frame image is any frame image or multiple frames of images in the key frame;
  • the photometric error is minimized according to the first depth map of the key image frame in the monocular video, and the current camera pose between the reference frame image and the key frame in the monocular endoscopic image is determined.
  • the current camera pose triangulates the high gradient image points in the reference frame image and the key frame, determines the second depth map of the key frame, performs Gaussian fusion of the first depth map and the second depth map, and updates the first depth map of the key frame.
  • a depth map If the next camera pose between the next image frame and the key frame of the reference frame image exceeds the preset camera pose, the updated first depth map is determined as the dense depth map of the key frame.
  • depth map estimation one frame of image or multiple frames of images can be selected for estimation.
  • each pixel of each image in the key frame is triangulated.
  • Bayesian probability estimation strategy to get dense depth map.
  • multiple images in the key frame are selected for iterative calculation to obtain the depth value corresponding to each pixel, then the depth map is smoothed and filtered to eliminate some noise in the depth map, which can improve the efficiency and accuracy of depth estimation.
  • the first depth map of the key frame can be a dense depth map obeying the Gaussian distribution obtained by initializing the depth values of the high gradient points in the key frame, or it can be the depth of the previous key frame of the key frame.
  • the value is a dense depth map projected according to the camera pose. For example, if the key frame to be depth estimated is the first key frame in the image sequence, the first depth map of the key frame is the dense depth map obtained by initialization; if the key frame with depth estimation is the first key frame in the image sequence For key frames other than one key frame, the first depth map of the key frame is a dense depth map obtained by projecting the depth value of the previous key frame.
  • Luminosity error refers to the measurement difference between the high gradient point in the projected image and the corresponding high gradient point in the reference frame image.
  • the projected image is based on the initial camera pose between the reference frame and the key frame in the image sequence.
  • the high gradient points corresponding to the pixels in the frame are projected to the reference frame image.
  • the current camera pose includes the rotation and translation between the reference frame and the key frame.
  • the second depth map of the key frame refers to the image sequence according to the The new dense depth map obtained by triangulating the current camera pose between the reference frame image and the key frame; the next frame image of the reference frame image refers to the next frame image adjacent to the pre-reference frame image in the image sequence ,
  • the posture of the latter camera includes the maximum threshold of the posture of the latter camera, which can be preset according to actual conditions and requirements, and there is no specific limitation here.
  • a dense depth map refers to an image that includes depth values corresponding to a large number of feature points, or an image that includes both high gradient points and depth values corresponding to low gradient points.
  • the depth estimation obtains the depth map and the depth value, which is convenient for subsequent restoration of the spatial coordinates of the pixel.
  • S104 Perform image reconstruction based on the pose parameters of the key frame and the depth map of the key frame to obtain a three-dimensional point cloud.
  • 3D reconstruction refers to the establishment of a 3D model from the input data.
  • Each frame of data scanned by the depth camera not only contains the color RGB image of the point in the scene, but also includes each point to the vertical plane where the depth camera is located. This distance value is called the depth value, and these depth values together constitute the depth map of this frame.
  • the depth map can be regarded as a grayscale image.
  • the grayscale value of each point in the image represents the true distance from the position of the point in reality to the vertical plane where the camera is located.
  • Each point in the RGB image corresponds to a point on the camera. A three-dimensional point in the local coordinate system.
  • the process of 3D reconstruction can be image acquisition, camera calibration, feature extraction, stereo matching, 3D reconstruction, etc., where stereo matching refers to establishing a correspondence between image pairs based on the extracted features. That is, the imaging points of the same physical space point in two different images are mapped one by one.
  • stereo matching refers to establishing a correspondence between image pairs based on the extracted features. That is, the imaging points of the same physical space point in two different images are mapped one by one.
  • stereo matching pay attention to the interference of some factors in the scene, such as lighting conditions, noise interference, distortion of the geometric shape of the scene, surface physical characteristics, and camera characteristics, etc., in order to obtain a high-precision three-dimensional point cloud, and also enhance the vision effect.
  • S104 may include steps C1 to C3, which are specifically as follows:
  • Step C1 Obtain the pixel coordinates of the key frame
  • the pixel coordinate system and the pixel coordinates of each image in the key frame can be determined.
  • the pixel coordinates indicate the position of the pixel in the image, and the key frame can be determined.
  • the pixel position of each image is convenient for subsequent three-dimensional reconstruction of the image.
  • Step C2 calculating the target space coordinates according to the depth map, the pose parameters of the key frame, and the pixel coordinates of the key frame;
  • the depth value corresponding to the depth map of each image in the key is obtained, and the depth value and the pose parameter of the key frame and the pixel coordinates of each image of the key frame are calculated to obtain the spatial coordinates of each image, namely
  • the conversion from two-dimensional coordinates to three-dimensional coordinates, according to the depth value obtained by accurate depth estimation, also improves the accuracy of the calculated target space coordinates.
  • Step C3 Obtain the color information of each pixel in the key frame, and perform point cloud fusion on the key frame according to the color information of each pixel in the key frame and the target space coordinates to obtain the Describe the three-dimensional point cloud.
  • the pixel coordinates [u, v] in the two-dimensional image, the corresponding point cloud contains color information and spatial position information, and the color information is represented by the RGB value of the pixel.
  • the pose parameters of the key frame and the pixel coordinates of the key frame are calculated to obtain the target space coordinates as [x, y, z], and the space coordinates are restored from the pixel coordinates [u, v] and its depth value d by the following formula Means:
  • d represents the depth of the pixel, which is derived from the depth estimation of the REMODE scheme
  • (x',y',z') is the coordinate value in the camera coordinate system
  • (Ri,ti) is the bit corresponding to the frame Pose parameters.
  • the point cloud is a set of discrete points.
  • the point cloud stores the spatial coordinates and color information corresponding to the pixels of the frame.
  • the multi-frame point cloud Stored in a container, and then the repeated point cloud is removed by a filter, and a three-dimensional point cloud of multiple frames of images can be obtained.
  • the above-mentioned three-dimensional reconstruction method may draw a point cloud of multiple frames of images during fusion to obtain finer three-dimensional information.
  • obtaining the pixel coordinates of the key frame in step C1 includes steps C11 to C13:
  • Step C11 Project the camera coordinate system onto the plane where each pixel of the image to be corrected is located, and obtain the pixel coordinates of the preset coordinates in the pixel coordinate system;
  • the coordinates of the pixel points in the camera coordinate system are defined, and the correspondence between the camera coordinate system and the image coordinate system is calculated by projection, and then the pixel coordinate system is obtained through the correspondence between the image coordinate system and the pixel coordinate system.
  • the pixel coordinates here are the same as the process of the pixel coordinates obtained in the above-mentioned distortion correction, and will not be repeated here.
  • Step C12 Map the pixel coordinates of the preset coordinates in the pixel coordinate system to the camera coordinate system to obtain the image sequence and the pixel coordinates corresponding to the image sequence;
  • the corrected image sequence and the pixel coordinates corresponding to the image sequence can be obtained through the coordinate system transformation method of the distortion correction.
  • the specific processing process here is the same as the above-mentioned distortion correction process, and will not be repeated here.
  • Step C13 Obtain the pixel coordinates of the key frame based on the pixel coordinates corresponding to the image sequence.
  • the key frame is determined from the image sequence, and the pixel coordinates of the key frame can be obtained. According to the pixel coordinates of each image of the key frame, the moving position relationship of each image relative to the camera can be determined, so as to improve the single The processing efficiency of the endoscopic image.
  • FIG. 3 shows a three-dimensional reconstruction device 300 for monocular endoscopic images provided by an embodiment of the present application.
  • the three-dimensional reconstruction device 300 for monocular endoscope images provided by the present application includes:
  • the acquiring module 310 is configured to acquire the distortion images of a plurality of checkerboard calibration boards taken by a monocular endoscope, and perform distortion correction on the distortion images of the checkerboard calibration boards to obtain an image sequence;
  • the determining module 320 is configured to determine a key frame from the image sequence
  • the calculation module 330 is configured to obtain the pose parameters of the key frame and estimate the depth map of the key frame;
  • the generating module 340 is configured to perform image reconstruction based on the pose parameters of the key frame and the depth map of the key frame to obtain a three-dimensional point cloud.
  • the device for 3D reconstruction of monocular endoscopic images may be a terminal device, a server, or a device capable of human-computer interaction.
  • the obtaining module 310 specifically includes:
  • the first acquisition unit is configured to acquire the corner points of the chessboard in the distortion images of the multiple chessboard calibration boards, and calibrate the monocular endoscope based on the corner points of the chessboard to obtain the monocular endoscope Camera parameters and distortion parameters of the mirror;
  • a first determining unit configured to determine an image to be corrected from the distorted image according to the camera parameter and the distortion parameter
  • the first processing unit is configured to perform distortion correction on the image to be corrected based on the camera coordinate system to obtain the image sequence.
  • the obtaining module 310 further includes:
  • a second acquiring unit configured to acquire the preset coordinates of each pixel of the image to be corrected in the camera coordinates
  • a second processing unit configured to project the camera coordinate system onto the plane where each pixel of the image to be corrected is located, to obtain the pixel coordinates of the preset coordinates in the pixel coordinate system;
  • the third processing unit is configured to map the pixel coordinates of the preset coordinates in the pixel coordinate system to the camera coordinate system to obtain the image sequence.
  • the determining module 320 specifically includes:
  • the third acquiring unit is configured to acquire the local features of each image in the image sequence, and perform feature point matching on each image in the image sequence based on the local features of each image to obtain a matching result;
  • the second determining unit is configured to use the first image as a key frame when the matching result is that the number of feature points matched by the first image and the second image is greater than or equal to a preset threshold, wherein the first image And the second image are any two adjacent frames of images in the image sequence.
  • the determining module 320 further includes:
  • a third determining unit configured to use the first image as a key frame when the number of feature points matched by the first image and the second image is greater than or equal to a preset threshold
  • the fourth processing unit is used to initialize the pose of the first image
  • the first estimation unit is used to estimate the pose parameters of the key frames in the image sequence.
  • the determining module 320 further includes:
  • a fourth determining unit configured to determine a reference frame image from the key frame, wherein the reference frame image is any frame image or multiple frames of images in the key frame;
  • the second estimation unit is configured to perform depth estimation processing on each pixel of the reference frame image based on the pose parameter to obtain the depth map of the key frame.
  • the generating module 340 includes:
  • the fourth acquiring unit is used to acquire the pixel coordinates of the key frame
  • the third estimation unit is configured to calculate the target space coordinates according to the depth map, the pose parameters of the key frame, and the pixel coordinates of the key frame;
  • the first generating unit is used to obtain the color information of each pixel in the key frame, and perform a point cloud on the key frame according to the color information of each pixel in the key frame and the target space coordinates Fusion to obtain the three-dimensional point cloud.
  • the generating module 340 further includes:
  • a first projection unit projecting the camera coordinate system onto a plane where each pixel point of the image to be corrected is located, to obtain the pixel coordinates of the preset coordinates in the pixel coordinate system;
  • the second projection unit is configured to map the pixel coordinates of the preset coordinates in the pixel coordinate system to the camera coordinate system to obtain the image sequence and the pixel coordinates corresponding to the image sequence;
  • the second generating unit is configured to obtain the pixel coordinates of the key frame based on the pixel coordinates corresponding to the image sequence.
  • FIG. 4 is a schematic structural diagram of a terminal device 400 provided by an embodiment of the present application.
  • the terminal device 400 includes a memory 410, at least one processor 420, and is stored in the memory 410 and can be stored in the processor 420.
  • the processor 420 executes the computer program 430, the above-mentioned three-dimensional reconstruction method is implemented.
  • the terminal device 400 may be a desktop computer, a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR)/virtual reality (VR) device, a notebook computer, an ultra mobile personal computer (ultra -Mobile personal computer (UMPC), netbook, personal digital assistant (personal digital assistant, PDA) and other terminal devices, the embodiment of this application does not impose any restrictions on the specific types of terminal devices.
  • AR augmented reality
  • VR virtual reality
  • UMPC ultra mobile personal computer
  • PDA personal digital assistant
  • the terminal device 400 may include but is not limited to a processor 420 and a memory 410. Those skilled in the art can understand that FIG. 4 is only an example of the terminal device 400, and does not constitute a limitation on the terminal device 400. It may include more or less components than those shown in the figure, or a combination of certain components, or different components. , For example, can also include input and output devices.
  • the so-called processor 420 may be a central processing unit (CPU), and the processor 420 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSP), and application specific integrated circuits (Application Specific Integrated Circuits). , ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory 410 may be an internal storage unit of the terminal device 400, such as a hard disk or a memory of the terminal device 400. In other embodiments, the memory 410 may also be an external storage device of the terminal device 400, such as a plug-in hard disk equipped on the terminal device 400, a smart media card (SMC), and a secure digital (Secure Digital). Digital, SD) card, flash card (Flash Card), etc. Further, the memory 410 may also include both an internal storage unit of the terminal device 400 and an external storage device. The memory 410 is used to store an operating system, an application program, a boot loader (Boot Loader), data, and other programs, such as the program code of the computer program. The memory 410 may also be used to temporarily store data that has been output or will be output.
  • a boot loader Boot Loader
  • the embodiments of the present application also provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in each of the foregoing method embodiments can be realized.
  • the embodiments of the present application provide a computer program product.
  • the steps in the foregoing method embodiments can be realized when the mobile terminal is executed.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the computer program can be stored in a computer-readable storage medium.
  • the computer program can be stored in a computer-readable storage medium.
  • the steps of the foregoing method embodiments can be implemented.
  • the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms.
  • the computer-readable medium may at least include: any entity or device capable of carrying computer program code to a terminal device, a recording medium, a computer memory, a read-only memory (Read-Only Memory, ROM), and a random access memory (Random Access).
  • Memory RAM
  • electrical carrier signals telecommunications signals
  • software distribution media For example, U disk, mobile hard disk, floppy disk or CD-ROM, etc.
  • computer-readable media cannot be electrical carrier signals and telecommunication signals.
  • the disclosed apparatus/network equipment and method may be implemented in other ways.
  • the device/network device embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units.
  • components can be combined or integrated into another system, or some features can be omitted or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Endoscopes (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un procédé de reconstruction tridimensionnelle pour une image d'endoscope monoculaire. Le procédé comprend : l'acquisition d'une pluralité d'images déformées, photographiées par un endoscope monoculaire, d'une cible d'étalonnage en damier, et la réalisation d'une correction de distorsion sur la pluralité d'images déformées de la cible d'étalonnage en damier pour obtenir une séquence d'images (S101) ; la détermination d'une trame clé à partir de la séquence d'images (S102) ; l'acquisition d'un paramètre de pose de la trame clé, et l'estimation d'une carte de profondeur de la trame clé (S103) ; et la réalisation d'une reconstruction d'image sur la base du paramètre de pose de la trame clé et de la carte de profondeur de la trame clé pour obtenir un nuage de points tridimensionnel (S104). L'invention concerne en outre un appareil de reconstruction tridimensionnelle (300) pour une image d'endoscope monoculaire, et un dispositif terminal (400). Une erreur provoquée par une distorsion d'imagerie d'un endoscope monoculaire est réduite, et l'effet d'affichage d'une image est également amélioré.
PCT/CN2020/129546 2019-12-12 2020-11-17 Procédé et appareil de reconstruction tridimensionnelle pour image d'endoscope monoculaire, et dispositif terminal WO2021115071A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911275140.9A CN111145238B (zh) 2019-12-12 2019-12-12 单目内窥镜图像的三维重建方法、装置及终端设备
CN201911275140.9 2019-12-12

Publications (1)

Publication Number Publication Date
WO2021115071A1 true WO2021115071A1 (fr) 2021-06-17

Family

ID=70518247

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/129546 WO2021115071A1 (fr) 2019-12-12 2020-11-17 Procédé et appareil de reconstruction tridimensionnelle pour image d'endoscope monoculaire, et dispositif terminal

Country Status (2)

Country Link
CN (1) CN111145238B (fr)
WO (1) WO2021115071A1 (fr)

Cited By (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113596432A (zh) * 2021-07-30 2021-11-02 成都市谛视科技有限公司 可变视角的3d视频制作方法、装置、设备及存储介质
CN113793413A (zh) * 2021-08-13 2021-12-14 北京迈格威科技有限公司 三维重建方法、装置、电子设备及存储介质
CN113838138A (zh) * 2021-08-06 2021-12-24 杭州灵西机器人智能科技有限公司 一种优化特征提取的系统标定方法、系统、装置和介质
CN113888715A (zh) * 2021-09-22 2022-01-04 中南大学 一种基于虚拟多目内窥镜的高炉料面三维重建方法及系统
CN113884025A (zh) * 2021-09-16 2022-01-04 河南垂天智能制造有限公司 增材制造结构光回环检测方法、装置、电子设备和存储介质
CN113902846A (zh) * 2021-10-11 2022-01-07 岱悟智能科技(上海)有限公司 一种基于单目深度相机和里程传感器的室内三维建模方法
CN113925441A (zh) * 2021-12-17 2022-01-14 极限人工智能有限公司 一种基于内窥镜的成像方法及成像系统
CN114119574A (zh) * 2021-11-30 2022-03-01 安徽农业大学 一种基于机器视觉的采摘点检测模型构建方法及采摘点定位方法
CN114155349A (zh) * 2021-12-14 2022-03-08 杭州联吉技术有限公司 一种三维建图方法、三维建图装置及机器人
CN114529613A (zh) * 2021-12-15 2022-05-24 深圳市华汉伟业科技有限公司 一种圆阵列标定板的特征点高精度坐标提取方法
CN114565739A (zh) * 2022-03-01 2022-05-31 上海微创医疗机器人（集团）股份有限公司 三维模型建立方法、内窥镜及存储介质
CN114723885A (zh) * 2022-04-06 2022-07-08 浙江大学 一种基于rgbd图像稠密三维重建的植物耐寒性分析的方法
CN114820935A (zh) * 2022-04-19 2022-07-29 北京达佳互联信息技术有限公司 三维重建方法、装置、设备及存储介质
CN114820787A (zh) * 2022-04-22 2022-07-29 聊城大学 一种面向大视场平面视觉测量的图像校正方法及系统
CN114862961A (zh) * 2022-04-13 2022-08-05 上海人工智能创新中心 标定板的位置检测方法、装置、电子设备及可读存储介质
CN114882058A (zh) * 2022-04-26 2022-08-09 上海人工智能创新中心 一种角点检测方法、装置及标定板
CN114926515A (zh) * 2022-06-08 2022-08-19 北京化工大学 基于时空域深度信息补全的红外与可见光图像配准方法
CN114972531A (zh) * 2022-05-17 2022-08-30 上海人工智能创新中心 一种标定板、角点检测方法、设备及可读存储介质
CN115100294A (zh) * 2022-06-29 2022-09-23 中国人民解放军国防科技大学 基于直线特征的事件相机标定方法、装置及设备
CN115115687A (zh) * 2022-06-24 2022-09-27 合众新能源汽车有限公司 车道线测量方法及装置
CN115914809A (zh) * 2022-09-27 2023-04-04 中南大学 一种幽光高温工业立体内窥镜
CN116439825A (zh) * 2023-04-14 2023-07-18 合肥工业大学 面向微创术中辅助决策的体内三维信息测量系统
CN116664766A (zh) * 2023-05-15 2023-08-29 重庆大学 一种4d stem图像数据处理方法和装置
CN117173342A (zh) * 2023-11-02 2023-12-05 中国海洋大学 基于水下单双目相机的自然光下移动三维重建装置及方法
CN117218089A (zh) * 2023-09-18 2023-12-12 中南大学 一种沥青路面构造深度检测方法
CN117237553A (zh) * 2023-09-14 2023-12-15 广东省核工业地质局测绘院 一种基于点云图像融合的三维地图测绘系统
CN117392329A (zh) * 2023-12-08 2024-01-12 中国海洋大学 一种基于移动式多光谱光度立体设备的使用方法
WO2024022062A1 (fr) * 2022-07-28 2024-02-01 杭州堃博生物科技有限公司 Procédé et appareil d'estimation de pose d'endoscope et support de stockage
CN117557660A (zh) * 2024-01-09 2024-02-13 北京集度科技有限公司 数据处理方法、装置、电子设备以及车辆
CN117893693A (zh) * 2024-03-15 2024-04-16 南昌航空大学 一种密集slam三维场景重建方法及装置
CN118442947A (zh) * 2024-07-08 2024-08-06 海伯森技术（深圳）有限公司 一种投影图案生成方法、工作距离确定方法及介质

Families Citing this family (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111145238B (zh) * 2019-12-12 2023-09-22 中国科学院深圳先进技术研究院 单目内窥镜图像的三维重建方法、装置及终端设备
CN111798387B (zh) * 2020-06-24 2024-06-11 海南大学 一种用于共聚焦内窥镜的图像处理方法及系统
CN111882657B (zh) * 2020-06-29 2024-01-26 杭州易现先进科技有限公司 三维重建的尺度恢复方法、装置、系统和计算机设备
CN111784754B (zh) * 2020-07-06 2024-01-12 浙江得图网络有限公司 基于计算机视觉的牙齿正畸方法、装置、设备及存储介质
CN111862120B (zh) * 2020-07-22 2023-07-11 苏州大学 一种单目slam尺度恢复的方法
CN111899345B (zh) * 2020-08-03 2023-09-01 成都圭目机器人有限公司 一种基于2d视觉图像的三维重建方法
CN112348869B (zh) * 2020-11-17 2024-08-16 的卢技术有限公司 通过检测和标定恢复单目slam尺度的方法
CN112330729B (zh) * 2020-11-27 2024-01-12 中国科学院深圳先进技术研究院 图像深度预测方法、装置、终端设备及可读存储介质
CN112261399B (zh) * 2020-12-18 2021-03-16 安翰科技(武汉)股份有限公司 胶囊内窥镜图像三维重建方法、电子设备及可读存储介质
CN114663575A (zh) * 2020-12-23 2022-06-24 日本电气株式会社 图像处理的方法、设备和计算机可读存储介质
CN112802123B (zh) * 2021-01-21 2023-10-27 北京科技大学设计研究院有限公司 一种基于条纹虚拟靶标的双目线阵相机静态标定方法
CN112907620B (zh) * 2021-01-25 2024-06-28 北京地平线机器人技术研发有限公司 相机位姿的估计方法、装置、可读存储介质及电子设备
CN112767489B (zh) * 2021-01-29 2024-05-14 北京达佳互联信息技术有限公司 一种三维位姿确定方法、装置、电子设备及存储介质
CN112927340B (zh) * 2021-04-06 2023-12-01 中国科学院自动化研究所 一种不依赖于机械摆放的三维重建加速方法、系统及设备
CN113223163A (zh) * 2021-04-28 2021-08-06 Oppo广东移动通信有限公司 点云地图构建方法及装置、设备、存储介质
CN113610887B (zh) * 2021-05-26 2024-08-13 江苏势通生物科技有限公司 胶囊内窥镜运动拍摄路径的确定方法、存储介质和设备
CN113569843B (zh) * 2021-06-21 2024-08-23 影石创新科技股份有限公司 角点检测方法、装置、计算机设备和存储介质
CN113724194B (zh) * 2021-06-22 2024-01-30 山东交通学院 一种发动机内窥式火焰测量系统及图像处理方法
WO2023007641A1 (fr) * 2021-07-29 2023-02-02 株式会社エビデント Dispositif de reconstruction tridimensionnelle, procédé de reconstruction tridimensionnelle et programme
CN113744410A (zh) * 2021-09-13 2021-12-03 浙江商汤科技开发有限公司 网格生成方法、装置、电子设备及计算机可读存储介质
CN114022527B (zh) * 2021-10-20 2024-09-20 华中科技大学 基于无监督学习的单目内窥镜深度及位姿估计方法及装置
CN116168143A (zh) * 2021-11-25 2023-05-26 华为技术有限公司 一种多视图三维重建的方法
CN114332028A (zh) * 2021-12-30 2022-04-12 小荷医疗器械(海南)有限公司 内窥镜图像的处理方法、装置、可读介质和电子设备
CN114677572B (zh) * 2022-04-08 2023-04-18 北京百度网讯科技有限公司 对象描述参数的生成方法、深度学习模型的训练方法
CN114972990A (zh) * 2022-05-27 2022-08-30 深圳市优必选科技股份有限公司 点云数据的标注方法、装置及终端设备
CN114782470B (zh) * 2022-06-22 2022-09-13 浙江鸿禾医疗科技有限责任公司 消化道的三维全景识别定位方法、存储介质和设备
CN115471556B (zh) * 2022-09-22 2023-11-14 南京博视医疗科技有限公司 一种单目相机图像目标点三维定位方法及装置
CN116704152B (zh) * 2022-12-09 2024-04-19 荣耀终端有限公司 图像处理方法和电子设备
CN115908366A (zh) * 2022-12-13 2023-04-04 北京柏惠维康科技股份有限公司 数据处理方法、装置、电子设备及存储介质
CN116452752A (zh) * 2023-04-28 2023-07-18 重庆理工大学 联合单目稠密slam与残差网络的肠壁重建方法
CN116958147B (zh) * 2023-09-21 2023-12-22 青岛美迪康数字工程有限公司 基于深度图像特征的目标区域确定方法、装置和设备
CN117115358B (zh) * 2023-10-11 2024-07-12 世优（北京）科技有限公司 数字人自动建模方法及装置
CN117765174A (zh) * 2023-12-19 2024-03-26 内蒙古电力勘测设计院有限责任公司 一种基于单目云台相机的三维重建方法、装置及设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103247075A (zh) * 2013-05-13 2013-08-14 北京工业大学 基于变分机制的室内环境三维重建方法
US20140111507A1 (en) * 2012-10-23 2014-04-24 Electronics And Telecommunications Research Institute 3-dimensional shape reconstruction device using depth image and color image and the method
CN108898630A (zh) * 2018-06-27 2018-11-27 清华-伯克利深圳学院筹备办公室 一种三维重建方法、装置、设备和存储介质
CN109448041A (zh) * 2018-10-29 2019-03-08 重庆金山医疗器械有限公司 一种胶囊内镜图像三维重建方法及系统
CN109544677A (zh) * 2018-10-30 2019-03-29 山东大学 基于深度图像关键帧的室内场景主结构重建方法及系统
CN111145238A (zh) * 2019-12-12 2020-05-12 中国科学院深圳先进技术研究院 单目内窥镜图像的三维重建方法、装置及终端设备

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019169540A1 (fr) * 2018-03-06 2019-09-12 斯坦德机器人（深圳）有限公司 Procédé de slam visuel à configuration groupée, terminal et support d'informations lisible par ordinateur
CN108416840B (zh) * 2018-03-14 2020-02-18 大连理工大学 一种基于单目相机的三维场景稠密重建方法
CN109087349B (zh) * 2018-07-18 2021-01-26 亮风台（上海）信息科技有限公司 一种单目深度估计方法、装置、终端和存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140111507A1 (en) * 2012-10-23 2014-04-24 Electronics And Telecommunications Research Institute 3-dimensional shape reconstruction device using depth image and color image and the method
CN103247075A (zh) * 2013-05-13 2013-08-14 北京工业大学 基于变分机制的室内环境三维重建方法
CN108898630A (zh) * 2018-06-27 2018-11-27 清华-伯克利深圳学院筹备办公室 一种三维重建方法、装置、设备和存储介质
CN109448041A (zh) * 2018-10-29 2019-03-08 重庆金山医疗器械有限公司 一种胶囊内镜图像三维重建方法及系统
CN109544677A (zh) * 2018-10-30 2019-03-29 山东大学 基于深度图像关键帧的室内场景主结构重建方法及系统
CN111145238A (zh) * 2019-12-12 2020-05-12 中国科学院深圳先进技术研究院 单目内窥镜图像的三维重建方法、装置及终端设备

Cited By (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113596432A (zh) * 2021-07-30 2021-11-02 成都市谛视科技有限公司 可变视角的3d视频制作方法、装置、设备及存储介质
CN113596432B (zh) * 2021-07-30 2024-04-30 成都市谛视科技有限公司 可变视角的3d视频制作方法、装置、设备及存储介质
CN113838138A (zh) * 2021-08-06 2021-12-24 杭州灵西机器人智能科技有限公司 一种优化特征提取的系统标定方法、系统、装置和介质
CN113793413A (zh) * 2021-08-13 2021-12-14 北京迈格威科技有限公司 三维重建方法、装置、电子设备及存储介质
CN113884025A (zh) * 2021-09-16 2022-01-04 河南垂天智能制造有限公司 增材制造结构光回环检测方法、装置、电子设备和存储介质
CN113884025B (zh) * 2021-09-16 2024-05-03 河南垂天智能制造有限公司 增材制造结构光回环检测方法、装置、电子设备和存储介质
CN113888715A (zh) * 2021-09-22 2022-01-04 中南大学 一种基于虚拟多目内窥镜的高炉料面三维重建方法及系统
CN113888715B (zh) * 2021-09-22 2024-07-30 中南大学 一种基于虚拟多目内窥镜的高炉料面三维重建方法及系统
CN113902846A (zh) * 2021-10-11 2022-01-07 岱悟智能科技(上海)有限公司 一种基于单目深度相机和里程传感器的室内三维建模方法
CN113902846B (zh) * 2021-10-11 2024-04-12 岱悟智能科技(上海)有限公司 一种基于单目深度相机和里程传感器的室内三维建模方法
CN114119574A (zh) * 2021-11-30 2022-03-01 安徽农业大学 一种基于机器视觉的采摘点检测模型构建方法及采摘点定位方法
CN114119574B (zh) * 2021-11-30 2024-08-13 安徽农业大学 一种基于机器视觉的采摘点检测模型构建方法及采摘点定位方法
CN114155349A (zh) * 2021-12-14 2022-03-08 杭州联吉技术有限公司 一种三维建图方法、三维建图装置及机器人
CN114155349B (zh) * 2021-12-14 2024-03-22 杭州联吉技术有限公司 一种三维建图方法、三维建图装置及机器人
CN114529613A (zh) * 2021-12-15 2022-05-24 深圳市华汉伟业科技有限公司 一种圆阵列标定板的特征点高精度坐标提取方法
CN113925441B (zh) * 2021-12-17 2022-05-03 极限人工智能有限公司 一种基于内窥镜的成像方法及成像系统
CN113925441A (zh) * 2021-12-17 2022-01-14 极限人工智能有限公司 一种基于内窥镜的成像方法及成像系统
CN114565739A (zh) * 2022-03-01 2022-05-31 上海微创医疗机器人（集团）股份有限公司 三维模型建立方法、内窥镜及存储介质
CN114723885A (zh) * 2022-04-06 2022-07-08 浙江大学 一种基于rgbd图像稠密三维重建的植物耐寒性分析的方法
CN114862961A (zh) * 2022-04-13 2022-08-05 上海人工智能创新中心 标定板的位置检测方法、装置、电子设备及可读存储介质
CN114862961B (zh) * 2022-04-13 2024-06-07 上海人工智能创新中心 标定板的位置检测方法、装置、电子设备及可读存储介质
CN114820935A (zh) * 2022-04-19 2022-07-29 北京达佳互联信息技术有限公司 三维重建方法、装置、设备及存储介质
CN114820787B (zh) * 2022-04-22 2024-05-28 聊城大学 一种面向大视场平面视觉测量的图像校正方法及系统
CN114820787A (zh) * 2022-04-22 2022-07-29 聊城大学 一种面向大视场平面视觉测量的图像校正方法及系统
CN114882058A (zh) * 2022-04-26 2022-08-09 上海人工智能创新中心 一种角点检测方法、装置及标定板
CN114882058B (zh) * 2022-04-26 2024-06-07 上海人工智能创新中心 一种角点检测方法、装置及标定板
CN114972531A (zh) * 2022-05-17 2022-08-30 上海人工智能创新中心 一种标定板、角点检测方法、设备及可读存储介质
CN114926515B (zh) * 2022-06-08 2024-05-14 北京化工大学 基于时空域深度信息补全的红外与可见光图像配准方法
CN114926515A (zh) * 2022-06-08 2022-08-19 北京化工大学 基于时空域深度信息补全的红外与可见光图像配准方法
CN115115687A (zh) * 2022-06-24 2022-09-27 合众新能源汽车有限公司 车道线测量方法及装置
CN115100294A (zh) * 2022-06-29 2022-09-23 中国人民解放军国防科技大学 基于直线特征的事件相机标定方法、装置及设备
WO2024022062A1 (fr) * 2022-07-28 2024-02-01 杭州堃博生物科技有限公司 Procédé et appareil d'estimation de pose d'endoscope et support de stockage
CN115914809A (zh) * 2022-09-27 2023-04-04 中南大学 一种幽光高温工业立体内窥镜
CN116439825A (zh) * 2023-04-14 2023-07-18 合肥工业大学 面向微创术中辅助决策的体内三维信息测量系统
CN116664766A (zh) * 2023-05-15 2023-08-29 重庆大学 一种4d stem图像数据处理方法和装置
CN116664766B (zh) * 2023-05-15 2024-07-12 重庆大学 一种4d stem图像数据处理方法和装置
CN117237553A (zh) * 2023-09-14 2023-12-15 广东省核工业地质局测绘院 一种基于点云图像融合的三维地图测绘系统
CN117218089A (zh) * 2023-09-18 2023-12-12 中南大学 一种沥青路面构造深度检测方法
CN117218089B (zh) * 2023-09-18 2024-04-19 中南大学 一种沥青路面构造深度检测方法
CN117173342A (zh) * 2023-11-02 2023-12-05 中国海洋大学 基于水下单双目相机的自然光下移动三维重建装置及方法
CN117392329B (zh) * 2023-12-08 2024-02-06 中国海洋大学 一种基于移动式多光谱光度立体设备的使用方法
CN117392329A (zh) * 2023-12-08 2024-01-12 中国海洋大学 一种基于移动式多光谱光度立体设备的使用方法
CN117557660A (zh) * 2024-01-09 2024-02-13 北京集度科技有限公司 数据处理方法、装置、电子设备以及车辆
CN117557660B (zh) * 2024-01-09 2024-04-12 北京集度科技有限公司 数据处理方法、装置、电子设备以及车辆
CN117893693B (zh) * 2024-03-15 2024-05-28 南昌航空大学 一种密集slam三维场景重建方法及装置
CN117893693A (zh) * 2024-03-15 2024-04-16 南昌航空大学 一种密集slam三维场景重建方法及装置
CN118442947A (zh) * 2024-07-08 2024-08-06 海伯森技术（深圳）有限公司 一种投影图案生成方法、工作距离确定方法及介质

Also Published As

Publication number Publication date
CN111145238A (zh) 2020-05-12
CN111145238B (zh) 2023-09-22

Similar Documents

Publication Publication Date Title
WO2021115071A1 (fr) Procédé et appareil de reconstruction tridimensionnelle pour image d'endoscope monoculaire, et dispositif terminal
US7733404B2 (en) Fast imaging system calibration
WO2020259271A1 (fr) Procédé et appareil de correction de la distorsion d'image
CN101630406B (zh) 摄像机的标定方法及摄像机标定装置
WO2021139176A1 (fr) Procédé et appareil de suivi de trajectoire de piéton sur la base d'un étalonnage de caméra binoculaire, dispositif informatique et support de stockage
US8447140B1 (en) Method and apparatus for estimating rotation, focal lengths and radial distortion in panoramic image stitching
WO2021136386A1 (fr) Procédé de traitement de données, terminal et serveur
US20190385285A1 (en) Image Processing Method and Device
WO2022095596A1 (fr) Procédé d'alignement d'image, appareil d'alignement d'image et dispositif terminal
CN113516719B (zh) 一种基于多单应性矩阵的相机标定方法、系统及存储介质
CN112470192A (zh) 双摄像头标定方法、电子设备、计算机可读存储介质
CN111383264B (zh) 一种定位方法、装置、终端及计算机存储介质
Daftry et al. Flexible and User-Centric Camera Calibration using Planar Fiducial Markers.
CN113379815A (zh) 基于rgb相机与激光传感器的三维重建方法、装置及服务器
US20220405968A1 (en) Method, apparatus and system for image processing
JP6086491B2 (ja) 画像処理装置およびそのデータベース構築装置
CN111260574B (zh) 一种印章照片矫正的方法、终端及计算机可读存储介质
CN112907657A (zh) 一种机器人重定位方法、装置、设备及存储介质
CN109902695B (zh) 一种面向像对直线特征匹配的线特征矫正与提纯方法
CN112102378A (zh) 图像配准方法、装置、终端设备及计算机可读存储介质
JP2018036884A (ja) 光源推定装置及びプログラム
WO2022174603A1 (fr) Procédé de prédiction de pose, appareil de prédiction de pose, et robot
CN112634377B (zh) 扫地机器人的相机标定方法、终端和计算机可读存储介质
CN113361400B (zh) 一种头部姿态估计方法、装置及存储介质
JP3452188B2 (ja) ２次元動画像中の特徴点の追跡方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20897788

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20897788

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 20.01.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20897788

Country of ref document: EP

Kind code of ref document: A1