TW200807309A - Method and system for reconstructing 3-D endoscopic images - Google Patents

Abstract

A method and a system for reconstructing three-dimensional (3-D) endoscopic images are disclosed. The present invention uses the projection relation between an endoscopic camera and a lesion area, and combines a 3-D tracker fixed on a surgical instrument to reconstruct a 3-D structure of the observed area. After correcting endoscopic images distortion, the steps of extracting and tracking features is performed, and then the shape of the image is reconstructed by using matrix decomposition via the fact that image projection points are obtained from the multiplication of a camera projection matrix and 3-D coordinates. The present invention adds an additional dimension to the original 2D endoscopic images to compensate for loss of depth problem in regular endoscopic systems.

Description

200807309 IX. Invention Description. ...... : - · · : - : :···· Ϊ··.:;.:. [Technical Field of the Invention] The present invention relates to a reconstruction of a three-dimensional image. The method, in particular, is directed to a method of reconstructing a three-dimensional image of an endoscope. L first rain technology] During the operation, the doctor must confirm the relative position of the surgical guard and the surrounding tissue to ensure the quality of the operation, but in the endoscopic surgery, the doctor can only see through the lens! A two-dimensional image is used to observe the structure of the body. 'Physicians can't immediately recognize the relationship between the size and the size of the space and the relationship between the three dimensions, so that the risk of increasing the 35-course procedure is related to the diagnosis and treatment of the auxiliary physician. It has been published 'for example: based on computed tomography (CT) images, applied to the detection of colorectal-related diseases and the fixation of three-dimensional snoring devices on endoscopes, in order to: 'to go' or - ray The age-appropriate two-dimensional image of the Lu 円 円 与 与 与 与 10 10 10 10 10 10 10 10 10 10 10 10 建 建 建 建 建 建 建 建 建The existing virtual endoscope system of the 忒& field is based on electrical images, and it is necessary for the professional to take the stomach or the magnetic resonance. Not only the person who knows the 仏格印贝, but also the manpower Material cost and needle probability. At present, Dong Tianyi and A have suffered from the injury of the file--the surgical aid of the sputum-tracking device, the medical learning ceremony I obtained before A, and the commercial image of the singer Basics, there are also medical costs or therefore 'needs to be a method and device for reconstructing 3D images, 2, ''',: the dynamics of the surgical tool in the body and the stereoscopic view of the lesion area to instantly get the physician 14 The virtual environment wanders the model of the auxiliary tool to become a clinical diagnosis and treatment. 5 200807309 [Invention] The object of the present invention is to provide a three-dimensional reconstruction method and a three-dimensional model by which a normal scale is established. Provide physicians with accurate 3D 1 information to reduce the risk of surgery and save on medical costs and reduce the risk of radiation damage to patients. The invention can also reconstruct a pre-recorded endoscope image sequence and a plurality of real three-dimensional coordinates into a three-dimensional model of a normal scale for evaluation and education training in surgery or treatment. According to a preferred embodiment of the present invention, the reconstructed 3D image of the present invention includes at least: obtaining a two-dimensional image by using an endoscope, and the two-dimensional image is subjected to a step of acquiring a feature. The feature of the two-dimensional image is: the sign is filtered to remove the unreliable track step' to filter out the feature tracking second step: the feature point of the tracking error. For the trajectory of feature points, this is the case: Γ::matrix' constructs a distorted three-dimensional model by affine reconstruction. + ^ ^ Γ dimension model 'Euclidean reconstruction method for 3D model correction ^ The first degree of anomaly 3D model, and obtain the first - moving parameter matrix to obtain the first: , with =1 - silk value. The sensor is fixed to the surgical tool and the surgical tool is used to determine the array between the tracker main body, the sensor, and the sensing point. == This obtains the -to-transformation moment value provided by the tracker device. A third coordinate point is obtained by calculating the hand odor matrix and the second coordinate value, so that at least four corresponding features are exchanged at any point of the feature currency, and the third coordinate value is used to change the matrix. A second-degree anomaly three-dimensional model is used to obtain a sixth coordinate value. 6 200807309 A comparison and calculation step is performed on the 'th, coordinate value and the fifth coordinate value to obtain a second conversion matrix. Using the third transformation matrix, the first coordinate value is converted into a sentence; the value is used, and the inverse matrix of the third transformation matrix (G) is used to convert: a movement parameter matrix (M) into a second movement. The parameter matrix is a three-dimensional model of the world of poverty. Using the world coordinate endoscope movement parameter matrix and the feature point world coordinate values to calculate the scale measurement matrix (the multiple precision factors in the 趵, and update the accuracy factor and the scale measurement matrix: "Compare the accuracy factor before updating) The difference between the value and the updated value of the precision f, if the difference is less than a predetermined threshold, the two-dimensional image of the world seat and the quasi-model is obtained, and if the difference is greater than the preset threshold, Then, the scale measurement matrix has been updated (the step of degenerating the affine reconstruction step until the difference between the value of the dominant factor before the update and the value of the updated accuracy factor is less than a preset threshold. 'According to the present invention In another preferred embodiment, a plurality of additional feature points are selected by an arbitrary point in an area to be observed, and the additional feature points are projected to the two-dimensional image to obtain a plurality of additional feature point tracks. Some additional.: The eigen-point trajectory is incorporated into this scale measurement matrix (7)' to make this scale measurement matrix (7)' a feature point-augmented matrix ("Sigh π and construct this with a affine reconstruction method for this feature point augmentation matrix" Twist According to still another preferred embodiment of the present invention, a human-wide-angle image correcting step is performed to correct image distortion generated by the wide-angle lens before performing the feature tracking and digging step, the wide-angle image correcting step At least including: • normalizing the brightness of the two-dimensional image (intensiiyn〇rmalizati〇n), and performing binarization on the two-dimensional image subjected to brightness normalization processing by high-pass filter (four)h and threshold division method To distinguish between the foreground and the previous 7200807309 scene, and take out the plurality of centers of gravity of the graphics on the two-dimensional image; close the closing of the image processing morphology to the two-dimensional image that has been binarized. The operation fills the gap caused by the binarization processing; and, for the plurality of lines of the center of gravity, the linear distortion is corrected by the method for calculating the linearity error. According to a preferred embodiment of the fork of the present invention, This feature tracking and extraction step is performed by the Kaim4e-Lixaas_Tomasi (KLT) feature tracking method and the edge feature (Edge Featm (4).) φ According to this hair In an embodiment, the step of filtering the unreliable track includes: at least a point of the A. sign in a first two-dimensional image, a second two-dimensional image, and a third two-dimensional image, to obtain the feature point located at the first a first feature of the two-dimensional image, a coordinate value, a second feature point coordinate value, and a third feature point coordinate value; the vector represented by the first feature point to the second feature point is a first vector and a second feature point The vector represented by the third feature point is a second vector, and the angle between the second food quantity and the first vector is a test angle. If the test angle is a preset threshold, the filter value is filtered. The second vector. _ [Embodiment; 3 please refer to the vine 2 recommendation, which shows the system diagram of the three-dimensional image of the reconstructed endoscope of the present invention. The system tracker device 20 and the endoscope device 30 of the present invention are comprised; the 肀 chaser device 20 includes a transmitter 2 tracker host 22, an ink 24, and an inductor 26 mounted to the surgical tool 28. The endoscope strip of the present invention includes an endoscope 34 and an endoscope main unit 32. The tracker device 20 of the present invention is manufactured by, for example, Ascension Technology, Corp., VA, USA, 8 200807309

miniBIRD 3D tracker. When applying the system of the present invention, during endoscopic surgery, the physician can observe the internal structure of the body by extending the endoscope 34 into the body, and use the immobilized spring 263⁄4 surgical tool 28 to treat the lesion. deal with. Since the tracker main unit 22 can capture the electromagnetic signal emitted by the transmitter 21 through the beetle turtle device 26 to calculate the stray position, the tracker main unit 22 can be directly connected to the computer 24, and the surgical tool can be obtained immediately. 2 8 Position in 3D space. Under the & raise, a physician can take a thin and violent word on the endoscope 35 on the area to be observed. Then, the system of the present invention reconstructs the three-dimensional image structure of the sculpt by using the image sequence and the plurality of three-dimensional coordinates obtained by the tracking device 20 to detect the observation area, and simultaneously provides the orientation of the surgical tool. Weng Yibao let. In order to realize the establishment of a three-dimensional chamber junction meal T, the method for reconstructing the three-dimensional image of the endoscope of the present invention includes steps of image deformation correction, feature point extraction and tracking, and decomposition method combining three-dimensional coordinates. Please refer to FIG. 2A, which is a flow chart showing a method of reconstructing a three-dimensional image of a mirror according to a preferred embodiment of the present invention. First, perform image distortion correction travel 100. Usually, the endoscope is equipped with a wide-angle lens, so the endoscope cost: the two-dimensional image in the observation area of Qin ^ will appear geometric distortion distortion phenomenon, the chain I emperor will be imaged as a curve after taking the image through the endoscope. And this phenomenon is most evident in the image green. Therefore, before performing the three-dimensional reconstruction of the endoscope image, the image layer must first be sold to correct the wide-angle image to remove the wide-angle lens to bring the old Qin deformation. First, prepare a calibration plate printed with & multiple equal-pitch dots. “Pre-processing the image. Because the endoscope only uses a single light source fixed to the lens plus a wide-angle lens to scatter light, usually close to the middle of the image. The brightness of the area will be higher than the periphery of the image. 200807309 Therefore, it is necessary to perform the Intensity Normalization process on the acquired corrected image. The following describes the brightness normalization process: Suppose the size of the 2D image is fTx孖, let /(/,/) be the brightness of (ί,/)嵩, /avg is the average brightness of the entire image, if it is raised at each point GJ) Τ 〇, define a window (mow) of size mxm, then normalize the bang of the vines to the corpse:

1 mil mil mxmu=_m/2v=_m/2

Next, the image is binarized by High Pass Filter and Thresholding to divide each pixel in the 2D image into foreground and background, and image processing The Closing operation of Morphology fills the gap. After the raising of the image, the center of gravity of each dot on the image 0 is used as a correction point. With these correction points, the line of the poison is m, by C. Brauer-Burchardt, 'Automatic Corredimi Weak Radial Lens Distortion in Single Views of Urtea^ Scenes Using Vanishing In the book "Points", calculate the linearity error _: language to correct the deformation image. First, define the deformation correction transformation as follows: r = rW(l + d2ra+d4f4) (2) where, the tip, <is an unknown parameter, r And 〆 are the distances of the fog image points before and after correction to the center of the image. Then, the correction points on the same line are selected, and the connection of these points is originally a straight line, which is now changed by the endoscope image: 200807309 Take three consecutive correction points as a group, and use the area of the square as the error function, and then add the parameter to the triangle area and the minimum time of all points. Then, perform feature extraction and tracking. Steps, shape parameters. Capture the 2D image to perform the feature point ', with the result of the endoscope, the scene will immediately ring to = only ^ vertical 'this feature capture and chase, Poem According to this (four) degree, please refer to the flow chart of the first step. First of all, the characteristics of the operation and tracking of the endoscope mirror image before the virtual image _ 202a. By (four) mirror to shoot the human 4 helmet Ming & do a few ^ ^ S organization, its color texture can be ",, m" 'adverse feature extraction and chasing, because r (Hist °g_E. (4) like Γ: special:. In addition 'endoscope image When the light source and the lens:: display; = not: the most common situation is the edge area of the image: - After the average of the shirt, although the details are strengthened, this bright phenomenon is more Seriously, the present invention overcomes this shortcoming by the above-mentioned image deformation correction technique. Thus, the phenomenon that the image features are still unevenly distributed has been removed. Point: Ben: d202Lb, in order to track and manipulate the feature method of two-dimensional image _ angle = fr Γ use urTomasi (KLT) feature to track 縱 縱 冉隅 characteristics, but when applied to the endoscope image sequence, the beauty characteristics are physiological organization, so it is not easy Find obvious corner features. Therefore,:

The monthly test and the edge feature of the bond are measured to effectively increase the special X KLT feature tracking algorithm: The following two features of the feature tracking algorithm can be divided into angle extraction and feature tracking 11 200807309 (1) First, the first image of the image sequence is captured, the image is grayed out, and Gaussian smoothing is used.

Low-Pass Filter) removes noise interference and calculates the brightness gradient image in the horizontal and vertical directions of the image: Then 'create a covariance matrix cw for each image point (zV): Σ qsw c

Sa(Ix(Cl) ) Sa(Ix((1)Iy(q)) S σ(Ι x(q)I y(q)) ^a(^y(q)2) (where w is (6) ) is the center of the sub-window; & is a Gaussian filter from 1 to find the two eigenvalues of the cw matrix (Eigenvaiues) ei, e2. The smaller eigenvalue is greater than a predetermined threshold, then Xi:-A corner feature. With this judgment, a good initial feature point can be found in the first image. (2) Feature tracking 5 According to the initial feature point found in the previous step, calculate the next feature 2 The position. Suppose there are currently two images 〇, 纟中/Image Special 2: Known, and will be used to track their corresponding points in the 7 image features. / Like / 4 Temple sign X, after displacement D = ^ After ^ see in the image, the tracking error S can be defined as: \[U{X + D)^I{X)f0)^X)dX (4) Point ί中"" is the point J The center's search window, defined as the feature's right-handed heart--appropriate size area, specifies that the amount of points in the search window can usually be set to U as a Gaussian distribution. If the displacement is evenly distributed, then Change to: 12 200807309 (5) ί ί [J(<X + f > - 7(χ ~ ^)Ϋ w(X)dx

Solve the & difference function, the error value I, the best displacement value solution for the car _ soil β ί, 瑕 犄 & quot Μ Μ Μ Μ Μ Μ Μ Μ Μ 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠 叠= D(i) + must be: to update the tracking error until convergence, ... is the number of iterations. On the convergence date, the tracking error 4 is the minimum 'and the displacement ", the position of the next image. The bell tube flute, after the corresponding position of the image, can be found with the second image feature point in the same way as the strokes of the image, and the above steps are repeated at the corresponding points of the three images of the younger brother. An image can be traced to the image sequence. The following describes the steps for detecting the edge features of step 2 〇2 b: Φ After completing the image pre-processing, calculate the optimal threshold (〇pUmai plus (4) one to divide the image into foreground and background. The valve actually used in the present invention The value is (1 - coffee. 〆 to avoid undersampling, #中伙 - an empirically defined parameter. In addition, if more than half of the neighborhood of a foreground pixel is the background, the pixel is also marked as the background. The invention According to the above steps, the foreground pixel is defined as a feature mask, and the image is subjected to high-pass filtering: (High-Pass FUter) for a convolution (c〇nv〇iuti〇n), if a pixel After high-pass filtering is greater than a predefined threshold and is located in the feature mask, it is marked as Edge Candidate. Then, the high-pass filtered values of these candidate points are sorted from large to small, and Pushing a stack (Stack) The pixel with the largest value is at the top of the stack (τ〇ρ), and the minimum value is at the bottom of the stack (Bottom), each time it is taken from the stack (ρ〇ρ) , check this point neighborhood (the size defined by the user) Region) Are there edge feature point, if no, this point is a feature edge, and repeat the pixel access - Neighborhood Check - labeled step 13200807309 edge points in mind, the stack until clearance.

m After completing step 202b, KLT feature tracking is performed by the above (2) feature tracking method (step 202c). Then, step 202d is performed to filter the unreliable traces of the faulty K. Referring to FIG. 2C, a schematic diagram of the step of filtering the unreliable traces according to the embodiment of the present invention is shown. The image, Pi and Pi + s represent the corresponding points of the dipoles corresponding to the corresponding points of the i, i and i + s sings, s is an integer; VAB is a vector from pi_s to pi; Υηχ: is from Pi to Pi + s vector, if the angle between Vab and VBC is not large, the trace of the feature points to be tested after the is, i, and i + s images is T, if the angle 0 is too large, it means The change track of the measured feature points is too large and untrustworthy, so it must be eliminated. The acceptable range of angle Θ is determined by the value defined in advance. Please continue to refer to Figure 2A. After step 202 (202d) is completed, step 204 is performed to trace the two-dimensional coordinates of the feature represented by the image coordinate system, and the internal matrix of the Λ綦 Λ綦 mirror is converted into the value of the internal coordinate system coordinate system. And to establish a scale measurement matrix 1 - 1 uip vn ··· vip ” ·: (6)

Uf\ ...ujp

Yfx ··· vjp__lfxP where %2❼),- is 0)·(ι+~), the feature of the feature point captured and traced by the brother factory is at the coordinates of the fth image, brother and sister) The center of gravity of the set of feature points is projected on the image coordinates of the image of the image of the Ζ Ζ 张 2008 2008 2008 2008 2008 2008 2008 2008 2008 2008 2008 2008 2008 。 。 。 。 。 。 。 。 。 。 。 。 。 / and corpse are the number of images and feature points. Each value in the matrix has a corresponding %, % is the accuracy factor of the reconstructed 3D model, and will be approximated by the Iterati^ reading method, the first constructed scale measurement matrix r The date is zero. Perform affine reconstruction 爹^ 2Θ^, for the scale measurement matrix (酽), to reconstruct and construct the 3D model by affine transformation. The affine reconstruction system uses the singular value decomposition method to measure the scales. The φ product of the matrix with three ranks (Rank) is the affine movement parameter of the mirror (Affine)

Motion Matrix); Affine Shape Matrix. The 3D model created by the affine reconstruction has a distortion phenomenon, so it is necessary to modify the model by performing the Euclidean reconstruction. Perform the Euclidean reconstruction step 2 0 8 to correct the three-dimensional model of this distortion. Since =Μ·left=ΜϋΆ, it is necessary to calculate the matrix ugly to convert the endoscopic affine movement parameter i and the affine three-dimensional coordinate cloud of the lane sign to the endoscope mirroring parameter of the decomposition coordinate system (Euclidean) Moti (5n Matrix) M (the first moving parameter matrix) and the Euclidean Shape Matrix S (the first one), ie, the edge and $ = ugly - 1. left, where the decomposition coordinate system The focus of each feature point is the stupid Euclidean Coordinate SystsE; the three-cone coordinate S is the set of feature point coordinates under the decomposed coordinate system. In wins, the cover anses et al. in "Robust Factorization" The method proposed in the book is to calculate the conditional ugly matrix so that the above steps are the Euclidean reconstruction. The three-dimensional model scale constructed by European reconstruction is abnormal to the real world model. Therefore, the present invention provides a combination of three-dimensional The coordinate decomposition method enables the 3D model of the scale anomaly generated after the Euclidean reconstruction to be restored to the original 15 200807309. The normal scale is continued. Please continue to refer to Figure 1 and Figure 2A. After completing step 202 (202d), perform the steps. 210, In the two-dimensional image, in the Weng Yangzheng, using the surgical tool 28 equipped with the sensor 26, at least four corresponding feature points corresponding to the feature points of the two-dimensional image are selected in the observation area, and converted by a conversion Steps, obtaining the world coordinate values of the corresponding feature points, and the origin of the world coordinate system in which the world coordinate value of the winning point is overlapped is the origin defined by the chase cloud *2# (in this embodiment, The transmitter 21 is ft) for this purpose. The sensor 26 is fixed to the handle end of the surgical tool 28, and the coordinate value selected by the hand leaner 2S 1S end and the world coordinates of the sensor are not 1 Therefore, if the world coordinate value of the corresponding feature point is to be obtained, the coordinate value of the coordinate value selected by the top of the surgical tool 28 and the world coordinate sensed by the sensor is defined. The top point of the surgical tool 28 is defined. The coordinate value of the selected coordinate value is the sensor coordinate system, and the sensor coordinate is to move the origin of the world coordinate system to the coordinate system of the sensor 26. It is assumed that the conversion relationship of the sensor coordinate mark coordinate system is , among them, for the hand Weng Gong 28 The sensor coordinates of any point in the space are selected to the extravagant. The turtle stupid cover vector is the vector from the origin of the sensor coordinate system to the sensing point. 4—The world coordinate vector of the sensing point is the world coordinate vector of the sensing point. The origin of the axe in the axe to the vector of the sensing point; the matrix J represents the mutual conversion relationship between the sensory seat and the world coordinate system of the parasitic wife. Here, the name is called the induction image matrix. Since the tracker device 20 knows the induction The device 26 is located at the three-dimensional position of the fourth production port, and the rotation matrix thereof is obtained, so that the sensor coordinates # 换 matrix χ can be obtained, and since the sensor 26 is fixed to the handle end of the surgical tool 28, the surgical tool 16 200807309 28 The position of the top end relative to the sensor 26 is fixed, so the sensor coordinate value of any point in the door is a fixed value, that is, 3 ^ value. From the above, it can be seen that if the fixed value of the household is obtained, and then the sensor coordinate conversion matrix j provided by the tracker is used, the sensing point selected by the surgeon can be expressed in the world coordinate system. In order to obtain a sensor point, select the sensor point selected by the surgical tool 28 to obtain the world coordinate value (the second coordinate value) of the Xin painting, and then use the tracker system to extract the sensor: 26 The sensor coordinate conversion matrix a at the sensing point is the first to change the sensor coordinate value (the third highest value) of the sensing point of the fixed value. From the above description, it can be known that the sensor coordinate conversion matrix (second conversion matrix) corresponding to each pair of points and the turtle coordinate value (third coordinate value) of the fixed value can be used. The sensor coordinate value corresponding to the corresponding feature point selected by the surgical tool 28 (the first 祆 value) is converted to the 襟 value (the fifth coordinate value). , * In step · · step 212, corresponding These corresponding feature points are in the three umbrella model of scale anomaly' to obtain this m (the sixth coordinate system is in the decomposition coordinate system, and the eight solutions are in the decoupling system.鬌 幸 r r = value 'to obtain - three times three coordinate system bar _ Jin G (di sin transformation matrix), and the calculation of the sub-section of the old 之 之 之 之 特征 特征 5韵尹Α/Γ _ Λ/ί ^ 1 , brother and dynamic parameter matrix Af converted from y μ μ = Μ σ to the world seat 酽 = HJL in the knowledge of not to be, then the world coordinates three to raise A. The product of „ r ,ffi . and $ is still equal to the scale measurement matrix, and f is the rotation of the world coordinates _ A ^ ^ ^ ^...1 ^ The set (seventh coordinate value) has the same size as the object of the real world; f is the world coordinate system 17 200807309 endoscope movement parameter matrix (second moving parameter matrix). In order to find the furnace system conversion matrix σ' assumes that the feature point %=[j^ f in the decomposition coordinate system can be converted to the world coordinate system by the displacement of the ra and then the matrix coordinate Ga is ςJ[iJ /, % is the second / second of the two-dimensional image The feature point is located, the solution, the coordinate value, the z-th feature point of the two-dimensional image is located in the world m coordinate value; Gfl is -3x3 matrix; ?; is -3χΐ matrix. Therefore, after Euclidean reconstruction That is to say, the feature point world coordinates obtained by the three-dimensional tracer and its estimated three-dimensional coordinates w«, substituted into i Γα].γ, get the simultaneous equations, to Nong Xiaoping: Calculate the matrix I []. Finally, put ~displacement, fl, then the matrix Ga is equal to the coordinate system transformation matrix G. The displacement & will be compensated back after the completion of the reconstruction. In fact, I r , Λ 上 丨For an affine transformation (Affine T (10) sformation), in addition to the above coordinate conversion In addition, it also contains a modified shape to make 2 near-actual shape functions, and the affine transformation assumes that the difference in shape correction value per point is not large enough - the overall conversion to represent the ^ point in the space division is scattered, then this conversion may not be applicable. If W's life tends to change the moment... the rank number is not three': Table::: Find the coordinate system to turn ^^, ', and D. The track with t trace error is not filtered out. Therefore (4) Re-record The step of the trace (2〇2d). In step 216, the matrix of the set feature point world coordinates & is expressed as: (7) ^ (:j) = lqwi^Ta if qwi exists, [^(· 5 〇otherwise. : The specials under ::4, and the endoscopic shift parameter matrix raises each of the different scales to measure the matrix r to update the value of the accuracy factor and measure the moment ", again, due to 18 200807309 1 rot' = (- Fresh (j), so there will be two sets of precision factor values corresponding to (MJ) and (-paper J), and the back projection error is small.

In step 218, if the difference between the updated value and the initial value is less than the preset threshold, the accuracy of the three-dimensional model of the world coordinate is met, and the three-dimensional image of the three-dimensional model of the world coordinate is obtained. The image has a normal size. If the accuracy factor is updated and the difference before being updated is greater than the preset: threshold, then return to step 206' to measure the moment p of the updated scale without re-entering the affine reconstruction step, and repeat the cycle until accurate The difference between the updated value of the degree factor and the value before being updated is less than the preset threshold. The method of repeatedly updating the three-dimensional model of the world coordinate is called the iterative method, wherein the step 2 ι only needs to be performed once, that is, the world coordinate value of the feature points required in each step of the step 212 is the same, . In the process of surgery, the lesion may be unevenly covered by other tissues or beyond the scope of the image, so that the reconstructed three-dimensional model lacks feature points in important areas to describe the geometric shape of the region. On the other hand, when the three-dimensional coordinate of the feature point is selected by the surgical tool 34, ^: the tool 34 must accurately point to the feature point to avoid an error. If any point =, it is not limited to the previously extracted feature point, which can be greatly increased. Click on the efficiency, = add geometric details, and improve the reconstruction accuracy. These can be arbitrarily selected and added to the refactoring process in the following steps. After three-dimensional reconstruction, the additional feature points are reconstructed from the two-dimensional material by using the endoscope movement parameter matrix under the world coordinate system to form a new obstruction «), and merged with the original two-track and heart) , can get an augmented scale measurement, matrix 200807309 Π 1 vn

xmgW ufi lfP Uf\ v^ffn u

V ff 'team, ff wind' 2/x(P+A) (δ) t where 乂 is the number of additional feature points ·, augmented scale measurement matrix awg 『 tea coloring for affine reconstruction, Euclidean reconstruction And iterative method to reconstruct the 3D model, which can improve the accuracy of the 2D model. The preferred embodiments, objects, and features of the present invention have been described, i.e., manufactured using a variety of variations, alterations, and replacements, and are not inconsistent with the spirit of the disclosure of July and further applications listed below. The scope of the patents is separate and this is obvious to those skilled in the art. BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features, and advantages of the present invention will become more apparent. The system of the present invention is a schematic diagram of the method for reconstructing the three-dimensional image of the reconstructed endoscope according to the preferred embodiment of the present invention. The chase 2 is a schematic diagram showing the flow of the feature acquisition and tracking steps in accordance with the preferred embodiment of the present invention. The track is unreliable according to the preferred embodiment of the present invention. 20 200807309 [Main component symbol description 21: Transmitter 24: Computer 28: Surgical tool 32: Endoscope host

20: Compliance with the device 22: chasing the host 26: a suffering from the sound 30: 1% 3⁄4 mirror device 34: Η 栽 plant 20G: capture. Change correction 202: beast sign and capture 20 is: 芸 sample 芸Image pre-processing 2023⁄4: Zhener's realization characteristics and corner characteristics 202c: Feature tracking 202d ··Filtering unreliable traces 204: Establishing scale measurement matrix 206: Cao Wenzhong 208: Euclidean reconstruction 2 1Θ: Point World Coordinate Value 2 12: The decomposed coordinate system coordinate value of the characteristic point of Shi Xianying is 214: the conversion matrix of the fee-specific coordinate system, the feature points under the world coordinate system, and the movement parameter matrix of the shadow pain group 21 €: The value of each precision factor in the scale measurement matrix is used to update the tile attenuation matrix 218: if the difference between the updated accuracy factor and the pre-update accuracy factor is less than the threshold value 21

Claims (1)

200807309 X. Patent application scope 1. A method for reconstructing a three-dimensional image, which comprises at least: obtaining a two-dimensional image by an endoscope; performing a feature tracking step on the two-dimensional image Obtaining the complex number of the two-dimensional image Gu Luzheng_" performing an unreliable trajectory step on the feature points to filter out the characteristic points of the idiots in the feature tracking and capturing step; Aiming at the feature points of the feature points, a scale measurement matrix (〇, with an affine reconstruction method to complete the steps to reconstruct a three-dimensional model; for the twisted three-dimensional model, A three-dimensional model correction step is performed by an Euclidean reconstruction method to construct a scale anomaly three-dimensional model, and a first moving parameter matrix (M) and a first coordinate value (S) are obtained; Select one of the sensing points in a to obtain a second coordinate value; fix the sensor to a arrow and select the sensing point with the surgical tool to fix the tracking and the drug The relative positional relationship between the sensing points, and one of the first conversion matrices provided by the dream of the dizzy steam; using the first conversion moment _ the second coordinate value of the emperor to calculate a third coordinate value Selecting, by the surgical tool, at least four corresponding feature points in the plurality of points to obtain a plurality of fourth coordinate values and a plurality of second conversion matrices; 22 200807309 utilizing the second conversion matrix and the The third coordinate value will be the fourth The scalar value is converted into a plurality of fifth coordinate values; corresponding to the corresponding feature points at the scale, the three exceptions are Cai Wang, m obtains a plurality of sixth coordinate values; the sixth coordinate value and the fifth coordinate child suffers Performing a comparison and calculation step to obtain a third conversion matrix (G); using the third conversion matrix (G), the first argon (5) is replaced by a seventh coordinate value (yi) And using the third conversion pain engine (the inverse matrix of the time, converting the first moving parameter matrix (Μ) into a group of two #翁 parameter matrix (Mf) to obtain a world coordinate three-dimensional model; using the second Moving the parameter matrix (MD and the seventh production value (V) to calculate a plurality of precision factors in the scale measurement matrix, and updating the accuracy factors and the scale measurement matrix (F); and comparing the pre-updates The difference between the value of the precision factor and the updated value of the accuracy factor, if the difference is less than a pre-attack, _ obtain a three-dimensional image of the three-dimensional model of the world coordinate, if the difference is too much Set the threshold, then the scaled measurement matrix will be updated (F) the step of the S-poor Qin, until the difference between the value of the precision factor before the update and the value of the more precise factor is less than the preset threshold. The method of claim 1, further comprising: arbitrarily selecting a plurality of additional feature points by using the sensor in an area to be observed, and projecting the additional feature points to the two-dimensional image to obtain a plurality of additional features Point trajectory; and 23 200807309, etc. - additional feature point trajectories are incorporated into the scale measurement matrix (π), so that the scale measurement matrix (the 忐M ^ Et ^ ^ ) becomes a feature point augmentation matrix ((10) passenger vertical) The ellipsometric three-dimensional 挨 type is constructed by the affine reconstruction method of the array (α% 。). 3. The method described in claim 1 includes at least: Before the step, adding an image deformation correction step 'to correct the image deformation correction step generated by the wide-angle lens comprises at least: performing brightness normalization processing on the one-dimensional shirt image; The two-dimensional image subjected to the brightness normalization processing is binarized by a high-pass chopper and a threshold division method to distinguish the foreground and the background, and the plurality of centers of gravity of the respective patterns on the two-dimensional image are taken out; The two-dimensional image subjected to the binarization processing fills the gap caused by the binarization processing by the closed processing (cl〇sing) operation of the image processing medium type (M0rph0l0gy); and the plurality of straight lines formed by the center of gravity 'Correction of the deformed image by calculating the linearity error. 4. The method of claim i, wherein the feature reclamation step is performed by Kanade-LuCas-Tomasi (KLT) feature tracking method and Edge feature extraction method. 5. The method of claim 1, wherein the feature 24 200807309 acquisition step is one of a Kanade-LUcas-T〇masi (KLT) feature tracking method and an edge feature tracking method. Conducted. 6. The method of claim 2, wherein the unreliable step of at least comprises: comparing and comparing one of the feature points to be measured in a _ 7A ^ a shirt, a brother two two Recording and m-dimensional image t, so that the feature point is located at a first feature point of the first two-dimensional image, Wl·^m^^, a charge point position, and a third feature point position; The point position to the second feature point position vector is a first vector; ^ (3) takes the second feature point position to the third special I point position represents a second vector; and ▲ takes the The angle between the second vector and the first vector is a test angle, and if the value of the test angle is greater than the _pre-set threshold, the feature point to be tested is removed. 7. The method of claim 1, wherein the method of linearity error is a method of C. Brauer-Burxhardt. Calculating the linear error 8 · As described in the scope of claim i, the conversion matrix (G) is a three-by-three matrix. ,,μ弟 25