TWI323425B - Method and system for reconstructing 3-d endoscopic images - Google Patents

Description

1323425 IX. Description of the Invention [Technical Field] The present invention relates to a method for reconstructing a three-dimensional image, and more particularly to a method for reconstructing a three-dimensional image of an endoscope. [Prior Art] During the operation, Physicians must read the position of the street tools and surrounding organizations to ensure the quality of the operation.

Obtaining a two-dimensional image through the endoscope to observe the internal structure of the plexus' physician can't immediately recognize the size of the region and its three-dimensional space fishing relationship, and the risk of surgery increases. In order to assist the physician in the diagnosis and treatment of the cabinet, related computer-aided sputums have been published one after another, as follows: based on the computed tomography (cτ) scene image, the method for detecting and diagnosing the colorectal-related diseases. Or the 3D tracker is fixed on the endoscope, which combines the 2D image of the endoscope with the rattan model constructed with the computed tomography image, so that the doctor can see the three-dimensional structure of the area. The existing virtual endoscope system is mostly based on electric tomography or nuclear magnetic resonance imaging. It is necessary for professionals to operate Cao Ge's expensive equipment to obtain data, which not only increases the cost of time and manpower and material resources, but also increases the probability of radiation damage to patients. . At present, the use of the three-dimensional chasing (four) knot hand balance assist system is also based on such medical images obtained before surgery, and has the disadvantages of cost of treatment or n-shot. β therefore 'requires a method and device for reconstructing three strokes, so as to instantly provide the dynamics of the surgical tool in the body and the three-dimensional geometric model of the lesion area to enable the physician to swim in the virtual environment and observe the lesion to become clinical diagnosis and treatment. Auxiliary Tools 5 1^23425 [Summary of the Invention] Because: The purpose of the invention is to provide a three-dimensional reconstruction method to establish a normal scale of the three-dimensional information ruling team * * 乂 乂 for medical doctors The probability of injury L is too much risk and saves on the cost of treatment and reduces (4) people by light shot. The chance. The present invention can also refactor the real three-dimensional coordinates into a (four) and several therapeutic aspects of evaluation and education training / dimensional model, (four) for surgery or method = - preferred embodiment, the present invention - three The line feature of the image is obtained by the endoscope. The image of the H image is tracked by a feature and a feature capture step is used to filter the feature points. , and steps to filter out the feature tracking into: the feature points of the vertical error. For the trajectory of the feature points, the summer matrix is measured by the scale of the wind, and the three-dimensional model is distorted by the Euclidean reconstruction. Niu Ji., Recognize the eight-form method for 3D model correction i ’ 奸 t scale anomaly 3D model and obtain the first moving parameter matrix filament value. Use the sensor to arbitrarily select one of the space A, and use a value. Fix the sensor to the heart of the surgical tool... point, sensing point, to fix the tracker main unit, sensor, and sense of relative position. Using the first-conversion matrix and the second coordinate value calculation: the::; value 1 surgical tool 'in any special (four), at least w material should be: to obtain the fourth coordinate value and the plurality of second conversion moments (four) with the The second rotation = the second collateral converts the fourth coordinate value to the fifth coordinate value. For the pain, some corresponding feature points are obtained from the scale anomaly three-dimensional model to obtain the sixth coordinate value coordinate value and the fifth coordinate value for _comparison and calculation (4) to obtain the 筮 transformation matrix. Using the third conversion matrix to convert the first coordinate value to the first value, and using the inverse matrix of the third conversion matrix (6), converting the moving average matrix (f) into a second moving parameter matrix (from), to Dimensional model. Using the world coordinate endoscope to move the parameter matrix, collect the world coordinate value (jS〇 calculate the scale measurement matrix (叼: an accuracy factor' and update the accuracy factor and the scale measurement matrix factor. Ratio: Pre-update accuracy The difference between the value of the degree factor and the updated accuracy 枳 three-dimensional ' 'if the difference is less than a predetermined threshold value, then the two-dimensional image of the world seat J, type is obtained, and if the difference is greater than the preset threshold, Then, the updated measurement matrix (7) is replaced by the step of affine reconstruction, until the difference between the value of the X sub-month before the update and the value of the updated accuracy factor is set to a threshold. In another preferred embodiment, a plurality of additional feature points are selected in the area to be observed, and the additional feature points are projected: the two-dimensional image is obtained to obtain a plurality of additional feature point trajectories and These additional traces are combined with this scale measurement matrix (7), so that the scale measurement matrix (7) ... a feature point augmented matrix (α 叼 叼 and this feature point augmented matrix ((10) constructed this distortion by affine reconstruction method Three-dimensional model. For example, in the feature tracking and operation, a wide-angle image correction step is added to correct the image deformation generated by the wide-angle lens. The wide-angle image correction step includes at least: party normalization of the two-dimensional I image ( intensitynGrmalizati〇n) processing; the 2D image of the brightness normalization process is binarized by a high-pass filter ((4) Pass and threshold segmentation method to distinguish the foreground and back 1323425 scene, and take out the 2D image a plurality of centers of gravity of the upper graphs; and the gaps caused by the binarization processing of the two-dimensional images that have been binarized by the morphological processing of the image processing For a plurality of straight lines formed by the center of gravity, the deformed image is corrected by calculating a linearity error. ^ According to still another preferred embodiment of the present invention, the feature tracking and capturing step is performed by Kanade-LuCaS-T〇masi (KLT) feature tracking method and edge feature tracking method. According to another preferred embodiment of the present invention, the step of filtering unreliable tracks includes at least comparing the features. Pointing on a first two-dimensional image, a second two-dimensional image, and a third two-dimensional image to obtain a first feature point coordinate value and a second feature point coordinate value of the feature point located in the first-two-dimensional image And a third feature point coordinate value; the vector represented by the first feature point to the second feature point is a first direction 1 and the vector represented by the second feature point to the third feature point is a second vector, then The angle between the second vector and the first vector is a test angle. If the value of the test angle is greater than a predetermined threshold, the second vector is filtered out. For the implementation of the formula, please refer to FIG. A schematic diagram of a system for reconstructing a three-dimensional image of an endoscope. The system tracker device 2 and the endoscope device 30 of the present invention, wherein the tracker device 20 includes a transmitter 2, a tracker host 22, and a computer 24 And an inductor 26 mounted on the surgical tool 28. The endoscope system of the present invention includes an endoscope 34 and an endoscope main unit 32. The tracker device 20 of the present invention is a miniBlRD three-dimensional tracking device produced by, for example, Ascensi Technology, Inc. (Ascension Technology, C〇rp., VA, USA). During the sinus surgery, the physician H performs endoscopic structural organization and: an intrinsic endoscope 34 to observe the in vivo treatment. Because::===Tool 28, to the disease 15 host 22 can capture the electromagnetic signal sent by g 21 through its sensor 20 to the position of Gu Shi Zeng, juice & '[Induction 26], so as long as the tracking will be direct Connected to the computer 24, the camera instantly acquires the position in the one-dimensional two room of the surgical tool. Under this structure,

On the field line, take an endoscope 34 to shoot. Next, the vehicle of the present invention = the image sequence and the coordinates observed by the (4) device 2 G _ to reconstruct the three-dimensional image structure of the observation area: the door is buckled for the orientation and travel information of the surgical tool. In order to realize the establishment of three-dimensional chamber society, the 曰^^-... will be from 乍胫 to,. OBJECT OF THE INVENTION The method of reconstructing an endoscope of the present invention includes a step of extracting and tracking a characteristic point and a tracking method. A schematic diagram of a flow chart of a preferred embodiment of the present invention is shown. j. Reconstructing the 3D image of the endoscope. Step 2 of the image distortion correction by right. . Usually, the endoscope is equipped with == two. The two-dimensional image in the observation area taken by the endoscope will... The image will be imaged after the endoscope is taken by the endoscope: this phenomenon is The edges of the image are most noticeable. Therefore, before performing the two-dimensional reconstruction of the endoscope shirt, it is necessary to perform a wide-angle image correction on the wide-angle image of the image of the wide-angle lens. First, prepare a calibration plate printed with a plurality of equally spaced dots. Once the mirror is only pre-processed using a single optical image fixed to the lens, due to internal scattering, it is usually close to the middle of the image. The upper wide-angle lens will be more complex with the light than the image periphery, 1323425. Therefore, it is necessary to process the corrected image. Inltity Normalization is processed. The following description shows the brightness normalization process. The dimension of the image is the brightness of the (zV) point, and iflVg is the average brightness of the entire image. 'If each point (z', y) is medium, '- a window of size Awxm, then After the formalization, every point of the ancient sound I is:

Hhj) (1) ^/2 mil Σ Σ7(/+Μ^+ν) =-m/2 v=-w/2 Next, the high pass filter (High Pass FUter) and threshold division method (Thresholding) are used for the image. Binary processing to divide each pixel in the 2D image into foreground and background, and fill the gap with the Closing operation of Morphology in image processing. Then, the center of gravity of each dot is used as the correction point, and the straight line formed by these correction points is c.Brauer-Burchardt in "Automatic c〇rrecu〇n Radial Lens Distortion in Single Views of Urban Scenes

Using Vanishing PQints"—#" Calculates the deformation image by calculating the linearity error. First, define the deformation correction transformation as follows: again r = r'/(I + d2ra +d4r'4 ) (where) ' where 'K is The unknown parameters, r and 〆 are respectively the distance between the image point and the corrected image point after the correction. Then, select the calibration points at the same point, the connection of these points ® * a mu of the original 4 I line, now within The mirror image becomes 1323425 into a curve. With three consecutive correction points as a group, the area of the triangle is the error function, and then the one with the smallest square is formed by the small + method. Deformation parameters. When preparing the triangle area and minimum of all points, Du. 1 J will always produce the best deformation parameters. Pick up, take the steps of feature extraction and tracking - rarely push A / from township 202, to The feature points are captured and traced by the endoscope. This feature reveals the results that should be tracked, and immediately affects the accuracy of the 3D field '', Τ 。. According to the characteristics of this method, ... Trace 2. 2 2. First of all 'the step of pre-processing of endoscopic image (4). Since the endoscope is the human organ tissue, the color texture = 迠 no obvious change, the unfavorable feature operation ' y = t ., / Ττ. Μ 本 本 本 本 本 本 本 本 本 本 本 本 本 本 本 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化 强化Characteristics, beware... It is common to see that the edge area of the image is P. -1疋 After the image is averaged, although the fine-曰 曰 氺 氺 莛 莛 莛 莛 莛 莛 莛 但 但 但 但 但 但 但 但 但 但 但 但The above-mentioned image distortion correction technique is used to overcome this disadvantage. Thus, the phenomenon that the image is unevenly distributed has been removed. Next, the receiver proceeds to step 202b to track and capture two. Point. The invention is characterized by recall, τ 芦~

The monthly system uses Kanade-Lucas_Tomasi (KL method to detect the horned scorpion emblem, 徊雍* &, 0 & .) to track the inter-β lemma, but the endoscopic image sequence characteristics are physiological 纟And Hong, from τ ρ & violence in the image ', I am not easy to find obvious horn features. Therefore, it is necessary to increase the edge feature of the feature, so that the feature is detected to effectively increase the feature point. KLT feature tracking algorithm: The following description KLT feature tracking algorithm can be divided into two parts. /, Holland micro tracking 11 1323425 (i) Corner capture first "the first image of the image sequence" and then grayscale the image, and use a Gaussian Smoothing L〇w-Pass Filter ) Remove noise interference' and calculate the horizontal gradient image in the horizontal and vertical directions of the image: /,, '. Next, establish a covariance matrix for each image point (z·y): c ΚΙ (4)2) sjjx(q, iy(qy) w~klSAiAqVyiq)) Team (4)2) J (3) where ice is centered on Gv) Sub-window; & Gaussian filter Next, find the two eigenvalues (EigenvaIues) e|, e2 of the cw matrix. If the smaller feature value 'is greater than a predetermined threshold, then the point is a corner feature. Use this judgment to find a good initial feature point in the first image. (2) Feature Tracking Calculate the position of the next image based on the initial feature points found in the previous step. Suppose there are currently two images /, J, where the feature points of the image are known and will be used to track their corresponding points in the j image. If the image/feature point is χ, after the displacement = then appears in the image J ’ then the tracking error ^ can be defined as: (4)

\[[J{X^D)-I{X)f〇}{X)cDC where π is a search window centered on point J, defined as

St ^ 1 Wang is another center of the appropriate size, ω (7) specifies the search window

6Α T1 ^ 3S : Weight, usually set to i or distributed in Gaussian distribution. If the bit quantity is evenly distributed, the above formula becomes: 12 1323425 (5) Receive (x+f)_ take -f)]v Exactly solve the error function, the error value, and the difference in Newton's method is the difference. Time: Most...^

D(M1) updates the tracking error until it converges to D(t) + AD > U sigh, where ί is the number of iterations. The convergence h, the tracking error, is the smallest, and the displacement amount 々 can be obtained especially at the position of the next image. After calculating the corresponding position of the first image feature point in the second image, the two image feature points can be used to find the corresponding point in the third image in the same way, and the above steps are repeated until the last - The "image" can obtain the tracking track of the feature points of the image sequence. The following describes the detection steps of the edge feature of step 2〇2b: After completing the image pre-processing, calculate the optimal threshold (〇叫福加心. (4): 2 to divide the image into foreground and background. The valve actually used in the present invention The value is 'to avoid undersampling' which is an empirically defined parameter. In addition, if more than half of the neighborhood of a foreground pixel is a background, the pixel is also marked as a background. The foreground pixel obtained by the present invention according to the above steps The collection is characterized by a feature mask, and the image is then convolved with a high-pass filter (c〇nv〇luti〇n). If a pixel is high-pass filtered, it is larger than the first definition. The threshold is located in the feature mask and is marked as Edge Candidate. Then, the high-pass filtered values of these candidate points are sorted from large to small, and are sequentially stacked. C^Stack), the pixel with the largest value is at the top of the stack (τ〇ρ), and the minimum value is at the bottom of the bottom (Bottom). Each time the (ρ〇ρ)_ point is taken from the stack, check this point. Domain (-a region defined by the user) There are edge feature points. If not, then this point is the edge feature, and repeat the steps of taking the pixel—check the neighborhood—label 13 mark: until the stack is empty, the β step is followed by the above (2) feature tracking. The method is to enter the two points = step two). Then, 'go step d, to compare the special ifπ' to m, please refer to m, which shows the schematic diagram of the steps of filtering out unreliable trajectories according to this «* example, where :, Pi-S, ^ and pi+s The representative point is corresponding to the corresponding point of the s 1+s image, s is an integer v v is 5th "the first amount; VBC is the number of "A", VAB" by Pi · Pi The vector of the main Pl P, + s, if the angle between VAB and Vbc is not #, the trajectory of the u, first and i + s images is: the angle Θ is too large, indicating the feature point to be tested The change trajectory is 'untrusted' and therefore must be rejected. The angle (4) Qualification range is determined by a predefined threshold. » Continue to refer to Figure 2A. After completing step 2Q2(2()2d), proceed to step 2〇 4. The feature represented by the image coordinate system is used to track the two-dimensional coordinates of the trajectory, and the endoscopic mirror. The P parameter matrix (Calibrati〇n is converted into the value of the coordinate system of the endoscope device ((a '<_々, And establish scale measurement

v" ...V W = (θ)

UJP V u/\

2/xP, /丨中中~~)·(叫),, (x广兄Γ^<ηε) is the feature of the feature extraction and tracking _;. feature points at the coordinates of the third image, (\〇,兄'.) The center of gravity of the set of feature points is projected on the f th 1323425 image coordinates 'available, the center of gravity of each feature point of the image is calculated: the ratio? The number of images and feature points are respectively. The matrix (4) each corresponding gas '~ is the precision factor of the reconstructed 3D model, and = the Iterative method approximates the true ~ value, the first-time construction scale measures the matrix π, the value of ^ zero.

Perform affine reconstruction step 2G6, for scale measurement matrix (%, affine reconstruction and construction of 3D model, where affine weight (4) singular value decomposition method is used to decompose scale scale matrix (7) into two ranks (Rank) The product of the matrix of three is 左W force left, which is divided into the affine movement parameter of the endoscope (尬^ Motion Matrix); the left is the affine three-dimensional coordinate of the feature point ix). The three-dimensional model established by affine reconstruction will be distorted, so Euclidean reconstruction is needed to correct the model. A step 208 of the Euclidean reconstruction is performed to correct the three-dimensional model of the distortion. Therefore, it is necessary to calculate the matrix...to convert the endoscopic affine movement parameter ώ and the affine three-dimensional coordinates of the feature point to the left, and convert it into the EucHdean Motion Matrix M of the decomposition seat & The parameter matrix) and the three-dimensional coordinates (EucHdean Shape Matrix) are only the first coordinate value), that is, from ==··/ and left, where the decomposed coordinate system is the Euclidean space coordinate system with the center of gravity of each feature point as the origin ( Euclidean Coordinate System); the three-dimensional coordinate S is a set of feature point coordinate values under the decomposition coordinate system. Here, the matrix / / is calculated with constraints in the manner proposed by Aan®s et al. in "Robust Factorization" to make jj / = from · $. The above steps are the Euclidean reconstruction. The three-dimensional model scale constructed by European reconstruction is abnormal to the real world model. Therefore, the present invention provides a decomposition method combining three-dimensional coordinates, so that the dimensional anomaly three-dimensional model generated after Euclidean reconstruction can be restored to the original 15 1323425. . Please continue to refer to Figure 1 and Figure 2A. After step 202 (202d) is completed, step 2 1 0 is performed to select at least four corresponding to the two-dimensional image in the observation area by using the surgical tool 28 on which the other sensor 26 is mounted in the feature points in the two-dimensional image. The corresponding feature points of the feature points are obtained by a conversion step, and the world coordinate values of the corresponding feature points are obtained. The origin of the world coordinate system where the corresponding feature point world coordinate values are located is the original defined by the tracker device 2 Point (in this embodiment, the transmitter 21 is the origin). Since the sensor is located at the handle end of the surgical tool 28, the coordinate value selected by the top of the surgical tool 28 is not the same as the world coordinate value sensed by the sensor. Therefore, if the target point is obtained, the corresponding feature point is obtained. For the world coordinate value, it is necessary to determine the conversion relationship between the coordinates of the top of the surgical tool 28 and the coordinates selected by the sensor and the world coordinates sensed by the sensor. The coordinates of the coordinate value selected at the top of the sputum surgical tool 28 are sensed, and the sensor coordinate system is the coordinate system that moves the origin of the world coordinate system to the coordinate system of the sensor 26. Assume that the sensor coordinate system and the world coordinate system are switched off = -3 · 'where 'the coffee is the surgical tool 28 in the space · any point - the sensor point coordinate vector of the sensing point, the sensor coordinates coordinate system coordinate system The vector from the origin to the sensing point; the first is the two-coordinate coordinate vector, the world coordinate vector of the sensing point is the direction of the original coordinate point of the world coordinate system, and the matrix 4 represents the coordinate system of the sensor and the coordinate system of the world: mutual conversion relationship Named here as the sensor coordinate conversion matrix: The tracer device 20 knows that the sensor 26 is located at the three-dimensional position of the sensing point ^ its rotation matrix i?, so the sensor coordinate transformation matrix J = _Γ-L〇1 ' Since the sensor 26 is fixed to the handle end of the surgical tool 28, the position of the top end of the surgical tool 16 1323425 28 relative to the sensor 26 is fixed at any point, and the sensor coordinate values are fixed values, that is, for the space Median. It can be seen from the above that if the household value of the fixed value is obtained, and then, the sensor coordinate conversion matrix for the fixed value can be selected: the sense point point H Ύ selected by the system will be the top of the surgical tool 28

4, should be point, the table is not in the world coordinate system H device 26 points selected surgical tools 28 selected points to obtain 7 two = boundary coordinates (second coordinate value), and then by the chaser system The proposed ================================================================================================================ (Second conversion matrix) and the value of the sensor display value (third coordinate value)' can be converted by the sensor coordinate value (fourth coordinate value) of the corresponding feature point selected by the surgical tool 28 World; coordinate value (fifth coordinate value). In step 212, the corresponding feature points are corresponding to the coordinate values (the sixth coordinate system) of the & dislocation #" in the three-dimensional model of the scale anomaly. In step 214, calculating and comparing the corresponding feature point world coordinate values and the decomposition coordinate system coordinate values to obtain a three-by-three coordinate system conversion matrix Θ third conversion matrix), and calculating the Euclidean reconstruction step The feature point 5 under the decomposition coordinate system and the endoscopic movement parameter matrix Μ by <5' = (7.5 and 衂 conversion to the world coordinate system, then the world coordinate three-dimensional model - Μ·5·', from 'and The product of y is still equal to the scale measurement matrix 酽' y is the set of feature point coordinates (the seventh coordinate value) under the world coordinate system and has the same scale as the real world object; it is the 17 1323425 1323425 parameter under the world coordinate system. Matrix (the second moving parameter matrix h system transformation matrix σ, assuming that the feature of the decomposition coordinate system is ^ can be multiplied by the moment ^ and then multiplied by the moment (four) to convert two; located in the decomposition coordinates: the second feature of the two-dimensional image Point to the world to / Γ Dimensional image! · A feature point, therefore I coordinate value, fly to -3x3 matrix, fly as a matrix. This: after the Euclidean reconstruction, that is, by this "three-dimensional seat "...Bu 1' and its estimate Dimension II?2 1..substitute _ I Γ"Μ, get - 联立方(四) After the 仏 id, the 转换 之后 之后 转换 转换 转换 转换 转换 位移 位移 位移 位移 位移 位移 位移 位移 位移 位移 位移 位移 位移 位移 位移 位移 位移 位移 位移 位移 〇 〇 〇 〇 〇 〇 〇 In addition to the coordinate conversion described above, it also contains the modified shape, the function of the difference = the shape of the shape. The affine transformation assumes that the shape of each point is corrected. Cui = not = with - the overall transformation is represented, and the space is changed. = The type of conversion may not be _. If the calculated number of ranks of the coordinate system is not two, it means that the track with the tracking error is not filtered, so the step of removing the unreliable trace must be re-executed (202d). In step 216, the matrix & of the set feature point world coordinates is expressed as: s, (:, 0 qwi~Ta if qwi exists' ^(:J) otherwise. (7) The feature under the second world coordinate system is called an endoscope. The moving parameter moment is f / ^ scale ❹ (four) F in each of the precision factor miscellaneous update accuracy factor value and the scale measurement matrix F, again, from 18 ^ 323425 from s (from). (a $), so there will be two The values of the group accuracy factors are individually corresponding to and (_, from, and the back projection error is small. In step 218 If the difference between the value after the right update and the initial value is less than the preset threshold, the accuracy of the three-dimensional model of the world coordinate is met, and the three-dimensional image of the three-dimensional model of the world coordinate is obtained, and the three-dimensional image has a normal scale. If the accuracy factor is updated and the difference before the update is greater than the preset threshold, then return to step 206 to re-execute the affine reconstruction with the updated scale measurement matrix, and repeat the loop until the accuracy The difference between the updated value of the factor and the value before being updated is less than the preset threshold. The method of repeatedly updating the three-dimensional model of the world coordinate is called the iterative method, wherein step 2 ι only needs to be performed once, that is, Step 212 has the same world coordinate value for the corresponding feature points required for each iteration. During the operation, the lesion may be unevenly covered by other tissues or beyond the scope of image shooting, resulting in uneven feature points in the observation area, resulting in the reconstructed three-dimensional model lacking feature points in important areas to describe the geometric shape of the area. On the other hand, when the three-dimensional coordinate of the feature point is selected by the surgical tool 34, the surgical tool 34 must accurately point to the feature point to avoid an error. If it can be arbitrarily selected, it is not limited to the feature point previously captured, and the point can be greatly increased. Select the efficiency ~ rate '..., increase the geometric details and improve the reconstruction accuracy. These optional feature points can be added to the refactoring process in the following steps. After the three-dimensional reconstruction, the additional feature points are imaged back to the two-dimensional image by using the endoscope moving parameter matrix under the world coordinate system to form a new trajectory 0, <), and the original feature trajectory ( Ά.) Merging, you can get an augmented scale measurement matrix 19 (8) augW: U\PU\\ < vw, ui 4fN, Vff V, UJP uf\ 2MP+Ns)

Among them, iV continues to: =: the number of points; augmented scale measurement matrix ¥, 4 structure, Euclidean reconstruction and iterative wire reconstruction 3D model, the accuracy of the - and quasi-models is improved. The invention, the purpose, and the features of the present invention have been described, that is, the use of many different, changed, and sloppy inventions and the spirit of the invention are not related to the disclosed For those skilled in the art, it is obvious that the scope of the patent is separate. For the sake of familiarity and simplicity of the present invention, in order to make the above and other aspects of the present invention understandable, the following is a preferred embodiment: the levy and the advantages are more obvious. The following is a detailed description of the present invention in conjunction with the drawings. FIG. 1 is a schematic diagram showing the reconstruction of the present invention. The system of the brother-and-shirt image is shown in the second diagram of the method of the three-dimensional image according to the present invention. The reconstructed endoscope of the embodiment only, the second diagram I shows the more suitable vertical shape according to the present invention. The flow chart of the steps. The feature of & 梅 与 and 2C is a schematic representation of the trajectory steps in accordance with the present invention. Filtering unreliable according to preferred embodiment 20 Main component symbol description 20 Tracker device 22: Tracker host 21: Transmitter 26: Sensor 24: Computer 30: Endoscope device 28. Operational workmanship/, 34 : Endoscope 32: Endoscope Host 200: Image Deformation Correction 202 • Feature Tracking and Capture 202a. Endoscope Image Pre-Processing 202b: Detection of Realization Features and Corner Features 202c. Feature Tracking 202d 'Filtering No Reliable trajectory 204 Establish scale measurement matrix 206 Affine reconstruction 208: Euclidean reconstruction 210 Take the corresponding feature point world coordinate value 212 Get the corresponding feature point decomposition coordinate system coordinate value 214 Calculate the coordinate system conversion matrix, under the world coordinate system Point and image capture device movement parameter matrix 216: update the value of each precision factor in the scale measurement matrix and update the scale measurement matrix 218 of γ: if the updated accuracy factor is different from the accuracy factor 4 before the update Value is less than the preset threshold 21

Claims (1)

1323425 X. Shen Qing patent scope • 1 · A method of reconstructing the three males/hot ^. —·· 隹 像 , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Feature tracking and a feature fetching step to obtain a plurality of feature points of the two-dimensional image; performing _, and genus omnidirectional trajectory steps on the feature points to filter out the feature tracking and capturing steps The characteristic point of r's falsification error; entanglement - the scale measurement matrix () which is composed of the enchanting/sounding of these feature points, and the method of anamorphic reconstruction Steps to enter the affine reconstruction to construct a distorted three-dimensional model, • ############################################################################# / Xiangmu constructed a scale anomaly three-dimensional model, and obtained a first moving parameter matrix (slave) and a first coordinate value of 0); using a sensor to randomly select one of the sensing points in the space to obtain a first coordinate Value; fix the sensor to a surgical tool and The surgical tool points the sensing point to fix the tracker, the sensor, the relative positional relationship between the sensing point and the one obtained by the tracker, and the conversion utilizes the first The conversion matrix and the second coordinate value are calculated to obtain a three-coordinate value; the service--the first use of the surgical tool's any point in the feature points--small and soil-less four corresponding feature points to obtain a plurality of The fourth coordinate value and the plural flute _ ' array; spear ~ conversion moment 22 1323425 Converting the fourth coordinate values into a plurality of fifth coordinate values by using the second transformation matrix and the third coordinate value; and correspondingly selecting the corresponding feature points on the scale abnormal three-dimensional model to obtain a plurality of sixth coordinate values Performing a comparison and calculation step on the sixth coordinate value and the fifth coordinate values to obtain a third conversion matrix (G); using the third conversion matrix ((?), the first coordinate value (The magic transformation is converted into a seventh coordinate value (π), and the first moving parameter matrix (Μ) is converted into a second moving parameter matrix (ΛΓ) by using the inverse matrix of the third conversion matrix (σ) Obtaining a three-dimensional model of the world coordinate; ^ using the second moving parameter matrix (Μ) and the seventh coordinate value (lS,) to calculate the scale measurement matrix (the plurality of precision factors in the 叼, and updating the precisions) The difference between the factor and the scale measurement matrix (酽); and the value of the accuracy factor, if the difference is less than a predetermined threshold, then The difference between the values of the accuracy factors is less than the preset threshold. Comparing the values of the precision factors before the update and the updated ones. 2. The method according to claim 1 of the patent application is to use the sensor as a feature point in an area to be observed and append the points Feature points are projected to the two additional feature point trajectories; and at least include: clicking a plurality of additional one-dimensional images to obtain a complex 23 U23425 (more) the replacement page merges the additional feature point trajectories into the ruler _ .al θ the scale measurement matrix... "degree measurement matrix W, so that the feature point augmentation matrix (the feature point augmentation matrix (10) jumps to the dimensional model The array is constructed by the method of affine reconstruction. The method described in item i of the patent application scope includes at least: Before correcting 2 lines of the feature tracking and reclining steps, the image-deformation band change/correction image distortion caused by the wide-angle lens, and the image is also positively changed to at least include: normalizing the brightness of the image-dimensional image Processing; the two-dimensional image subjected to the brightness normalization processing is binarized by a high-pass inter-channel value segmentation method to distinguish the foreground and the background, and the plurality of centers of gravity of the graphics on the two-dimensional image are extracted; Filling the two-dimensional image that has been binarized with the closing of the Morphology of the image processing to fill the gap caused by the ternary processing; and the plural formed by the center of gravity Straight line, correcting the deformation image by a method of calculating the linearity error. 4. The method of claim 1, wherein the feature reclamation step is performed by a Kanade-Lucas-Tomasi (KLT) feature tracking method and an edge feature extraction method. 5. The method of claim 1, wherein the feature is 24 1323425 Positive mi = take step (four) a K_de-LUcas_T〇masi (KLT) feature line feature (Edge Feat paper) tracking method is carried out. 'And 6. The method described in the scope of the patent application is not reliable. The at least one step includes: ', in the filter to compare the feature points - the feature points to be tested are at the "second" point... ;it image and a third two-dimensional image to obtain the special: point position 'the first feature point position of the flute-dimensional image, the second feature point position and a third feature point position; The first feature point position to the second feature point bit vector is a first vector; the second feature point position of the clothing to the third feature vector is a second vector; and “representing the second vector and The angle of the first vector is - the test angle, if the point is greater than a pre-set threshold, then the method to be tested is filtered out. 7. The method according to claim 3, wherein the straight line error is Method $c(4)(4)·Method for calculating straight line by Can. °, left 8. As described in the patent item i, the 苴 conversion matrix (6) is a -three by three matrix.