CN101836852B - Medical endoscope containing structured light three-dimensional imaging system - Google Patents

Abstract

The invention provides a medical endoscope containing a structured light three-dimensional imaging system, which relates to a medical endoscope, solves the problem that the conventional three-dimensional observing technology can not be directly applied to the medical endoscope due to space limit in the brain surgery of nasal cavity expansion. The medical endoscope comprises a working scope tube, a computing and processing module and a structured light channel, wherein the working scope tube comprises an imaging channel and an illuminating channel; illuminating optical fibers are arranged in the illuminating channel; the signal input end of the computing and processing module is connected with the electric signal output end of the imaging channel; the structured light channel is arranged in the illuminating channel; the light beam outputted from the tail end of the illuminating optical fiber is received by the structured light channel; the light beam generates structured light after passing through the structured light channel; and the structured light is outputted out of the illuminating channel through the structured light channel. The invention overcomes the defects of the prior art and can be applied to the brain surgery of nasal cavity expansion.

Description

The medical endoscope that comprises structured light three-dimensional imaging system

Technical field

The present invention relates to a kind of medical endoscope.

Background technology

The endoscopic imaging technology is a kind of typical medical imaging technology, is all bringing into play important effect at aspects such as medical diagnosis and surgical navigationals.Along with people's is to the pay attention to day by day of diagnosis and surgical navigational precision, and the application of three-dimensional imaging technology in medical endoscope is also increasing.With the cerebroma operation is example, in traditional cerebroma Therapeutic Method, adopts incision patient's the skull or the modus operandi of facial skeleton like craniotomy, tends to badly damaged patient's appearance, and needs the long post-operative recovery phase.Recently, nasal cavity expansion cerebral surgery operation (EEN, Expanded Endonasal Neurosurgery) has obtained very big concern in the cerebroma clinical treatment.This operation plan imports nasal cavity with miniature endoscope and surgical device, and also excise the exact position that finds out cerebroma.The smooth implementation of EEN operation needs effective guide of scope image navigation system, and the performance of navigation system can directly have influence on the accurate positioning property and the operating susceptiveness of cerebroma.General main two sub-systems that comprise of image navigation system of endoscope: (1) is used for obtaining in real time the endoscopic imaging system of operative site image; (2) be used for the location map of the surgical apparatus navigation system on preoperative CT or the MRI data.Fig. 1 is the structural representation of a typical rigid body endoscope; As shown in Figure 1; There are two passages in endoscope at internal work mirror tube portion: imaging passage 0-4 and illumination channel 0-6, the passage 0-4 that wherein forms images are used for organ surface is formed images, and illumination channel 0-6 then is used for output beam.The optical element (counting from end) of imaging passage 0-4 comprises the object lens 0-3 that disperses camera lens 0-5, is used to focus that is used to observe big field-of-view angle, bar-shaped conducting parts 0-2 and the magnification eyepiece 0-7 that is used for converted image; Illumination channel only comprises lighting fiber 0-1, is used to be connected to the light source place.0-4 compares with the imaging passage, and it is much simple that the structure of illumination channel 0-6 is wanted.

The image navigation system of endoscope that is applied to EEN operation at present still has very big unsound property, owing to can not from the image that obtains or video, restore the three-dimensional scenic certain deviation (maximum deviation can reach about 2cm) of appearance that causes navigating.If lack the necessary three-dimensional scene information of operative site, the doctor often need attempt the contact tissue surface experiencing depth distance, or relies on the personal experience to make subjective judgment.Thereby accurate three-dimensional visualization scene will significantly improve susceptiveness and the localized accuracy of cerebroma of surgeon's operation technique, has great technology and medical application value.

Recent years, the dimensional Modeling Technology of endoscopic images has obtained development to a certain degree and has obtained preliminary achievement in research.Yet owing to the endoscope that uses in the EEN operation must arrive the skull bottom section so that can pass nasal cavity very for a short time, some conventional stereopsises technology often can not directly be used owing to spatial constraints like the multi-eye stereo vision.Up to now, the report that does not occur feasible EEN three dimensional structure modeling technique aspect both at home and abroad as yet.

Summary of the invention

The objective of the invention is to solve in the nasal cavity expansion cerebral surgery operation, can not directly apply to the problem of medical endoscope because conventional stereopsis technology receives spatial constraints, a kind of medical endoscope that comprises structured light three-dimensional imaging system is provided.

The medical endoscope that comprises structured light three-dimensional imaging system; It comprises work mirror pipe; Said work mirror pipe comprises imaging passage and illumination channel; Said illumination channel internal illumination optical fiber, it also comprises the computing module, the signal input part of said computing module connects into the electrical signal of picture passage;

It also comprises the structured light passage; Said structured light passage places in the illumination channel; The light beam of the end output of lighting fiber is received by the structured light passage, produce structured light behind this light beam process structured light passage to, and said structured light is exported outside the illumination channel by the structured light passage; Said structured light passage is made up of amasthenic lens group, miniature grid screen and projection lens group, and through amasthenic lens group, miniature grid screen and projection lens group after, export successively by generating structure light for the light beam of the end output of lighting fiber;

Distance B between the terminal and miniature grid screen of said lighting fiber confirms that according to the raster resolution standard said distance B need satisfy following constraints:

$\frac{H}{D}\≥\frac{L}{D+Z},$

Wherein H representes the radius of structured light passage, and L representes the radius in optical fibre illumination zone, and Z representes the distance between miniature grid screen and the target.

The medical endoscope that comprises structured light three-dimensional imaging system of the present invention; Through in the illumination channel of medical endoscope, setting up structured light passage, and comprehensive the use, and can be applied in the nasal cavity expansion cerebral surgery operation based on the structural light three-dimensional method for reconstructing of grid deformation and based on the three-dimensional shape information that three-dimensional rebuilding method obtains medical tissue and organ surface that defocuses of method of geometry; 3-D view of the present invention realizes there is not the occupying volume external space through the structural light three-dimensional method for reconstructing.

Description of drawings

Fig. 1 is the structural representation of typical rigid body endoscope; Fig. 2 is the structural representation of medical endoscope of the present invention; Fig. 3 is the structural representation of the structured light passage in the medical endoscope of the present invention; Fig. 4 is the generalized section of medical endoscope of the present invention.

The specific embodiment

The specific embodiment one: combine Fig. 2 and Fig. 3 that this embodiment is described; The medical endoscope that comprises structured light three-dimensional imaging system of this embodiment; It comprises work mirror pipe, and said work mirror pipe comprises imaging passage 1 and illumination channel 2, said illumination channel 2 internal illumination optical fiber 3; It also comprises computing module 4, and the signal input part of said computing module 4 connects into the electrical signal of picture passage 1;

It also comprises structured light passage 5; Said structured light passage 5 places in the illumination channel 2; The light beam of the end output of lighting fiber 3 is received by structured light passage 5, and this light beam produces structured light through structured light passage 5 backs, and said structured light is exported to outside the illumination channel 2 by structured light passage 5.Above structure can be referring to Fig. 2.

The diameter of lighting fiber 2 is about 10 microns (μ m), uses the reason of thin optic fibre bundle to be that it can be approximated to be a point source.

Endoscope as shown in Figure 2 comprises imaging passage 1 and illumination channel 2.Imaging passage 1 comprises and is used to observe dispersing camera lens, the object lens that are used to focus, being used for the bar-shaped conducting parts and the magnification eyepiece of converted image of big field-of-view angle; Illumination channel 2 internal illumination optical fiber 3, lighting fiber 3 is connected to the light source place.Structured light passage 5 is set in the illumination channel 2, is used for generating structure light.Wherein Target is a target.

With expansion intranasal cerebral surgery operation is example, owing to need endoscope be inserted from nasal cavity, so endoscope's size must be small and exquisite as far as possible.As shown in Figure 4, about 4 millimeters of the diameter of rigid body endoscope, about 2.8 millimeters of imaging passage 1 diameter, about 1 millimeter of structured light passage 5 diameters.

Referring to Fig. 3; Said structured light passage 5 can be made up of amasthenic lens group 51, miniature grid screen 52 and projection lens group 53; Through amasthenic lens group 51, miniature grid screen 52 and projection lens group 53 after, export successively by generating structure light for the light beam of the end output of lighting fiber 3.

Miniature grid screen 52 is key for design elements, and it need possess following characteristics: solid, fairly regular and resolution very high (being of a size of standard with grid unit); In this embodiment, the material of said miniature grid screen 52 can adopt the carbon polymeric material, specifically can select carbon nano tube/epoxy resin composite or carbon nano-tube/poly ammonia ester composite for use.

Make b1 represent the distance of miniature grid screen 52, make b2 represent the distance of miniature grid screen 52, then can make 1:3＞b1:b2＞1:5 to projection lens group 53 equivalent photocentres to the equivalent photocentre of amasthenic lens group 51.

The light beam of said structured light passage 5 outputs is structured light, because endoscope's size and structural limitations can't produce complicated structure light coding pattern, and can only produce single raster mode.

When projection lens group 53 can use thin camera lens module approximate; Distance b 2 is determined by two factors; Said two factors are respectively the equivalent photocentre of projection lens group 53 to the amplification R apart from Z2 and projection lens group 53 between the target, and b2=Z/R is promptly arranged, wherein can be according to the statistical study of EEN Clinical symptoms is estimated apart from Z2; Z2 is 10～20mm when conchoscope is formed images, and amplification R is by the light source decision of being adopted.

Said amasthenic lens group 51 can be made up of first planoconvex lens 511 and second planoconvex lens 512; And the convex surfaces of the convex surfaces of first planoconvex lens 511 and second planoconvex lens 512 is placed relatively; The plane surface of first planoconvex lens 511 is as the light input end of amasthenic lens group 51, and the plane surface of second planoconvex lens 512 is as the light output end of amasthenic lens group 51.

Said first planoconvex lens 511 and second planoconvex lens 512 all can adopt the two and lens of achromatism.

In this embodiment; The inner cold light that uses xenon or metal halide generation of endoscope is as light source; Consider the wide spectrum of light source, can adopt achromatic doublet, the not normal minimum of aberration that the camera lens variations in refractive index about optical wavelength is caused as amasthenic lens.

Make d1 represent the focal length of first planoconvex lens 511, make d2 represent the focal length of second planoconvex lens 512, then the distance between the photocentre of the photocentre of first planoconvex lens 511 and second planoconvex lens 512 can be d1+d2.

Said projection lens group 53 can be made up of the 3rd planoconvex lens 531 and Siping City's convex lens 532; And the convex surfaces of the 3rd planoconvex lens 531 and the convex surfaces of Siping City's convex lens 532 are placed relatively; The plane surface of the 3rd planoconvex lens 531 is as the light input end of amasthenic lens group 51, and the plane surface of Siping City's convex lens 532 is as the light output end of amasthenic lens group 51.

Make d3 represent the focal length of the 3rd planoconvex lens 531, make d4 represent the focal length of Siping City's convex lens 532, then the distance between the photocentre of the photocentre of the 3rd planoconvex lens 531 and Siping City's convex lens 532 can be d3+d4.

Distance B between the terminal and miniature grid screen 52 of lighting fiber 3 can confirm that said distance B need satisfy following constraints according to the raster resolution standard:

$\frac{H}{D}\≥\frac{L}{D+Z},$

Wherein H representes the radius of structured light passage 5, and L representes the radius in optical fibre illumination zone, and Z representes the distance between miniature grid screen 52 and the target.

Satisfying under the situation of above-mentioned constraints, selecting bigger D value, can guarantee to obtain higher raster resolution.

Said computing module 4 is used for the image that imaging passage 1 obtains is carried out three-dimensional reconstruction, obtains the 3-D view of said image.

The detailed process that the image that 4 pairs of imagings of computing module passage 1 obtains carries out three-dimensional reconstruction is:

For clear area in the target,, and utilize the surface three dimension shape of rebuilding said clear area based on the structural light three-dimensional method for reconstructing of grid deformation through the method for extraction grid angle point;

For fuzzy region in the target, adopt the surface three dimension shape that three-dimensional rebuilding method is rebuild said fuzzy region that defocuses based on method of geometry.

The detailed process of said three-dimensional rebuilding method based on grid deformation can be with reference to following process:

For any 1 P on the object, its coordinate in world coordinate system is (X _w, Y _w, Z _w), at the coordinate of photographing unit reference frame be

Coordinate at the projection lens reference frame is

The initial point of photographing unit reference frame is defined in the photocentre of the CCD lens of the interior photographing unit of imaging passage (1), and the initial point of projection lens reference frame is defined in the photocentre of structured light passage (5) inner projection lens group (53), camera images coordinate system (u _c, v _c) initial point be defined in the center of CCD Projection lens image coordinate system (u _p, v _p) initial point be defined in the center of projection lens group (53)

f _cBe the focal length of CCD lens, f _pIt is the focal length of projection lens group (53);

World coordinates (the X of spatial point P _w, Y _w, Z _w) and the photographing unit reference coordinate

There is following transformational relation:

$\left[\begin{array}{c}{X}_w^c\\ Y_w^c\\ Z_w^c\end{array}\right]=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]+\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]$

Consider the inclination deformation situation of image.When so-called inclination deformation just was meant imaging, the X axle and the Y axle of image were non-orthogonal, though in most cases X axle and Y axle are orthogonal, when optical axis and the incomplete quadrature of imaging plane, may cause X axle and Y axle non-orthogonal.

The inclination deformation angle of supposing X axle and Y axle is α, obtains:

$\left[\begin{array}{c}{u}_c\\ v_c\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_{c1}& \tan \mathrm{\α}{f}_{c1}& u_0\\ 0& f_{c2}& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]$

F wherein _C1Be the focal length of CCD lens U direction, f _C2Focal length for CCD lens V direction;

So:

$Z_w^c\left[\begin{array}{c}{u}_c\\ v_c\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_{c1}& \tan \mathrm{\α}{f}_{c1}& u_0\\ 0& f_{c2}& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_1\\ r_{21}& r_{22}& r_{23}& t_2\\ r_{31}& r_{32}& r_{33}& t_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]$

With the inner parameter matrix and the external parameter matrix abbreviation of photographing unit, and make

to obtain:

$S_c\left[\begin{array}{c}{u}_c\\ v_c\\ 1\end{array}\right]=\left[\begin{array}{cccc}{a}_{11}^c& a_{12}^c& a_{13}^c& a_{14}^c\\ a_{21}^c& a_{22}^c& a_{23}^c& a_{24}^c\\ a_{31}^c& a_{32}^c& a_{33}^c& a_{34}^c\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]$

In like manner because projector can be regarded as the inversion of photographing unit, so can obtain:

$S_p\left[\begin{array}{c}{u}_p\\ v_p\\ 1\end{array}\right]=\left[\begin{array}{cccc}{a}_{11}^p& a_{12}^p& a_{13}^p& a_{14}^p\\ a_{21}^p& a_{22}^p& a_{23}^p& a_{24}^p\\ a_{31}^p& a_{32}^p& a_{33}^p& a_{34}^p\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]$

Cancellation S _cAnd S _p, obtain:

$\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]={\left[\begin{array}{ccc}{a}_{11}^c-u_c{a}_{31}^c& a_{12}^c-u_c{a}_{32}^c& a_{13}^c-u_c{a}_{33}^c\\ a_{21}^c-v_c{a}_{31}^c& a_{22}^c-v_c{a}_{32}^c& a_{23}^c-v_c{a}_{33}^c\\ a_{11}^p-u_p{a}_{31}^p& a_{12}^p-u_p{a}_{32}^p& a_{13}^p-u_p{a}_{33}^p\end{array}\right]}^{-1}\left[\begin{array}{c}{u}_c{a}_{34}^c-a_{14}^c\\ v_c{a}_{34}^c-a_{24}^c\\ u_p{a}_{34}^p-a_{14}^p\end{array}\right]$

The image that is obtained by photographing unit is through after the decoding processing, and each code value can be mapped to the relevant position of the coding pattern that projection lens group (53) throwed, and promptly has a kind of corresponding relation between them:

φ(u _c，v _c)＝φ(u _p)，

The concrete form of following formula is by the decision of the coded system that adopted, the different coding mode corresponding different concrete forms;

For a structured light system of demarcating, the inside and outside parameter of photographing unit and projector is all known.If can implementation space point in the coupling (Correspondence) of picture point on the photographing unit and the subpoint on the projector, promptly confirm corresponding relation, then can obtain the coordinate of spatial point P, thereby realize three-dimensional reconstruction.Because endoscope's size and structural limitations, can't produce complicated structure light coding pattern, can only produce single raster mode, through extracting the grid angle point and utilizing the 3D shape that the trigonometric ratio method can reconstructed surface.

The said process that defocuses three-dimensional rebuilding method based on method of geometry is following:

Step 1, produce image focal plane r such as the T width of cloth at random _j, obtain image focal planes such as every width of cloth equal the out-of-focus image at z0 place in object distance light distribution I _{1, j}, obtain the light distribution I of out-of-focus image that object distance equals the image focal planes such as every width of cloth of z1 simultaneously _{2, j}Wherein, j=1,2 ..., T, 10mm≤z0≤20mm, 10mm≤z1≤20mm;

Step 2, based on { (I _{1, j}, I _{2, j}) | j=1,2 ..., T} makes up training sample set, introduces image to I _j=(I _{1, j}, I _{2, j});

Step 3, according to the minimum principle of inequality, have

$\hat{S},\hat{r}\text{=argmin}{\left|\right|I_j-H_S{r}_j\left|\right|}^2,$

Wherein, Represent the restorative image focal plane of Denging,

The depth information of presentation video is estimated, H _sCorresponding linearity defocused transformation operator when the expression degree of depth was S;

Step 4, each degree of depth rank S found the solution the linear operator of a correspondence

Make

Minimum, thus the corresponding linear operator of each degree of depth rank S obtained

I wherein _SRepresent that the out-of-focus image that a degree of depth is S is right;

Step 5, when endoscopic imaging, the adjustment camera obtain two width of cloth image I=(I ₁, I ₂)), utilize step 4 to obtain

According to formula

Obtain the depth information of image, and then realize three-dimensional reconstruction image.

The detailed process of the said content of step 4 is:

Each degree of depth rank S is found the solution the linear operator of a correspondence

Make

Minimum, wherein I _SRepresent that the out-of-focus image that a degree of depth is S is right;

Make up a large-spacing learning planning problem and learn linear operator

$\underset{H_S^{\⊥}}{\min }(1-\mathrm{\μ})\underset{i}{\mathrm{\Σ}}{\left|\right|H_S^{\⊥}{I}_i\left|\right|}^2+\mathrm{\μ}\underset{i,j}{\mathrm{\Σ}}(1-y_{i,l})[1+{\left|\right|H_S^{\⊥}{I}_i\left|\right|}^2-{\left|\right|H_S^{\⊥}{I}_l\left|\right|}^2],$

Order ${\left(H_S^{\⊥}\right)}^2=H_S^{\⊥},{\left(H_S^{\⊥}\right)}^t=H_S^{\⊥},$

If I _jCorresponding depth information is S, then y _{I, l}=1, otherwise y _{I, l}=0; Wherein, μ=0.5 is the compromise parameter;

Utilize the gradient descent method on the Stiefel stream shape, find the solution and obtain linear operator

Operation principle of the present invention is:

Structured light passage in the endoscope is launched the light of AD HOC; After projecting organ surface; Camera by in the imaging passage is caught the organ surface image; By the deformation data of computing module, utilize the structural light three-dimensional method for reconstructing to extract the 3D shape of organ then through light in the analysis image.

The medical endoscope that comprises structured light three-dimensional imaging system of the present invention; Through in the illumination channel of rigidity medical endoscope, setting up a pipeline; Focus lens group, miniature grid screen and projection lens group are set in pipeline; And combine to utilize the fibre bundle of endoscope to realize a structured light generation system, the comprehensive use based on the structural light three-dimensional method for reconstructing of grid deformation and based on the three-dimensional shape information that three-dimensional rebuilding method obtains medical tissue and organ surface that defocuses of method of geometry.

Claims (1)

1. the medical endoscope that comprises structured light three-dimensional imaging system; It comprises work mirror pipe; Said work mirror pipe comprises imaging passage (1) and illumination channel (2); Said illumination channel (2) internal illumination optical fiber (3), it also comprises computing module (4), the signal input part of said computing module (4) connects into the electrical signal of picture passage (1); It also comprises structured light passage (5); Said structured light passage (5) places in the illumination channel (2); The light beam of the end output of lighting fiber (3) is received by structured light passage (5); This light beam produces structured light through structured light passage (5) back, and said structured light is exported to outside the illumination channel (2) by structured light passage (5); Said structured light passage (5) is made up of amasthenic lens group (51), miniature grid screen (52) and projection lens group (53); After the light beam of the end output of lighting fiber (3) passes through amasthenic lens group (51), miniature grid screen (52) and projection lens group (53) successively, the output of generating structure light;

It is characterized in that the distance B between end and the miniature grid screen (52) of said lighting fiber (3) confirms that according to the raster resolution standard said distance B need satisfy following constraints:

$\frac{H}{D}\≥\frac{L}{D+Z},$

Wherein H representes the radius of structured light passage (5), and L representes the radius in optical fibre illumination zone, and Z representes the distance between miniature grid screen (52) and the target.

Images