CN102958426A

CN102958426A - Method and system for fluorescent imaging with background surgical image composed of selective illumination spectra

Info

Publication number: CN102958426A
Application number: CN2011800307160A
Authority: CN
Inventors: I·M·麦克道尔; C·J·哈瑟
Original assignee: Intuitive Surgical Operations Inc
Current assignee: Intuitive Surgical Operations Inc
Priority date: 2010-07-02
Filing date: 2011-06-22
Publication date: 2013-03-06
Anticipated expiration: 2031-06-22
Also published as: US9211058B2; US20160089013A1; JP6605654B2; KR101913622B1; EP2587988B1; KR20130097058A; KR20180004287A; JP6321224B2; JP7049302B2; JP6073784B2; CN105231976B; US20120004557A1; EP2587988A1; KR101813909B1; CN105231976A; JP2020006244A; JP2017104567A; CN102958426B; JP2013531538A; WO2012003127A1

Abstract

A surgical site is simultaneously illuminated by less than all the visible color components that make up visible white light, and a fluorescence excitation illumination component by an illuminator in a minimally invasive surgical system. An image capture system acquires an image for each of the visible color components illuminating the surgical site and a fluorescence image, which is excited by the fluorescence excitation component from the illuminator. The minimally invasive surgical system uses the acquired images to generate a background black and white image of the surgical site. The acquired fluorescence image is superimposed on the background black and white image, and is highlighted in a selected color, e.g., green. The background black and white image with the superimposed highlighted fluorescence image is displayed for a user of the system. The highlighted fluorescence image identifies tissue of clinical interest.

Description

Has the method and system by the fluorescence imaging of the background of elective irradiation spectral composition operation image

Related application

The application requires following priority and interests:

It is Ian McDowall and Christopher J.Hasser U.S. Provisional Application No.61/361260 that submission on July 2nd, 2010 is entitled as " METHOD AND SYSTEM FORFLUORESCENT IMAGING WITH BACKGROUND SURGICALIMAGE COMPOSED OF SELECTIVE ILLUMINATION SPECTRA " invention people, and this application full content is included in this for your guidance.

Technical field

This invention each side relates to endoscopic imaging, relates more specifically to mix visible images and optional image, in order to show for the surgeon provides the real-time video of enhancing.

Background technology

The da that Intuitive Surgical Inc. of markon's welfare Ya Senniweier (Intuitive Surgical.Inc.) sells

Surgery systems is a kind of Wicresoft operated from a distance surgery systems, and it provides many benefits for the patient, recovers and the shortening hospital stays such as the damage, the quickening that reduce health.Da

An important component part of surgery systems is that the dual pathways to visible images (namely left and right) video acquisition and demonstration can be provided, thereby provides stereovision for the surgeon.

This electronic three-dimensional imaging system can be exported the high definition video image to the surgeon, and can allow for example to change focal length/convergent-divergent with the feature of view that a kind of " amplification " is provided, this feature allows the surgeon to determine specific types of organization and characteristic, and the degree of accuracy work to increase.Yet in typical surgical field, some types of organization is difficult to determine, perhaps tissue of interest is covered by its hetero-organization at least in part.

By using picture-in-picture (PIP) to show, can show simultaneously to the surgeon the extra image information of operative site.Extra image information may be useful for improve surgical effect for the surgeon.Yet the less picture during PIP shows may be for the details that desirable level is shown and Yan Taixiao.In addition, owing to being the picture frame that separates, so the surgeon combines with image or the image mode of the imagination with two separation, this can be very tired.

Summary of the invention

In one aspect, in minimally invasive surgery system, operative site simultaneously by (i) than all visible color irradiation compositions composition still less that forms visible white light (sometimes being called as white light), and (ii) from the fluorescence excitation irradiation composition irradiation of irradiator.The image of each visible color composition of operative site is shone in image capture system collection in the minimally invasive surgery system, and shines the fluoroscopic image that composition excites by fluorescence excitation.Minimally invasive surgery system uses for the image that shines the collection of composition from each visible color of irradiator, generates the background black white image of operative site.This background black white image is called as the bandwidth restricted image.

The fluoroscopic image that gathers is superimposed on the background black white image, and highlights with the color (for example green) of selecting.The background black white image that shows the fluoroscopic image that highlights with the quilt that superposes for the user of system is referred to herein as enhancing and shows.Being highlighted fluoroscopic image determines interested clinical tissue.

Using minimally invasive surgery system to carry out in the surgical operation, the bandwidth restricted image of the fluoroscopic image that quilt with stack highlights is being provided in real time to the surgeon.This novel method can not bring time delay, this time delay with at a frame IT normal picture and then use two frames obtaining in different time points to generate the single frame that shows to the surgeon at another frame IT fluoroscopic image necessary connection is arranged.Equally, novel method described herein can not cause postponing or make up the deficiency of time of two frames that different time points gathers.

The processing that strengthen to show can not introduced any delay, and need not as with post processing frame storage frame afterwards.Therefore, with respect to utilizing time slicing be added to system on the 3 D visible image of fluoroscopic image, reduced the Storage and Processing requirement of system.

Equally, image highlight the part synchronous with background image all the time.By contrast, be stored and then with in time with the frame registration that occurs after a while the time, between frame, can change the position of the tissue that is highlighted, so when shown, the image that quilt is highlighted departs from physical location when the frame that comprises fluoroscopic image.Therefore, this novel method is for showing that mobile tissue (for example vascular system) is superior, wherein comprises method that the time gathers mismatch and will cause the motion artifacts that shows, and this can upset the surgeon.

In one example, a plurality of visible color irradiation compositions are combined to form white light.Irradiator in the minimally invasive surgery system provides at least two irradiation compositions simultaneously.As in the composition one of irradiation during for fluorescence excitation irradiation composition, other irradiation compositions that provided by irradiator comprise than all visible color in a plurality of visible color irradiation compositions and shine compositions composition still less.

In minimally invasive surgery system, video camera is caught substantially simultaneously first and is gathered component-part diagram picture and the second collection component-part diagram picture that is different from the first collection component-part diagram picture.Controller is connected to video camera, gathers the component-part diagram picture to receive.At this, represent substantially simultaneously degree such the time, namely this degree can consider that the element responds in the video camera sends to the time difference of catching triggering signal of video camera.

In one aspect, controller generates first, second, and third visible color composition of bandwidth restricted image.Each of these visible color compositions comprises or gathers the component-part diagram picture from first.Controller gathers the second visible color composition that component-part diagram looks like to be added into the bandwidth restricted image with second, gathers component-part diagram as both so that the second visible color composition comprises the first collection component-part diagram picture and second.The bandwidth restricted image does not comprise the visible color component-part diagram picture of the collection that is produced by at least one the visible color irradiation composition in a plurality of visible color irradiation compositions.

Controller generates the image that comprises the bandwidth restricted image, and wherein the bandwidth restricted image is sent to the display that is connected to controller.Aspect this, display receives first, second, and third visible color composition of self-controller.Display illustrates the image that comprises the bandwidth restricted image.Comprise that with respect to not comprising that second gathers the image section of component-part diagram picture, highlighting second gathers the image section of component-part diagram picture.

In one aspect, other irradiation compositions comprise the first visible color irradiation composition.First gathers the first visible color component-part diagram picture that component-part diagram similarly is collection, and the second collection component-part diagram similarly is the fluoroscopic image that is excited by fluorescence excitation irradiation composition.

In one aspect of the method, other irradiation compositions comprise the first visible color irradiation composition and the second visible color irradiation composition.In aspect this, video camera is also caught to win the third place and is gathered the component-part diagram picture.First gathers the first visible color component-part diagram picture that component-part diagram similarly is collection.Second gathers the fluoroscopic image that component-part diagram similarly is collection.The 3rd gathers the second visible color component-part diagram picture that component-part diagram similarly is collection.Controller generates first, second, and third color component from the bandwidth restricted image of the first and second visible color component-part diagram pictures that gather.In one example, the first visible color composition is blue composition, and the second visible color composition is green composition.

In a further aspect, other irradiation compositions are the first visible illumination composition and the second fluorescence excitation irradiation composition.In addition, in aspect this, video camera is caught to win the third place and is gathered the component-part diagram picture.First gathers the first visible color component-part diagram picture that composition is collection.Second gathers the first fluoroscopic image that component-part diagram similarly is collection.The 3rd gathers the second fluoroscopic image that component-part diagram similarly is collection.Controller is added into the 3rd images acquired the 3rd visible color composition of bandwidth restricted image.

In one aspect, irradiator comprises the first visible color irradiation composition source, the second visible color irradiation composition source and the 3rd visible color irradiation composition source, wherein the second visible color irradiation composition is different from the first visible color irradiation composition, wherein the 3rd visible color irradiation composition is different from the second visible color irradiation composition, and is different from the first visible color irradiation composition.Irradiator also comprises the fluorescence excitation irradiation source.

Minimally invasive surgery system also comprises power level and power-supply controller of electric, and it is connected to first, second, and third visible color composition irradiation source, and is connected to the fluorescence excitation irradiation source.Pattern changes device and is coupled to power level and power-supply controller of electric.The pattern change has the first state and the second state.

When pattern change utensil has the first state, power level and power-supply controller of electric (a) provide power to first, second, and third visible color irradiation composition source, and power is not provided to the fluorescence excitation source, and (b) first, second, and third visible color irradiation composition source has the first illumination levels.When pattern change utensil has the second state, power level and power-supply controller of electric (a) provide power to the first visible color irradiation composition source, at least part of and this fluorescence excitation source to the second visible color irradiation composition source provides power, and do not provide power to the 3rd visible color irradiation composition source, and the illumination levels that (b) reduces the first visible color irradiation composition source and the second visible color irradiation composition source.

In the method that generating enhanced images shows in minimally invasive surgery system, controller generates and comprises that first gathers the first visible color composition of the bandwidth restricted image of component-part diagram picture.Controller also generates and comprises that first gathers the second visible color composition of the bandwidth restricted image of component-part diagram picture.Controller gathers component-part diagram with second and looks like to be added into the second visible color composition.Second gathers composition represents the fluorescence that excited by the irradiation from the first fluorescence excitation irradiation source.Controller generates and comprises that first gathers the 3rd visible color composition of the bandwidth restricted image of component-part diagram picture.The bandwidth restricted image does not comprise the visible color component-part diagram picture of the collection that is produced by a visible color irradiation composition in a plurality of visible color irradiation compositions of white light.

This method is showing that screen display comprises the image of bandwidth restricted image.Comprise that with respect to not comprising that second gathers the image section of component-part diagram picture, highlighting second gathers the image section of component-part diagram picture.

In aspect another of the method that generating enhanced images shows in minimally invasive surgery system, carry out the enhancing image correction module at processor.Carry out the generation method, the method receives the first visible color component-part diagram picture that gathers at the first visible color composition input of a plurality of visible color composition inputs.The method also receives the fluoroscopic image that gathers at the second visible color composition input of a plurality of visible color composition inputs.

The method generates first signal on the first visible color composition outfan, this first signal comprises the first visible color component-part diagram picture of this collection.On the second visible color composition outfan, generate secondary signal, this secondary signal comprises the combination of the fluoroscopic image of the first visible color component-part diagram picture of collection and collection.Equally, on the 3rd visible color composition outfan, generate the 3rd signal, the 3rd signal comprises the first visible color component-part diagram picture of collection.The combination of first, second, and third signal is the image that comprises bandwidth restricted image and fluoroscopic image.

The method also on the 3rd visible color composition input of a plurality of visible color composition inputs, receives the second visible color component-part diagram picture that gathers.In aspect this, the first and the 3rd signal comprises the combination of the second visible color component-part diagram picture of the first visible color component-part diagram picture of collection and collection.Secondary signal comprises that the fluoroscopic image of collection adds the combination of the second visible color component-part diagram picture of the first visible color component-part diagram picture of collection and collection.

In aspect different, the method receives the second fluoroscopic image that gathers on the 3rd visible color composition input of a plurality of visible color composition inputs.In aspect this, the 3rd signal is the combination of the second fluoroscopic image of the first visible color component-part diagram picture of gathering and collection.

Description of drawings

Fig. 1 illustrates the view of the height summary of Wicresoft's operated from a distance surgery systems, and wherein this surgery systems comprises the 3 D visible system of enhancing.

Fig. 2 illustrates the schematic diagram of hardware and software (image is processed and the user interface) aspect of the 3 D visible system of enhancing.

Fig. 3 illustrates by the example flow chart of the process carried out of the enhancing 3 D visible system of Wicresoft's operated from a distance surgery systems as shown in Figure 1.

Fig. 4 illustrates the sequential in the system among Fig. 2 and Fig. 3, synchronously and an aspect of catching.

Fig. 5 A illustrates the expression of the normal color stereo-picture of the system acquisition of using Fig. 2.

Fig. 5 B illustrates the expression with the bandwidth restricted stereo-picture of the stack fluoroscopic image that highlights, and this bandwidth restricted stereo-picture uses the system acquisition of Fig. 2.

Fig. 6 is the block diagram of an aspect of enhancing image correction process shown in Figure 2.

When Fig. 7 illustrates sequential in Fig. 2 and the system shown in Figure 3, synchronously and another aspect of catching.

Fig. 8 is the block diagram of another aspect of enhancing image correction process shown in Figure 2.

In the accompanying drawings, the accompanying drawing that occurs first therein with the element of this reference marker of the first digit of reference marker indication.

The specific embodiment

As used herein, the electronic three-dimensional imaging comprises use two imaging bands (that is, the passage of left image and right image).

As used herein, the stereoptics path comprises two passages (for example, the passage of left image and right image) for the endoscope of the light that transmits self-organizing.The different views of the light representative tissue that in each passage, is transmitted.Light can comprise one or more images.In the situation that be without loss of generality or the suitability, the each side that the below more completely describes can also be used in the background of an order (field sequantial) stereo acquisition system and/or a sequential display system.

As employed in this article, exposure pathways comprises to tissue provides path in the endoscope of irradiation.

As using in this article, the image of catching in visible electromagnetic radiation spectrum is called as the visible images of collection.

As using in this article, white light is visible white light, and it is become to be grouped into by three kinds of (perhaps more kinds of) visible color, for example, and red visible color composition, green visible color composition and blue visible color composition.Therefore, white light has a plurality of visible color compositions.If provide the visible color composition by irradiator, then the visible color composition is called as visible color irradiation composition.White light can also refer to a more continuous spectrum in the visible spectrum, for example, and as what can from the tungsten filament of heating, see.

As using in this article, the black white image that uses irradiator to generate is called as the bandwidth restricted image, and wherein the composition that provides of irradiator lacks than all a plurality of visible color irradiation compositions of white light.

As using in this article, because fluorescence and captive image is called as the fluoroscopic image of collection in this article.There is multiple fluorescence imaging form.For example, cause fluorescence by using injectable dyestuff, fluorescin or fluorescent-labeled antibody.For example, can be excited by laser or other energy sources and cause fluorescence.Fluoroscopic image can provide for the important information in the crucial patient body of operation, such as pathological information (for example, fluorescence tumor) or anatomic information (for example, fluorescent labeling tendon).

By merge three-dimensional normal visible images and have fluoroscopic image stack replaceable solid thereon the bandwidth restricted visible images both, each aspect of the present invention has strengthened the three-dimensional video-frequency of minimally invasive surgery system and has caught and checked ability, and this minimally invasive surgery system is the da of Intuitive Surgical Inc.'s sale of the inferior Sunnyvale of markon's welfare for example

Wicresoft's operated from a distance surgery systems.(da

The registered trade mark of Intuitive Surgical Inc. of the inferior Sunnyvale of markon's welfare).The three-dimensional bandwidth restricted visible images that highlights fluoroscopic image with the quilt of stack provides to have pathological information and/or is highlighted stereo-picture for the anatomic information operative site of operation.The fluoroscopic image that is highlighted is determined clinical interested tissue.

Offered in real time with the three-dimensional bandwidth restricted image of the fluoroscopic image that highlights of stack and just to use Wicresoft's operated from a distance surgery systems to carry out operating surgeon.Continuous acquisition method (also being considered to time slicing) cause to postpone, and then to use two frames obtaining in different time points to be generated as the single frame of surgeon's demonstration relevant with catching a stereo-picture in the frame and fluoroscopic image in another frame in this delays.Therefore, with respect to service time burst in the system of three-dimensional color visible images stack fluoroscopic image, memorizer and the processing requirements of the system of describing herein are lowered.

Three-dimensional bandwidth restricted visible images is to form by using than all a plurality of visible color irradiation compositions composition still less that forms white light, so lost the colouring information in the bandwidth restricted visible images, but had little loss or not loss on the details.Three-dimensional bandwidth restricted visible images is enough to determine dissection, organize boundary mark and operating theater instruments, so that this image allows the safety operation of operating theater instruments.Owing to having the bandwidth restricted visible images, compare with fluoroscopic image, do not exist because visible color is shone the loss that the interference of composition causes.

Fluoroscopic image covers on the bandwidth restricted visible images, and color is increased, thereby the quantity of information about operative site of raising is provided, and this has reduced the risk of patient harm, and has improved operation efficient.The combination of the fluoroscopic image that this three-dimensional bandwidth restricted visible images and quilt are highlighted provides multiple benefit, include, but are not limited to, allow the surgeon to be identified in real time the positive tumor boundary of illing tissue's excision, and determine for example its hetero-organization of tendon, thereby avoid unnecessarily cutting this tissue.

Can show continuously three-dimensional bandwidth restricted image and the combination of fluoroscopic image to the surgeon.Replacedly, can between opening and closing, switch covering of two kinds of images folded (for example, at da

On the surgery systems surgeon control station, by using pedal or pointing clasper by double-clicking master control).

Fig. 1 illustrates the perform an operation view of height summary of system 100 of the long-range behaviour of Wicresoft, and this system 100 is da for example

Surgery systems, wherein this surgery systems 100 comprises the 3 D visible system of enhancing.In this example, surgeon's remote control of use surgeon control station 114 is installed in the endoscope 112 on the robotic manipulator arms 113.Also have and da

Other parts that surgery systems is relevant, cable etc., but these are also not shown in Fig. 1, to avoid weakening the disclosure.Can be for example about the further information of minimally invasive surgery system, U.S. Patent application No.11/762165(2007 submitted to June 13; Minimally invasive surgery system is disclosed) and the submission on the 18th of US Patent No. 6331181(2001 December; Openly surgical operation robot instrument, data framework and use) find in, wherein these two applications are included in this for your guidance.

Such as following more complete explanation, the irradiation system (not shown) is called as irradiator sometimes, and it is coupled to endoscope 112.Irradiation system optionally provide in following (a) white light and (b) than all visible color irradiation compositions composition and at least a fluorescence excitation irradiation composition still less of white light.Light from irradiation system is coupled at least one exposure pathways in the endoscope 112 by fibre bundle (referring to the fibre bundle 216 among Fig. 2).Light is by at least one exposure pathways in the endoscope 112 and irradiation patient's 111 tissue 103.

In one aspect, endoscope 112 also comprises two optical channels, the white light that it be used for to transmit comes the light of self-organizing for example to reflect, or from the reflected light of (a plurality of) visible color irradiation composition and fluorescence.The white light of reflection is used to form one or more normal visible images.Such as following more complete explanation, the reflected light that shines composition from (a plurality of) visible color is used to form the bandwidth restricted visible images.

When irradiation source just provided white light, in image capture system 120, the white light of self-organizing 103 reflections was hunted down as the normal visible color stereo-picture that gathers.Yet when the surgeon wanted to see the enhancing image, the surgeon changed to observing pattern the observing pattern of enhancing.

In the observing pattern that strengthens, at least one in a plurality of visible color irradiation compositions of disconnection white light is so that irradiation source provides all visible color irradiation compositions composition still less than white light.For example, if use three kinds of visible color irradiation compositions of white light, irradiation source provides maximum two kinds of visible color irradiation compositions.Therefore, aspect this of the observing pattern that strengthens, uses a kind of or two kinds of visible color irradiation compositions irradiation tissues 103, for example use all a plurality of visible color irradiation compositions composition and the fluorescence excitation still less than white light to shine composition.

In the observing pattern that strengthens, the left side of collection and right visible color component-part diagram picture are by catching for the element 121 of each the visible color composition that is provided by irradiator, and fluorescence is caught as left and right fluoroscopic image by element 122.Except any one or a plurality of wave filter that can stop fluorescence is removed, image capture system 120 is conventional image capture systems, and one or more wave filter can be used to stop any direct light or the reflected light of catching from one or more fluorescence excitations source.

Show that image controller 130 receives the information that gathers from image capture system 120.When the information that gathers is normal visible color stereo-picture, show the normal visible color stereo-picture that image controller 130 processing gather, it comprises the image that color correction gathers, and will be sent to through the visible color stereo-picture of the collection of color correction the observer at surgeon's control station 114 places, these control station 114 displayed map pictures.

Similarly, in the observing pattern that strengthens, show that image controller 130 receives the information that gathers from image capture system 120, the visible three-dimensional color component image of the collection of each the visible color composition that is namely provided by irradiation source and a left side and the right fluoroscopic image of collection.In the observing pattern that strengthens, show that image controller 130 uses the enhancing image correction process to replace normal color to proofread and correct and processes.

Strengthening image correction process uses the image of (a plurality of) the visible three-dimensional color component that gathers to generate the bandwidth restricted image.For example, if irradiation source provides two visible color irradiation compositions, i.e. the first visible color irradiation composition and the second visible color irradiation composition, the visible color component-part diagram of collection similarly are the combination of the second visible color component-part diagram picture of the first visible color component-part diagram picture of gathering and collection.

Show that the enhancing image correction process in the image controller 130 has made up the first and second visible color component-part diagram pictures that gather, and this combination is provided each color component input of the display to surgeon's control station 114.Display generates the bandwidth restricted image.Because image does not comprise the visible color component-part diagram picture for the collection of the 3rd visible color irradiation composition, so image is the bandwidth restricted image.

In addition, strengthen in the color component input that image correction process is added into the fluoroscopic image that gathers display, so that in the color component input receives the combination that fluoroscopic image adds the first and second visible color component-part diagram pictures of collection.Two other color component input that is input to display only receives the combination of the first and second visible color component-part diagram pictures of collection.In this example, if fluoroscopic image is added into green composition, the surgeon sees the similar three-dimensional black white image of operative site, and wherein tissue or other anatomical features generate the fluorescence that highlights as green in this black white image.

Now jointly referring to figs. 2 to Fig. 4, Fig. 2 is being described in more detail of each side of an example of minimally invasive surgery system 100 shown in Figure 1.Fig. 3 is the flow chart of system's operation among Fig. 2, and Fig. 4 is the irradiation tissue in system shown in Figure 2 and the sequential chart of catching image.

In embodiment illustrated in fig. 2, minimally invasive surgery system 200 comprises the irradiator of combined light source 210.Combined light source 210 comprises visible color composition source 211 and fluorescence excitation source 212.As long as combined light source 210 has the ability of following more complete description, source 211 and 212 specific implementation are exactly unessential.

At irradiation organizational process 302(Fig. 3) in, combined light source 210 at least one exposure pathways in stereo endoscope 201 uses, with irradiation tissue 203.In this example, combined light source 210 has two kinds of operator schemes: normal observing pattern and enhancing observing pattern.

In normal observing pattern, the irradiation that visible color composition source 211 provides in order to white light tissue 203, that is, and all the visible color irradiation composition sources in the use source 211.In normal observing pattern, do not use fluorescence excitation source 212.

In strengthening observing pattern, visible color composition source 211 provides than organizing with white light all visible color compositions composition still less of 203, for example, does not comprise a kind of or more visible color composition of white light in irradiation.In some respects, for example, possible is all visible color irradiation compositions that use white light in the observing pattern that strengthens, but comprise the inappreciable irradiation from one or more visible color irradiation compositions source, and from the observing pattern irradiation of the enhancing of remaining visible color composition irradiation source.Inappreciable irradiation means that irradiation that visible color irradiation composition source provides is low as to be enough to so that when inappreciable irradiation image and fluorescence were gathered image as common collection together, the inappreciable irradiation image of collection can not reduce the quality of the fluoroscopic image of collection.Therefore, provide inappreciable irradiation for one or more visible color irradiation compositions in fact to be equal to use than all visible color compositions one-tenth still less of white light irradiation tissue of assigning to.

In one aspect, three visible color become to be grouped into white light, and namely white light comprises the first visible color composition, the second visible color composition and the 3rd visible color composition.Each of three visible color compositions is different visible color compositions, for example red composition, green composition and blue composition.Using three kinds of visible color compositions is the signal explanation of a plurality of this compositions with the formation white light, rather than the intention restriction.

In strengthening observing pattern, fluorescence excitation source 212 provides the fluorescence excitation irradiation composition of the fluorescence that excites self-organizing 203.For example, be used to the specific fluorescence of excite tissue from the narrow band light in fluorescence excitation source 212, so that the fluoroscopic image of the particular organization in the capture tissue 203.

In the observing pattern that strengthens, the quantity of the visible color irradiation composition that is provided by visible color composition source 211 depends on the quantity of the different fluoroscopic images of catching.In this example, if catch a fluoroscopic image, then provide one or two different visible color irradiation compositions by visible color composition source 211.If catch two different fluoroscopic images, then provide a visible color irradiation composition by visible color composition source 211.

In one aspect, visible color composition source 211 comprises the source for each the different visible color irradiation composition in a plurality of visible color irradiation compositions of white light.In one example, for the RGB implementation, this source is light emitting diode (LED), i.e. a red LED, two green LED and a blue led.Table 1 provides the scope of the output wavelength of each LED that uses in this example.

Table 1

Visible color irradiation composition	Wavelength
		Red	670 nanometers (nm)
Green 1	555nm
		Green
2	532nm
		Blue	450nm

Using LED in visible color composition source 211 only is the signal explanation, is not the intention restriction.For example, can also substitute LED with a plurality of lasing light emitters and realize visible color composition source 211.Replacedly, visible color composition source 211 can be used with the xenon lamp of oval retroeflector and band filter coating, thereby is formed for the broadband white color irradiation light of visible images.The use of xenon lamp is not the intention restriction equally only as the signal explanation.For example, can use high pressure mercury arc lamp, other arc lights or other lamp sources, broadband.In the observing pattern that strengthens, in order to remove one or more visible color irradiation compositions from this provenance, band filter, prism etc. can be merged in combined light source 210.

Similarly, in enhancement mode, if fluorescence exciting wavelength appears in the visible spectrum, visible color composition source 211(Fig. 3 B so) can be used as visible color irradiation composition source and fluorescence excitation irradiation composition source.If fluorescence exciting wavelength appears at visible spectrum outer (for example, in the near infrared region (NIR)), then laser module (perhaps other energy sources, for example light emitting diode or the white light that is filtered) is used as fluorescence excitation source 212.

Therefore, in one aspect in, the light of the laser module in the fluorescence origin autofluorescence excitaton source 212 triggers.As example, use the laser excitation antibody reagent of 525nm, this antibody reagent obtains from Medarex company.

The specific fluorescent excitaton source of selecting for combined light source 210 depends on one or more used fluorescence.Present exciting and emission maximum of the various FDA that use in the body in table 2, wherein FDA is the fluorescent dye through approval.

Table 2

Fluorescent dye	Excitation maximum (nm)	Emission maximum (nm)
			Fluorescein	494	521
Fox Green	810	830
			Indigo carmine	In alkaline solution 436	In alkaline solution 528
Methylene blue	664	682

Table 3 presents the example for the common protein fluorescence group of biosystem.

Table 3

^*Approximate exciting and fluorescent emission conjugation maximum.

Those skilled in the art understand fluorescence and can be incorporated into reagent, then are bonded to again patient's particular organization.When selecting specific fluorogen, combined light source 210 comprises fluorescence excitation source 212, and it has the light that excites maximum wavelength for this fluorescence provides.Therefore, suppose to use interested one or more fluorogens and many different fluorogens, suitable light source can be included in the combined light source 210.

Example in above-mentioned table 2 and the table 3 only for the signal explanation, is not intended this aspect is limited to the specific example that presents.In view of the disclosure, can select interchangeable imaging of tissue characteristic, then based on the fluorescence that is used, can select suitable light source.

In normal or the observing pattern that strengthens, be imported into fibre bundle 216 from the light of light source or a plurality of light sources.Fibre bundle 216 provides exposure pathways to the stereo endoscope 201 with light, and this exposure pathways is directed to light again tissue 203.

By double-click master control clasper, Sound control and other similar method of switching of example such as foot switch, control operating theater instruments, can between normal and the observing pattern that strengthens, switch the video output on the three-dimensional display 241.The Fig. 2 that switches in that is used for changing between two kinds of observing patterns is expressed as display mode selection 250.

In response to user's input of selecting 250 from display mode, signal is provided to observing pattern checking process 301(Fig. 3 in user interface 260), when selecting normal observing pattern, again control signal is provided to irradiation organizational process 302.In one aspect, generate user interface 260 by computer code, wherein computer code is stored in the memorizer 132, carries out (Fig. 1) at processor 131.

In one aspect, normal observing pattern is default mode.Selecting 250 can not be used at, display mode aspect this, until the surgeon wants observing pattern is changed to the enhancing observing pattern from normal observing pattern, perhaps changing to normal observing pattern from strengthening observing pattern.

In normal observing pattern, irradiation organizational process 302 is sent to power and horizontal controller 215 in the combined light source 210 with normal observing pattern operation signal.Power and horizontal controller 215 are shown, and to be arranged in combined light source 210 be for easy, is not intention with the position limitation of power and horizontal controller 215 in this particular location.

In response to normal observing pattern operation signal, if source 212 is opened, power and horizontal controller 215 disconnect fluorescence excitation source 212, and enable visible color composition source 211, in order to provide white light to tissue 203.For example, when visible color composition source 211 comprises three kinds of visible color irradiation composition source, provide power/power to each of three provenances.Those skilled in the art recognize that for each provenance in 210, replace switching on and off power/power, controller 215 can be kept power/power and connect all the time, and will be from the export orientation in source to fibre bundle 216 with leave fiber optics 216 guiding, and realizes identical result.

Therefore, in normal observing pattern, irradiation organizational process 302 causes organizing 203 by white light.In the figure of irradiation shown in Figure 4, horizontal axis is the time, vertical axis representative source output level.Every kind of source output level in normal observing pattern operating process in three kinds of visible color irradiation composition is defined as 100%.Therefore, for moment before the t1 constantly among Fig. 4, be shown as 100% from each the output level in three kinds of visible color irradiation compositions, and the output level of fluorescence excitation irradiation composition is zero.

Come the visible light (Fig. 2) of self-organizing 203 by the stereoptics path in the endoscope 202, arrive image capture system 220.In aspect this, image capture system 220 comprises conventional stereo camera, and it comprises left image charge coupling device (CCD) 221A and right image C CD222A.

Therefore, in normal observing pattern, catch image process 303(Fig. 3) in, left image C CD 221A catches the left image 421A(of visible light Fig. 4), and right image C CD 222A catches the right image 422A of visible light.Left image C CD 221A catches redness, green and the blue image of the left image 421A of visible light, and the left image that namely gathers is coloured image.Similarly, right image C CD 222A catches redness, green and the blue image of the right image 422A of visible light.

Left image C CD 221A can be each a plurality of CCD that catch different visible color compositions with right image C CD 222A; Has the single CCD in the different CCD zone of catching specific visible color composition etc.Three chip ccd sensors only are shown.Can also use single cmos image sensor or three CMOS color image sensor assemblies with color filter array.

In normal observing pattern, be to show image controller 230(Fig. 2) the normal visible left visible images 421A of collection and the right image 422A(of normal visible light Fig. 4 of collection be provided), wherein show image controller 230 carries out image processing process 304(Fig. 3).In image processing process 304, color image trimming process 231 is processed the normal visible left visible images 421A of collection and the right image 422A of normal visible light of collection.Be sent to three-dimensional display 241 through the normal visible left visible images of the collection of color correction with through the right image of normal visible light of the collection of color correction, and organize in the Stereoscopic Video Presentation process 305 in generation and to show three-dimensional color image.

Processing in the normal observing pattern is equivalent to the processing in the conventional minimally invasive surgery system, is therefore known by those skilled in the art.Similarly, for each frame repeats process 301 to 305, so that the surgeon sees the real time video image of tissue 203.

In normal observing pattern process, provide the normal three-dimensional colour view 500A(Fig. 5 A that organizes 203A to the surgeon).Yet the surgeon may wish to see in organizing the 3-D view of 203A area-of-interest or a plurality of area-of-interest among the 203A organized that is highlighted.For example, the surgeon may wish to see the ill part of organizing 203A and/or particular organization's (for example tendon or organ).Therefore, at moment t1 place (Fig. 4), the surgeon uses display mode to select 250 observing pattern become the enhancing observing pattern.

In response to user's input of selecting 250 from display mode, the demonstration selection signal of enhancing is provided to the observing pattern checking process 301 in the user interface 260.Select signal in response to the demonstration that strengthens, checking process 301 will strengthen the image control signal to be provided to irradiation organizational process 302 and image processing process 304.

In response to the display control signal that strengthens, irradiation organizational process 302 is sent to power and horizontal controller 215 in the combined light source 210 with the shows signal that strengthens.In response to the shows signal that strengthens, power and horizontal controller 215 are connected fluorescence excitation source 212, and in this example, disconnect the 3rd visible color irradiation composition in the visible color composition source 211, so that only the first and second visible color irradiation composition and fluorescence excitation irradiation composition are provided to fibre bundle 216.

Therefore, use the first and second visible color irradiation composition, rather than the 3rd visible color irradiation composition irradiation tissue 203.Can also use fluorescence excitation irradiation composition 212 irradiation tissues 203.

Similarly, in one embodiment, power and horizontal controller 215 reduce the output level of the first and second visible color irradiation composition, for example the output level are reduced to 1/10th.Therefore, as shown in Figure 4, after moment t1, the output level of the first and second visible color irradiation composition reduces with respect to the output level before the moment t1, and the 3rd visible color composition irradiation output level is zero.Similarly, connect fluorescence excitation irradiation composition.

In this example, by CCD or part CCD, be that the 3rd visible color composition is caught the fluorescence that is excited by fluorescence excitation irradiation composition.For example, if fluorescence is in the near infrared region, the 3rd visible color composition is red composition, and the first and second visible color compositions are blue and green composition.

Replacedly, if fluorescence is in the scope of green visible color composition, the 3rd visible color irradiation composition will be green visible color composition, and will be disconnected in source 211, and the first and second visible color compositions are red and blue composition.If fluorescence is in ultraviolet ray range, the 3rd visible color irradiation composition will be blue composition, and will be disconnected in source 211, and the first and second visible color compositions are red and green composition.

As using in this article, " first ", " second " and " the 3rd " are for being used for distinguishing the adjective of visible color composition.Therefore, " first ", " second " and " the 3rd " are not intended to hint any order of the visible color composition in the visible wavelength spectrum.

In one aspect, for guaranteeing to organize on 203 the wavelength that is in or approaches the visible color irradiation composition that is disconnected without incident illumination, approach with the wavelength of the visible color irradiation composition that is disconnected or the wavelength of the overlapping colored visible color irradiation of vicinity (a plurality of) composition is being blocked to a certain degree.This guarantees that the fluoroscopic image that gathers can not cut down by any visible light that CCD caught of catching fluoroscopic image.For example, when visible color composition source 211 has four provenances that provide in the table 1, and the 3rd visible color irradiation composition is when being red visible color composition, in the observing pattern that strengthens, and the same green light source with the contiguous red light source wavelength of wavelength that disconnects.

Come the light (Fig. 2) of self-organizing 203 to arrive image capture system 220 through the stereoptics path in the endoscope 202.In one aspect, before image was hunted down, wave filter 221B and 222B were used to filter any reflection or the direct light from fluorescence excitation source 212.

In catching image process 303 (image 3), in the observing pattern that strengthens, left image C CD 221A catches a left side first and second visible color component-part diagram pictures of collection and left fluoroscopic image 421B(Fig. 4 of collection), and right image C CD 222A catches the right side first and second visible color component-part diagram pictures of collection and the right fluoroscopic image 422B of collection.For example, if the 3rd visible color composition is red visible color composition, the blueness among the left image C CD 221A and green CCD are respectively left image capturing blueness and green left visible color component-part diagram picture.Red CCD among the left image C CD 221A catches left fluoroscopic image.Similarly, the blueness among the right image C CD 222A and green CCD are respectively right image capturing blueness and green visible color component-part diagram picture.Red CCD among the right image C CD 222A catches right fluoroscopic image.

In this example, there is not the 3rd visible color composition irradiation source, therefore do not exist the 3rd visible color of self-organizing 203 reflections to become light splitting.This is so that the 3rd visible color of usually catching in the image becomes the CCD of light splitting or part CCD to can be used for catching fluoroscopic image.Therefore, need not video camera, the optical path in the endoscope that adds or the endoscope of adding and gather visible color component-part diagram picture and fluoroscopic image.At this, when the visible color component-part diagram picture in the statement image was relevant with visible color composition irradiation source, expression visible color composition irradiation source provided the light of the visible color component-part diagram picture in the synthetic image.

In the observing pattern that strengthens, a left side the first and second visible color component-part diagram pictures that gather and the left fluoroscopic image of collection, and the right fluoroscopic image of the right side the first and second visible color component-part diagram pictures that gather and collection is provided for the image controller 230 of carries out image processing process 304.Above the reference, as mentioned above, image processing process 304 has received the shows signal of enhancing.Therefore, image processing process 304 changes to from color image trimming process 231 and strengthens image correction process 232.

Strengthen image correction process 232 and on left and right image, all carry out identical process, therefore in description, do not consider a left side and right name.Fig. 6 is the input message that strengthens image correction process 232A for being input to of this example, and from the block diagram of the output information that strengthens image correction process 232A output.

Strengthen image correction process 232A and receive the first visible color component-part diagram of collection as AVC1 at the first color component input; Receive the second visible color component-part diagram of collection as AVC2 at the second color component input; And the fluoroscopic image AF that receives collection at the 3rd color component input.Strengthen the first visible color component-part diagram that image correction process 232A combination gathers as the second visible color component-part diagram of AVC1 and collection as AVC2, for example form component-part diagram picture average of two collections, be used for generating the bandwidth restricted visible images.The color component of each bandwidth restricted image is visible color component-part diagram picture average of two collections.

In this example, strengthen the second visible color composition that image correction process 232 is added into the fluoroscopic image AF that gathers the Bandwidth-Constrained image, this Bandwidth-Constrained image for the first visible color component-part diagram of gathering as the combination as AVC2 of the second visible color component-part diagram of AVC1 and collection.The result who adds is provided to the second visible color composition outfan.In this example, the second visible color composition outfan is the outfan of visible color composition, and empirical studies shows that this visible color composition is that the surgeon is preferred, for example green composition.

In this example, the first visible color component-part diagram of collection is sent to each the first and the 3rd visible color composition outfan as the second visible color component-part diagram of AVC1 and collection as the combination of AVC2.The first visible color component-part diagram that the fluoroscopic image AF that gathers adds collection is sent to the second visible color composition outfan as the combination as AVC2 of the second visible color component-part diagram of AVC1 and collection.It will be understood by those skilled in the art that, for example carry out with respect to the frame subelement based on pixel about strengthening image correction process 232 the operation described, and it is symbolistic " adding ", and may require the gamma correction of cancelling and reform, for example to realize clearly image.

Organize (Fig. 3) in the Stereoscopic Video Presentation process 305 in generation, at three-dimensional display 241(Fig. 2) show the output of self-reinforcing image correction process 232.In the observing pattern that strengthens, difference implementation 301 to 305 is so that the surgeon is referring to the real-time video enhancing image of tissue 203.

During the observing pattern that strengthens, provide the three-dimensional bandwidth restricted image of tissue 203 to the surgeon, area-of-interest 503(Fig. 5 B in this three-dimensional bandwidth restricted image) highlighted with particular color.In this example, wherein use two kinds of visible color composition irradiation sources, the bandwidth restricted image of tissue 203 is monochromatic bandwidth restricted images.

When the image 203A among Fig. 5 A is full-color and image 203B when being the bandwidth restricted monochrome image, details and information level among two figure equate, but cell picture 203B comprises the additional information that highlights 503.Be fast for the conversion between the observing pattern of

image

203A and 203B, and occur in real time.

The processing that strengthen to show can not cause any delay, and need not as with post processing frame storage frame afterwards.Process and to carry out in real time, thus 3-D view highlight part all the time with the monochromatic Partial synchronization of the bandwidth restricted of 3-D view.By contrast, when the frame that comprises fluoroscopic image is stored, then by in time with the frame registration that occurs after a while, the position that highlights tissue may be changed, and the image that therefore highlights when shown and physical location depart from.

When at moment t2(Fig. 4) locate, when the surgeon wanted that observing pattern is changed back to normal three-dimensional color image, the surgeon used display mode to select 250(Fig. 2) observing pattern is changed to normal observing pattern.In response to change, fluorescence excitation source 212 is disconnected; The 3rd visible color irradiation composition is switched on; And the first and second visible color irradiation composition is returned to fully output level.Referring to Fig. 4, this processing is described identical with above-mentioned normal observing pattern, therefore repeats no more.

In one aspect, by carrying out the image correction module 135(Fig. 1 that is stored in the memorizer 132 at processor 131), carry out image processing process 304(Fig. 3).In this example, image correction module 135 comprises color module 136 and the module 137 that strengthens.When processor 131 is carried out color module 136, carry out color image trimming process 231.When processor 131 is carried out the module 137 that strengthens, carry out enhancing image correction process 232.Two modules 136,137 use are shown.For example, it will be understood by those skilled in the art that two modules can be implemented as individual module.

Although process 304 as herein described is included in execution module on the processor, should understand, the software that the process 304 with process 231 and 232 can be carried out by any hardware, at processor in practice and the combination of firmware are implemented.Similarly, such as function described here, can carry out by unit, perhaps divide between different parts, the software that each in the parts can be carried out by hardware, at processor again and any combination of firmware are implemented.When dividing between different parts, parts can be concentrated in a position or be distributed in the system 100 for the purpose of distributed treatment.

Above-mentioned example only is used for the signal explanation, is not intended restriction.In the observing pattern that strengthens, can use the various combinations of fluorescence excitation source and visible color composition irradiation source.For example, the surgeon wants to observe illing tissue's (for example fluorescigenic tumor) and other particular organizations (for example fluorescigenic tendon), thereby avoids cutting this tissue.Therefore, use two kinds of different fluorogens, it needs fluorescence excitation source 212(Fig. 2) in two kinds of different fluorescence excitations sources (referring to Fig. 7).

At moment t3 place, in response to the display control signal that strengthens, irradiation organizational process 302 is sent to power and horizontal controller 215 in the combined light source 210 with the shows signal that strengthens.In response to the shows signal that strengthens, power and horizontal controller 215 are connected two kinds of fluorescence excitation irradiation compositions in the fluorescence excitation source 212, and in this example, disconnect the second and the 3rd visible color irradiation composition in the visible color composition source 211, so that only the first visible color irradiation composition, the first fluorescence excitation irradiation composition and the second fluorescence excitation irradiation composition are provided to fibre bundle 216.

With the first visible color irradiation composition, rather than the second and the 3rd visible color irradiation composition irradiation tissue 203.Equally, with the first and second fluorescence excitations irradiation composition irradiation tissue 203.

Similarly, in one embodiment, power and horizontal controller 215 reduce the output level of the first visible color irradiation composition, and for example output level is reduced to 1/10th.Therefore, as shown in Figure 7, after moment t3, the output level of the first visible color irradiation composition reduces with respect to the output level before the moment t3, and the second and the 3rd visible color irradiation composition output level is zero.Similarly, connect the first and second fluorescence excitations irradiation composition.

Come self-organizing 203(Fig. 2) light arrive image capture system 220 by the stereoptics path in the endoscope 202.In one aspect, before fluoroscopic image was hunted down, wave filter 221B and 222B were used to filter any reflection or the direct light from two fluorescence excitation irradiation compositions in fluorescence excitation source 212.

Therefore, catching image process 303(Fig. 3) in, in the observing pattern that strengthens, left image C CD 221A catches a left side first visible color component-part diagram picture of collection, a left side first fluoroscopic image of collection and a left side second fluoroscopic image 721B(Fig. 7 of collection).Right image C CD 222A catches the right side first visible color component-part diagram picture of collection, the right side first fluoroscopic image of collection and the right side second fluoroscopic image 722B(Fig. 7 of collection).

For example, if the 3rd visible color composition is red visible color composition, and the second visible color composition is blue, and the green CCD among the left image C CD 221A is the green left visible color component-part diagram picture of left image capturing.Red CCD among the left image C CD 221A catches left the second fluoroscopic image, and the blue CCD among the left image C CD 221 catches left the first fluoroscopic image.Similarly, the green CCD among the right image C CD 222A is the green visible color component-part diagram of right image capturing picture.Red CCD among the right image C CD 222A catches right the second fluoroscopic image, and the blue CCD among the right image C CD 222A catches right the first fluoroscopic image.

In this example, there is not the second visible color composition irradiation source, do not have the 3rd visible color composition irradiation source yet.Therefore, in the light that comes self-organizing 203, do not comprise the light related with the second and the 3rd visible color irradiation composition.This becomes the CCD of light splitting or CCD partly to can be used for catching fluoroscopic image so that usually catch second in the image with the 3rd visible color.

Above the reference, as mentioned above, image processing process 304 has received the shows signal of enhancing.Therefore, image processing process 304 changes to enhancing image correction process 232 from the color image trimming process 231 of

image

721B, 722B.

In addition, strengthen image correction process 232 and carry out identical process at left and right image, therefore in description, do not consider left and right name.Fig. 8 is input to the input message that strengthens image correction process 232B, and from the block diagram of the output information that strengthens image correction process 232B output.

Strengthen image correction process 232B and receive the first visible color component-part diagram of collection as AVC1 at the first color component input; Receive the first fluoroscopic image AF1 that gathers at the second color component input; And the second fluoroscopic image AF2 that receives collection at the 3rd color component input.Strengthen the average level amount that image correction process 232 also forms the visible color component-part diagram picture that gathers, to generate the visible images of bandwidth restricted, but because only gather a visible color component-part diagram picture, so average magnitude is that the first visible color component-part diagram of gathering is as AVC1.In this example, each color component of bandwidth restricted image is that the first visible color component-part diagram of gathering is as AVC1.

In this example, the first fluoroscopic image AF1 that enhancing image correction process 232B will gather is added into the second color component of bandwidth restricted image, and the result is provided to the outfan of the second visible color composition, and wherein this bandwidth restricted image is that the first visible color component-part diagram of gathering is as AVC1.The second fluoroscopic image AF2 that enhancing image correction process 232B also will gather is added into the 3rd color component of bandwidth restricted image, and the result is provided to the outfan of the 3rd visible color composition, and this bandwidth restricted image is that the first visible color component-part diagram of gathering is as AVC1.In this implementation, comprise from the pixel data of image A F1 with from any pixel of the pixel data of image A F2 and present certain Neutral colour.

In this example, the first visible color component-part diagram of collection is sent to the first visible color composition outfan as AVC1.The the first visible color component-part diagram that gathers adds that as AVC1 the first fluoroscopic image AF1 of collection is sent to the second visible color composition outfan.The the first visible color component-part diagram that gathers adds that as AVC1 the second fluoroscopic image AF2 of collection is sent to the output of the 3rd visible color composition.

Organize Stereoscopic Video Presentation process 305(Fig. 3 in generation) in, at three-dimensional display 241(Fig. 2) show the output of self-reinforcing image correction process 232.In the observing pattern that strengthens, difference implementation 301 to 305 is so that the image that the real-time video of surgeon's tissues observed 203 strengthens.

Therefore, in strengthening the observing pattern process, provide the three-dimensional bandwidth restricted image of tissue 203 to the surgeon, interested first area highlights with particular color in this three-dimensional bandwidth restricted image, and interested second area highlights with different colors.

Again, the processing that strengthen to show can not cause any delay, and need not as with post processing frame storage frame afterwards.Process and to carry out in real time, and therefore image highlight part all the time with the monochromatic Partial synchronization of the bandwidth restricted of image.

At moment t4(Fig. 7) locate, when the surgeon wanted that observing pattern is changed back to normal three-dimensional color image, the surgeon used display mode to select 250(Fig. 2) observing pattern is changed to normal observing pattern.In response to change, two kinds of fluorescence excitation irradiation compositions in the fluorescence excitation source 212 are disconnected, and the second and the 3rd visible color irradiation composition is switched on, and the first visible color irradiation composition is back to fully output level.Referring to Fig. 7, this processing is described identical with above-mentioned normal observing pattern, therefore repeats no more.

In the foregoing description, be close to endoscope video camera is installed.Yet this only is schematically, is not intended restriction.As long as video camera can gather by the used image from the photogenerated of optics, no matter the relative position of viewing optics device and video camera how, this process is all worked in the same manner.For example, process described herein and the structure endoscope that can have chip (chip-on-stick) on bar uses.The endoscope that has chip on the bar has short optics part, and wherein video camera is positioned at the optics rear near endoscope tip just.

Equally, stereo endoscope uses as example.This equally only is schematically, and is not intended to limit.In above-mentioned example, can use monoscope endoscope to substitute stereo endoscope.As mentioned above, by means of monoscope endoscope, only can acquisition and processing in the left and right image one.

Illustrate the foregoing description of each side of the present invention and each embodiment and accompanying drawing should not be regarded as restrictive because claim limits shielded invention.In the situation of the spirit and scope that do not deviate from this description and claim, can make various machineries, that consist of, structure, electric and change operation.In some instances, for fear of fuzzy the present invention, not shown or describe well-known circuit, structure and technology in detail.

Further, the technical term of this description is not intended to limit the present invention.For example, the spatial relationship term, such as " ... under ", " below ", " bottom " " ... on ", " top ", " near-end/nearside ", " far-end/distally " and similar terms can be used to describe shown in the accompanying drawing relation of an element or feature and another element or feature.Except the position shown in the accompanying drawing and orientation, these spatial relationship terms also are intended to comprise different parts (being the position) and the orientation (being gyrobearing) of employed device with operating.For example, if the device in the accompanying drawing is reversed, be described as other elements or feature " under " or the element of " below ", then will other elements or feature " on " or " top ".Therefore, on exemplary term " below " can comprise and under position and orientation.Device can be by other orientation (90-degree rotation or be in other orientations), and space correlation representation used herein is correspondingly explained.Similarly, along comprising various specific devices position and orientation with the description that centers on various axial-movements.

Unless context refers else, otherwise singulative " ", " one " and " should/described " also intention comprise plural form.Term " contains ", " comprising ", " comprising " etc. regulation have described feature, step, operation, element and/or parts, but do not get rid of existence or the increase of one or more other features, step, operation, element, parts and/or group.The parts that are described as being coupled can be direct-couplings electrically or mechanically, and perhaps they can be by one or more intermediate member INDIRECT COUPLING.

Memorizer refers to volatile memory, nonvolatile memory or both any combinations.Processor is coupled to the memorizer that contains the instruction of being carried out by processor.This can realize in computer system, perhaps replacedly, is connected digital interface is connected another computer with the digital carrier connection connection with mimic bus via modem and realizes.

At this, computer program comprises non-provisional medium, this non-provisional medium is configured to store about strengthening any one or the needed computer-readable code of its any combination in the operation that display system describes, perhaps in this non-provisional medium storage about any one or the computer-readable code of its any combination in the operation that strengthens display system and describe.The example of some computer programs be on CD-ROM CD, DVD CD, flash memory, ROM card, floppy disk, tape, computer hard disc driver, the network server and through the signal of the expression computer readable program code of Internet Transmission.Non-provisional tangible computer program comprises non-provisional tangible medium, this non-provisional tangible medium is configured to store any one or the computer-readable instruction of its any combination in the operation of describing about the display system that strengthens, any one in the operation that perhaps storage is described about the display system that strengthens in this non-provisional tangible medium or the computer-readable instruction of its any combination.Non-provisional tangible computer program is CD-ROM CD, DVD CD, flash memory, ROM card, floppy disk, tape, computer hard disc driver and other non-interim physical storage mediums.

In view of the disclosure, can in the various computing systems configuration, implement about strengthening in the operation that display system describes any one or its any instruction that is used in combination by the interested operating system of user and computer programming language.

All examples and exemplary reference all are nonrestrictive, should not be used to claim is limited to specific implementation as herein described and embodiment and equivalent thereof.Because the body part under title can cross reference or is applied to body part under the one or more titles, so title only is form, and should not be used to limit by any way theme.Finally, in view of the disclosure, even do not specifically illustrate in the accompanying drawings or describe at body part, the special characteristic of describing about an aspect or embodiment also can be applied to other open aspect or embodiment of the present invention.

Claims

1. minimally invasive surgery system, it comprises:

(i) video camera, wherein said video camera are caught first and are gathered component-part diagram picture and the second collection component-part diagram picture that is different from described the first collection component-part diagram picture;

(ii) controller, it is connected to described video camera,

Wherein said controller generates and comprises that described first gathers the first visible color composition of the bandwidth restricted image of component-part diagram picture;

Wherein said controller generates and comprises that described first gathers the second visible color composition of the described bandwidth restricted image of component-part diagram picture, and described the second visible color composition that described the second collection component-part diagram is looked like to be added into described bandwidth restricted image gathers component-part diagram as the two so that the described second visible color composition of described bandwidth restricted image comprises described the first collection component-part diagram picture and described second;

Wherein said controller generates and comprises that described first gathers the 3rd visible color composition of the described bandwidth restricted image of component-part diagram picture; And

Further, wherein said bandwidth restricted image does not comprise the visible color component-part diagram picture of the collection that visible color composition by white light produces; And

(iii) display, it is connected to described controller, thereby receive first, second, and third composition of described bandwidth restricted image, wherein said display shows the image that comprises described bandwidth restricted image, and gathers the image section of component-part diagram picture with respect to not comprising that or not described second image section that gathers the component-part diagram picture is highlighted comprising described second.

2. system according to claim 1 further comprises:

Irradiator, wherein said irradiator provide at least two kinds of irradiation compositions simultaneously, and wherein when being fluorescence excitation irradiation composition for one in the described irradiation composition, other irradiation compositions comprise all the visible color compositions composition still less than white light.

3. system according to claim 2,

Wherein said other irradiation compositions comprise the first visible color irradiation composition; And

Wherein said first gathers the first visible color component-part diagram picture that component-part diagram similarly is collection; And described the second collection component-part diagram similarly is the fluoroscopic image that is excited by described fluorescence excitation irradiation composition.

4. system according to claim 2, wherein said other irradiation compositions comprise described the first visible color irradiation composition and the second visible color irradiation composition.

5. system according to claim 4, wherein said video camera is caught to win the third place and is gathered the component-part diagram picture, and wherein said first to gather component-part diagram similarly be the first visible color component-part diagram picture that gathers; Described second gathers the fluoroscopic image that component-part diagram similarly is collection; And the described the 3rd gathers the second visible color component-part diagram picture that component-part diagram similarly is collection.

6. system according to claim 5,

Wherein said controller looks like to generate described first, second, and third color component of described bandwidth restricted image from the first and second visible color component-part diagrams of described collection;

7. system according to claim 6,

Wherein said the first visible color composition is blue composition, and described the second visible color composition is green composition.

8. system according to claim 6, described second color component of wherein said bandwidth restricted image is green composition.

9. system according to claim 2, wherein said other irradiation compositions are the first visible illumination composition and the second fluorescence excitation irradiation composition.

10. system according to claim 9, wherein said video camera is caught to win the third place and is gathered the component-part diagram picture, and wherein said first to gather component-part diagram similarly be the first visible color component-part diagram picture that gathers; Described second gathers the first fluoroscopic image that component-part diagram similarly is collection; And the described the 3rd gathers the second fluoroscopic image that component-part diagram similarly is collection.

11. system according to claim 10,

Wherein said controller is added into described the 3rd images acquired described the 3rd visible color composition of described bandwidth restricted image.

12. system according to claim 2, wherein said irradiator comprises:

The first visible color irradiation composition source;

The second visible color irradiation composition source, wherein said the second visible color irradiation composition is different from described the first visible color irradiation composition;

The 3rd visible color irradiation composition source,

Wherein said the 3rd visible color irradiation composition is different from described the second visible color irradiation composition, and is different from described the first visible color irradiation composition; And

The fluorescence excitation irradiation source; And

Further, wherein said system comprises:

Power level and power-supply controller of electric, it is connected to described first, second, and third visible color composition irradiation source, and is connected to described fluorescence excitation irradiation source; And

Pattern changes device, and it is coupled to described power level and power-supply controller of electric, and this pattern changes utensil the first state and the second state,

Wherein, when described pattern change utensil has described the first state, described power level and power-supply controller of electric (a) provide power to described first, second, and third visible color irradiation composition source, and power is not provided to described fluorescence excitation source, and (b) described first, second, and third visible color irradiation composition source has the first illumination levels; And

Wherein, when described pattern change utensil has described the second state, described power level and power-supply controller of electric (a) provide power to described the first visible color irradiation composition source, provide power at least part of described the second visible color irradiation composition source and described fluorescence excitation source, and do not provide power to described the 3rd visible color irradiation composition source, and the described illumination levels that (b) reduces described the first visible color irradiation composition source and described the second visible color irradiation composition source.

13. a method that generates the image demonstration that strengthens in minimally invasive surgery system, it comprises:

By controller, generate the first visible color composition of the bandwidth restricted image that comprises the first collection component-part diagram picture;

Generate the second visible color composition of the described bandwidth restricted image that comprises described the first collection component-part diagram picture by described controller, and gather component-part diagram with second and look like to be added into described the second visible color composition, wherein said second gathers composition represents the fluorescence that excited by the irradiation from the first fluorescence excitation irradiation source;

Pass through controller, generation comprises that described first gathers the 3rd visible color composition of the described bandwidth restricted image of component-part diagram picture, and wherein said bandwidth restricted image does not comprise the visible color component-part diagram picture by the collection of a generation in the visible color irradiation composition of white light; And

Showing that screen display comprises the image of described bandwidth restricted image, wherein comprising that with respect to not comprising that described second gathers the image section of component-part diagram picture, highlighting described second gathers the image section of component-part diagram picture.

14. method according to claim 13,

Wherein said first gathers the first visible color component-part diagram picture that component-part diagram similarly is collection.

15. method according to claim 13,

Wherein said first gathers the first visible color component-part diagram picture that component-part diagram similarly is collection, and described method further comprises:

By controller, receive the 3rd and gather the component-part diagram picture, the wherein said the 3rd gathers the second visible color component-part diagram picture that component-part diagram similarly is collection.

16. method according to claim 15,

Wherein said generation the first visible color composition also comprises and comprises that the described the 3rd gathers the component-part diagram picture;

Wherein said generation the second visible color composition also comprises and comprises that the described the 3rd gathers the component-part diagram picture; And

Wherein said generation the 3rd visible color composition also comprises and comprises that the described the 3rd gathers the component-part diagram picture, so that described bandwidth restricted image is the bandwidth restricted monochrome image.

17. method according to claim 13,

Receive the 3rd by described controller and gather the component-part diagram picture, the wherein said the 3rd gathers the second fluoroscopic image that component-part diagram similarly is collection.

18. method according to claim 17,

Wherein said controller gathers described the 3rd color component that component-part diagram looks like to be added into described bandwidth restricted image with the described the 3rd.

19. method according to claim 13 also comprises:

Use is than all a plurality of visible color irradiation compositions composition irradiation operative site still less of white light.

20. a method, it comprises:

Carry out the enhancing image correction module at processor, described execution comprises:

On the first visible color composition input of a plurality of visible color composition inputs, receive the first visible color component-part diagram picture that gathers;

On the second visible color composition input of described a plurality of visible color composition inputs, receive the fluoroscopic image that gathers;

On the first visible color composition outfan, generate the first signal of the first visible color component-part diagram picture that comprises described collection;

On the second visible color composition outfan, generate the secondary signal of combination of the fluoroscopic image of the first visible color component-part diagram picture comprise described collection and described collection; And

On the 3rd visible color composition outfan, generate the 3rd signal of the first visible color component-part diagram picture that comprises described collection, the combination of wherein said first, second, and third signal comprises the image that comprises bandwidth restricted image and described fluoroscopic image.

21. method according to claim 20 also comprises:

On the 3rd visible color composition input of described a plurality of visible color composition inputs, receive the second visible color component-part diagram picture that gathers; And

Wherein said first signal comprises the combination of the second visible color component-part diagram picture of the first visible color component-part diagram picture of described collection and described collection;

Wherein, described secondary signal comprises that the fluoroscopic image of described collection adds the combination of the second visible color component-part diagram picture of the first visible color component-part diagram picture of the above collection and described collection; And

Wherein, described the 3rd signal comprises the combination of the second visible color component-part diagram picture of the first visible color component-part diagram picture of described collection and described collection.

22. method according to claim 19 also comprises:

On the 3rd visible color composition input of described a plurality of visible color composition inputs, receive the second fluoroscopic image that gathers; And

Wherein, described the 3rd signal is the combination of the second fluoroscopic image of the first visible color component-part diagram picture of described collection and described collection.