JP2007244746A - Observation system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an observation system capable of compounding images for estimating a blood vessel pathway of a large flow rate in an endoscopic picture of a lymph vessel of a small flow rate or lymph node. <P>SOLUTION: Three-dimensional image data, such as a CT lymph node detected by a CT image memory 43b, are converted into two-dimensional image data corresponding to a parameter in a section direction by an XYZ processing part 46 composed of an adjustment processing function by a CPU 44 with image data, such as a fluorescent lymph node detected by a fluorescent image memory 43a, as a reference image. Coincidences of size of image data are judged by a first judgement part 47 and, when they do not coincide, adjustment processing is performed by executing expansion and contraction by a scale reduction processing part 48 so that both images coincide. Then, a composite image in which a CT blood vessel image is superimposed upon the fluorescent image is generated and the composite image is displayed to a monitor 8. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an observation system suitable for identifying lymphatic vessels, lymph nodes, blood vessels, and the like existing on the lower layer side of fat at a target site in a living body.

In recent years, endoscope apparatuses have come to be widely used in the medical field. Conventionally, as a technique for identifying blood vessel running by reflection observation, there is a method of observing using an absorption peak of hemoglobin in the near infrared region. However, although this technique can identify blood vessels on the surface of an organ, it is difficult to identify lymphatic vessels, lymph nodes, blood vessels, and the like that are present in the lower fat layer.
Japanese Patent Laid-Open No. 2001-299676 discloses indocyanine green (abbreviated as ICG) to a site to be observed, and displays a lymph node as a visible image by capturing a fluorescence image emitted from the ICG. An apparatus is disclosed.
JP 2001-299676 A

However, although the apparatus of the conventional example of the above publication can identify lymphatic vessels and lymph nodes under fat, it does not function effectively for blood vessels with a large flow rate. That is, in the blood vessel portion where the flow rate is large even when ICG is injected, ICG is rapidly dissipated from the observation target site from the time of injection, and thus there is a drawback that it is difficult to estimate the position path or running position of the blood vessel. .
For this reason, there has been a problem that it is difficult to perform an endoscopic operation or the like smoothly only with fluorescent images of lymph vessels and lymph nodes having a small flow rate obtained by an endoscope.

(Object of invention)
The present invention has been made in view of the above points, and provides an observation system capable of synthesizing an image capable of estimating blood vessel running with a high flow rate in an endoscopic image for a lymph vessel or lymph node with a low flow rate. Objective.

The observation system of the present invention is a state in which excitation light is irradiated to a target region including at least one of a lymph node and lymph vessel with a low blood flow rate and a blood vessel with a high blood flow rate in a living body to which a fluorescent substance is administered. Fluorescence generated by the fluorescent material in the endoscope is received by a fluorescence imaging means of an endoscope, an imaging signal that is photoelectrically converted and output from the fluorescence imaging means is input, signal processing is performed on the imaging signal, and the lymph node And a video processing means for generating a video signal for generating a fluorescent image including at least one reference image of the lymphatic vessel as a first image;
The three-dimensional image generation means inputs a three-dimensional image including at least one of the lymph nodes and lymph vessels and the blood vessels in the target region, and generates a two-dimensional second image from the three-dimensional image. And an adjustment processing means for performing an adjustment process for matching a corresponding image corresponding to the reference image in the second image to the position and size of the reference image;
Synthesizing means for synthesizing at least the image of the blood vessel portion in the two-dimensional image with the fluorescence image in the adjusted state after being adjusted by the adjustment processing means;
It is characterized by comprising.
With the above configuration, the fluorescence image obtained using the endoscope and the adjustment generated by the three-dimensional image generation means and the position and size of the two-dimensional corresponding image corresponding to the reference image in the fluorescence image are adjusted to the reference image. Since the image of the blood vessel portion generated by the three-dimensional image generation means after processing is combined with the fluorescence image by superposition or the like, it can be obtained by fluorescence imaging together with the fluorescence image of lymph nodes or lymph vessels with a small flow rate. An image of a blood vessel part having a large flow rate that is difficult is synthesized so that the blood vessel part can be estimated on an endoscopic image.

  ADVANTAGE OF THE INVENTION According to this invention, the image which can estimate the blood vessel running with a large flow volume and difficult to obtain as a fluorescence image can be synthesize | combined with the fluorescence image of a lymph vessel or a lymph node with a small flow volume. Therefore, according to the present invention, there is an effect that it is easy to perform an operation or the like under an endoscope smoothly.

  Embodiments of the present invention will be described below with reference to the drawings.

1 to 5 relate to Embodiment 1 of the present invention, FIG. 1 shows the overall configuration of the observation system of Embodiment 1 of the present invention, FIG. 2 shows the configuration of an image processing circuit by a camera control unit, and FIG. Shows the transmittance characteristics of the excitation filter and the excitation light cut filter, FIG. 4 shows an image obtained by a CT apparatus, an endoscope, and the like, and FIG. 5 shows the processing contents of the first embodiment.
As shown in FIG. 1, in the observation system 1 according to the first embodiment of the present invention, a camera head 4 having a built-in image pickup unit is mounted on, for example, an optical endoscope 3 whose distal end is inserted into a patient 2 such as the abdomen. Signal processing is performed on a camera-mounted endoscope (hereinafter simply abbreviated as an endoscope) 5, a light source device 6 that supplies illumination light to the optical endoscope 3, and an imaging unit built in the camera head 4. A camera control unit (abbreviated as CCU) 7, a monitor 8 that displays an endoscopic image captured by the imaging means by inputting a standard video signal output from the CCU 7, and 3 of the patient 2 And a CT (computer tomography) device 9 for obtaining a three-dimensional CT image.

The optical endoscope 3 includes, for example, a rigid insertion portion 11, a grip portion 12 provided at the rear end of the insertion portion 11, and an eyepiece portion 13 provided at the rear end of the grip portion 12. The light guide cable 14 is connected to the base of the grip portion 12.
A light guide 15 that transmits illumination light is inserted into the insertion portion 11, and the light guide 15 is provided at an end portion thereof via a light guide cable 14 connected to a base on the side of the grip portion 12. A connector 16 is detachably connected to the light source device 6.
A lamp 18 such as a xenon lamp that is lit by a lamp lighting power source supplied from a lamp lighting control circuit 17 is provided in the light source device 6, and the lamp 18 has a wavelength band from the visible region to the near infrared wavelength band. Generate light to cover.

Only the excitation light passes through the excitation filter 19 disposed on the illumination optical path, and the light from the lamp 18 is further collected by the condenser lens 20 and passes through the light guide of the light guide cable 14 to be an optical endoscope. Illumination light is incident on the three light guides 15. The illumination light incident on the light guide 15 is guided (transmitted) to the tip side. Then, the excitation light is irradiated from the distal end surface of the light guide 15 toward the target site 21 to be observed inside the patient 2 from the distal end surface of the light guide 15.
In this embodiment, indocyanine green (abbreviated as ICG) is injected into the site to be observed, so this excitation filter 19 is set to transmit a wavelength band around 680 nm to 780 nm, for example, as shown in FIG. Has been. In addition, the characteristic shown with the code | symbol 22 shown with a dotted line in FIG. 3 shows the light absorption characteristic of ICG mixed with the lymph. Moreover, the characteristic of the code | symbol 23 shown with a dashed-dotted line has shown the intensity distribution of the fluorescence which ICG generate | occur | produces.

Thus, the transmission characteristics in the wavelength band near 680 nm to 780 nm centered around approximately 730 nm by the excitation filter 19 generate the absorption characteristics and fluorescence of the ICG so as to selectively excite the ICG as the fluorescent material. It is set in consideration of characteristics. It should be noted that the wavelength band near 680 nm to 780 nm does not overlap with the excitation light cut filter 29 as shown in FIG. 3 to be described later and includes the ICG absorption peak wavelength band as much as possible. It is a wavelength band where the absorption is considerably large in a wavelength band slightly apart from the wavelength that becomes the absorption peak.
Moreover, the light guide 15 is arrange | positioned at the front-end | tip part of the insertion part 11, for example in ring shape, and the objective lens 24 is arrange | positioned in the center in the ring. A relay lens system 25 that transmits an optical image is disposed on the image forming position side of the objective lens 24.
The objective lens 24 condenses the light incident from the target portion 21 side, forms an optical image at the image forming position, and the optical image is the relay lens system 25 disposed in the insertion portion 11. Thus, the data is transmitted to the eyepiece 13 side behind the insertion unit 11.

The transmitted optical image can be enlarged and observed through an eyepiece 26 provided on the eyepiece 13 as shown in FIG. When the camera head 4 is attached to the eyepiece 13, the optical image transmitted through the imaging lens 27 in the camera head 4 is, for example, an electric charge as an imaging device having light receiving sensitivity up to the near infrared. An image is formed on a coupling element (abbreviated as CCD) 28. An excitation light cut filter 29 that cuts the excitation light and transmits the fluorescence wavelength by ICG is disposed in the imaging optical path immediately before the CCD 28.
As shown in FIG. 3, the excitation light cut filter 29 is set to a characteristic that transmits the intensity peak of fluorescence generated by ICG injected into the target region 21 and does not transmit the excitation light.
For this reason, in the example of FIG. 3, the excitation light cut filter 29 is from a value slightly shorter than this wavelength to a longer wavelength side than 830 nm, for example, 1000 nm so as to include 830 nm of the wavelength that is the intensity peak of ICG fluorescence. It is set to the characteristic that passes through. FIG. 3 shows an example in which a longer wavelength than 1000 nm is transmitted.

Accordingly, the fluorescence generated by the ICG is input to the camera head 4 side together with the reflected light reflected by the excitation light from the target region 21 side. The reflected light is shielded by the excitation light cut filter 29, and the CCD 28 The optical image that is actually formed becomes an image by fluorescence, that is, a fluorescence image. Thus, the CCD 28 constitutes a fluorescence imaging means.
Further, the camera cable 30 extended from the camera head 4 is connected to the CCU 7. The CCU 7 includes a CCD drive circuit 31 and an image processing circuit 32, and the CCD drive circuit 31 applies a CCD drive signal to the CCD 28.
Then, the CCD 28 performs photoelectric conversion by the application of the CCD drive signal, and an imaging signal (fluorescence imaging signal) corresponding to fluorescence imaging is input to the image processing circuit 32. The image processing circuit 32 constitutes video processing means for generating a video signal for the input fluorescent imaging signal.

The video signal generated by the image processing circuit 32 is output to the monitor 8, and the fluorescent image captured by the CCD 28 is displayed on the display surface of the monitor 8.
In addition, the image processing circuit 32 of the CCU 7 stores a pre-operative three-dimensional CT image of the patient 2 imaged by the CT apparatus 9. Then, as will be described later, adjustment processing is performed by the image processing circuit 32, and the two-dimensional image of the blood vessel in the CT image is combined with the fluorescence image as the endoscopic image captured by the CCD 28, for example, by superposition, and monitored. 8 is displayed. In addition, as schematically shown in FIG. 1, when the blood vessel 35, the lymphatic vessel 36, and the lymph node 37 are running in a mixed manner within the adipose tissue 34 that substantially constitutes the target region 21 in the patient 2. As will be described later, the endoscope 5 captures fluorescent images of the lymphatic vessels 36 and lymph nodes 37 having a small blood flow rate. On the other hand, it is difficult to obtain a fluorescent image for the blood vessel 35 portion where the blood flow rate is large.

For this reason, the image processing circuit 32 performs an adjustment process for adjusting the position and size of the three-dimensional image information for the blood vessel 35 obtained by the CT apparatus 9 as the three-dimensional image generation unit to the fluorescence image. Image processing is performed in which a three-dimensional image is displayed on the monitor 8 while being superimposed on the fluorescence image.
By superimposing the images in this way, for example, the operator can synthesize and display a two-dimensional image of the blood vessel 35 with a large flow rate on the fluorescence image obtained by the endoscope 5 so that the operator can In this case, the running state of the blood vessel can be grasped or estimated.
FIG. 2 shows the configuration of the image processing circuit 32 having the function of synthesizing the image of the blood vessel 35 by the CT apparatus 9 with the fluorescence image. The fluorescence imaging signal from the CCD 28 is converted from an analog signal to a digital signal (image) by the A / D converter 41 from the first input end, and then input to the preprocessing circuit 42.

The preprocessing circuit 42 performs preprocessing such as removing a halation portion in the input image. The fluorescence image output from the preprocessing circuit 42 is input to the fluorescence image memory 43a and stored in the fluorescence image memory 43a.
The fluorescent image stored in the fluorescent image memory 43a is extracted (detected) by the fluorescent lymph node detection circuit 45a to extract (detect) the fluorescent lymph node and fluorescent lymph vessel portion in the fluorescent image, and the extracted fluorescent lymph node and fluorescent lymph vessel are extracted. The image data of the part is input as a reference image to be referred to, for example, an XYZ processing unit 46 configured by an adjustment processing function by the CPU 44.
Extraction (detection) of fluorescent lymph nodes and fluorescent lymph vessel portions may be automatically performed by an image processing means having a feature amount extraction function, but a portion or range to be extracted by a user such as an operator from a keyboard or the like. Etc. may be marked (designated), and the marked part, range, etc. may be extracted.

  In the following description, the fluorescent lymph node detection circuit 45a is described as extracting the fluorescent lymph node and the fluorescent lymph vessel portion in the fluorescent image, but only one of them may be used, or a representative one of them may be used. It may be a position or a part. For example, only the fluorescent lymph node in the fluorescent image or a part of the fluorescent lymph node may be used by the fluorescent lymph node detection circuit 45a. Similarly, the fluorescent lymph node detection circuit 45a may extract only the fluorescent lymph vessel portion or a part thereof in the fluorescent image. This embodiment can also be applied to the case where only one of the fluorescent lymph node and the fluorescent lymph vessel is observed as a fluorescent image. The three-dimensional image data output from the CT apparatus 9 is stored in the CT image memory 43b from the second input end. The CT apparatus 9 can generate three-dimensional CT image data of the blood vessel 35, the lymphatic vessel 36, and the lymph node 37 by CT imaging.

The three-dimensional CT image data stored in the CT image memory 43b is extracted (detected) from the CT lymph node and CT lymph vessel portion in the CT image by the CT lymph node detection circuit 45b, and the changeover switch The data is input to the XYZ processing unit 46 configured by, for example, the adjustment processing function of the CPU 44 via the SW.
When the CT lymph node detection circuit 45b extracts (detects) three-dimensional image data of a CT lymph node and a CT lymph vessel portion in a CT image, it is automatically performed by an image processing means having a feature amount extraction function. However, a user such as an operator may mark (specify) a portion to be extracted, a range, or the like from a keyboard or the like, and extract the marked portion, range, or the like. Particularly in this case, it is desirable to extract a corresponding portion corresponding to the reference image on the fluorescent lymph node and fluorescent lymph vessel portion side as a corresponding image.

When three-dimensional image data such as a CT lymph node is first extracted from the CT image memory 43b by the CT lymph node detection circuit 45b, for example, a parameter i indicating the number of processings related to the direction of the body position of the patient 2 is set to an initial value i. = 1 and the three-dimensional image data of the CT lymph node and CT lymph vessel portion extracted in the state of the parameter is output to the XYZ processing unit 46.
Further, the CT image stored in the CT image memory 43b is extracted (detected) by the CT blood vessel detection circuit 45c from the CT blood vessel portion in the CT image, and the XYZ processing unit 46 is connected via the changeover switch SW. Is input.
In this case, the three-dimensional image data from the CT lymph node detection circuit 45b side is input to the XYZ processing unit 46 during the adjustment process (position and size adjustment), and a coincidence determination signal is output from the first determination unit 47. Then, the changeover switch SW is switched by this determination signal, and the three-dimensional image data of the CT blood vessel detection circuit 45c is input.

As described above, the three-dimensional image data of the CT lymph node and the CT lymph vessel part are input to the XYZ processing unit 46 during the adjustment processing. j is converted into two-dimensional image data.
That is, in order to be able to compare with the two-dimensional image data of the fluorescent lymph node and the fluorescent lymph vessel, the XYZ processing unit 46 performs the fluorescent lymph node and the cross-section with the cross section set by the parameter j (initial value j = 1). The three-dimensional image data of the fluorescent lymphatic vessel is converted into two-dimensional image data, and the converted two-dimensional image data is output to the first determination unit 47.
The first determination unit 47 compares and determines the CT lymph node and CT lymph vessel image data output from the XYZ processing unit 46 and the fluorescent lymph node and fluorescent lymph vessel image data. If the values match within the threshold value Vt, the first determination unit 47 outputs a match determination signal to the superimposition processing unit 50 and switches the changeover switch SW.

When the first determination unit 47 performs comparison determination of both image data, the first determination unit 47 compares whether or not the distance between a plurality of representative points in each image data match in both image data, that is, the sizes match. If the comparison determination matches within the threshold value Vt, then the comparison determination of the position regarding whether the positions of the representative points in both image data match only within the threshold value Vt, for example, by only parallel movement. I do. If the size and position match, the first determination unit 47 outputs a determination signal for matching.
In this case, the three-dimensional image data of the CT blood vessel portion detected by the CT blood vessel detection circuit 45c is converted into two-dimensional image data in the cross-sectional direction in the XYZ processing unit 46, and this two-dimensional image data is the first determination. The data is output to the superimposition processing unit 50 via the unit 47. In this case, the first determination unit 47 performs offset adjustment for alignment by parallel movement.

On the other hand, if it is determined that they do not match within the threshold Vt, the CT lymph node and CT lymph vessel image data output from the XYZ processing unit 46 is sent to the scale processing unit 48 via the first determination unit 47. The scale processing unit 48 performs processing for enlarging or reducing the image data of the CT lymph nodes and CT lymph vessels according to the determination result (comparison result). If only the positions do not match, the information is sent from the first determination unit 47 to the second determination unit 49.
That is, when it is determined that the CT lymph node side is larger than the fluorescent lymph node side, the scale processing unit 48 performs the reduction process at a predetermined reduction rate, and conversely, the CT lymph node side is determined to be smaller than the fluorescent lymph node side. If it does, the enlargement process is performed at a predetermined enlargement rate.
The image data scaled by the scale processing unit 48 is input again to the first determination unit 47, compared and determined, and the enlargement or reduction process and the determination process are repeated.

When the first determination unit 47 determines that the values match within the threshold value Vt, the first determination unit 47 outputs a match determination signal to the superimposition processing unit 50 as described above. In this case, the CT blood vessel image data is input to the superposition processing unit 50 via the scale processing unit 48 with the cross-sectional direction and scale in this state, and the superposition processing unit 50 receives the CT blood vessel image data and the fluorescent lymph node. A superimposed image in which the fluorescence image data of the detection circuit 45a is superimposed is output.
On the other hand, the first determination unit 47 changes the parameter j in the cross-sectional direction of the XYZ processing unit 46 when it is determined that the scale processing by the scale processing unit 48 does not match within the threshold value Vt even if it is repeatedly performed. Then, the same processing is performed in different directions. In this way, when the number of times of processing by changing the parameter j in the cross-sectional direction of the XYZ processing unit 46 exceeds the predetermined number n, and when the sizes match but the positions do not match, the first determination unit 47 Information on the determination result is output to the second determination unit 49.

When the processing parameter i related to the body posture direction or the like is within m, the second determination unit 49 sends a signal for changing the parameter i to the CT image memory 43b and is extracted by the CT lymph node detection circuit 45b. The direction of the CT lymph node or the like is changed and output to the XYZ processing unit 46. In this way, the processing parameter i is changed and the CPU 44 performs the same processing.
When the coincidence determination signal is output from the first determination unit 47, the superimposition processing unit 50 sends the superimposed image to the D / A converter 51, and the D / A converter 51 converts the superimposed image data into analog data. An image signal (video signal) is converted and output to the monitor 8, and a superimposed image in which the fluorescence image and the CT image are superimposed is displayed on the display surface of the monitor 8.
Note that when the coincidence determination signal is not input from the first determination unit 47, the superimposition processing unit 50 passes through the fluorescent image stored in the fluorescent image memory 43a without performing the superimposition process, and the D / A converter To 51. Alternatively, the fluorescent image stored in the fluorescent image memory 43a is always output to the monitor 8 in a state shifted from the superimposed image output from the superimposing processing unit 50, and this fluorescent image is always displayed on the monitor 8. It may be configured.

The operation of this embodiment having such a configuration will be described below with reference to the flowchart of FIG.
In the first step S1 of FIG. 5, CT imaging is performed by the CT apparatus 9 so as to include the peripheral portion of the target site to be operated on the patient 2 who is to perform a surgical operation with the endoscope 5, The device 9 generates a three-dimensional CT image. This three-dimensional CT image is stored in the CT image memory 43b of the CCU 7 connected to the CT apparatus 9.
FIG. 4A shows an example of a CT image obtained by the CT apparatus 9. FIG. 4A schematically shows an example of a CT image viewed from a predetermined direction. For example, when the blood vessel 35, the lymphatic vessel 36 and the lymph node 37 are included in the target region 21 shown in FIG. 1, CT images of the blood vessel 35a, the lymphatic vessel 36a and the lymph node 37a are obtained correspondingly. It is done.

In the next step S <b> 2, an operator who performs a surgical operation performs a treatment of administering or injecting ICG as a fluorescent material around the target site 21 of the patient 2 by injection or the like. In the next step S <b> 3, the surgeon inserts the insertion portion 11 of the endoscope 5 into the body of the patient 2 and starts fluorescence observation on the target site 21.
When this fluorescence observation is started, a fluorescence image obtained by the endoscope 5 is displayed on the monitor 8 as an endoscope image by the CCU 7. In this case, as shown in FIG. 4 (B), by imaging the fluorescence due to the administration of ICG, a fluorescence image of the lymph vessel 36b and lymph node 37b where the blood flow rate is small, that is, where the ICG is likely to stay is displayed. Is done. As an example of reference image display, FIG. 4D shows an image display example in the case of normal observation using reflected light instead of fluorescence observation. In this case, since all the excitation light that has passed through the excitation filter 19 is cut by the excitation light cut filter 29 and the image is taken, the image pickup condition is such that almost no image is obtained. It is indicated by a dotted line.

Along with the display of the fluorescent image, the image processing circuit 32 in the CCU 7 starts the processing in step S4 and subsequent steps, and obtains a CT image (more specifically, a CT blood vessel image) obtained by the CT apparatus 9 by the endoscope 5. A process for generating a superimposed image in which the size and position of the fluorescent image to be aligned is performed.
That is, as shown in step S4, the CT lymph node detection circuit 45b extracts the three-dimensional image data of the CT lymph node and the CT lymph vessel from the CT image memory 43b and outputs them to the XYZ processing unit 46 of the CPU 44. In this case, the three-dimensional image data extracted by the CT lymph node detection circuit 45b is obtained by setting the processing parameter i related to the direction such as the posture of the patient 2 to the initial value 1. By changing the value of the parameter i, three-dimensional image data in different directions (orientations) is extracted by the CT lymph node detection circuit 45b.

As shown in step S <b> 5, the XYZ processing unit 46 converts the image data of 3D to 2D CT lymph nodes and CT lymph vessels, and outputs the image data to the first determination unit 47.
In this case, the XYZ processing unit 46 sets the parameter j in the cross-sectional direction to an initial value 1. As shown in step S6a, the first determination unit 47 determines whether the image data on the CT lymph node side and the image data on the fluorescent lymph node side coincide with each other within the threshold Vt in size and position (by adjusting only the translation). The comparison judgment of no is performed.
If it is determined that they match, the first determination unit 47 switches the selector switch SW according to a determination signal indicating that they match as shown in step S7. Then, the three-dimensional image data of the CT blood vessel image extracted from the CT image memory 43b by the CT blood vessel detection circuit 45c is converted into two-dimensional image data by the XYZ processing unit 46, and passes through the first determination unit 47 and the scale processing unit 48. It is made to input into the superimposition process part 50. FIG.

That is, according to the determination result of step S6a, the size and alignment adjustment processing state (parameter i, j, k setting state) by the CPU 44 is a fluorescent image and a CT image (two-dimensional image) as a corresponding image corresponding thereto. ), The two-dimensional image of the blood vessel in the CT image is generated by the CPU 44 in this adjustment processing state, and this two-dimensional image is sent to the superimposition processing unit 50. It should be noted that the sizes of the two images are the same, and the amount of misalignment due to only the parallel movement is eliminated by offset correction of the parallel movement amount.
As shown in the next step S8, the CT blood vessel image and the fluorescence image on the fluorescent lymph node side extracted through the fluorescent lymph node detection circuit 45a are superimposed by the superimposition processing unit 50, and the monitor 8 is superimposed on FIG. A composite image as shown in FIG. In FIG. 5, after the process of step S <b> 8, the process may return to step S <b> 3 again, and the same process may be performed on the endoscopic image (fluorescence image) in a different frame.

On the other hand, if the first determination unit 47 determines that the sizes do not match in the determination process in step S6a, the first determination unit 47 determines whether the sizes match as shown in step S6b. If the sizes match and the positions do not match even during translation (because it is necessary to change the orientation of the CT image), the first determination unit 47 and the process of increasing the parameter i by one (step S15). ) To return to the process of step S4.
On the other hand, if the sizes do not match, the first determination unit 47 sends the image data on the CT lymph node side to the scale processing unit 48 together with the comparison result information, as shown in step S9.
The scale processing unit 48 performs processing for enlarging or reducing the image data on the CT lymph node side at a constant ratio according to the comparison result, and returns the processing to the first determination unit 47.

The first determination unit 47 sequentially counts an initial value of the parameter k of the number of scale processing times from which the image data on the CT lymph node side enlarged or reduced from the scale processing unit 48 is input, and the value of the parameter k For example, it is determined whether or not is a predetermined number of times p or more (step S10). If k ≧ p is not satisfied, the first determination unit 47 sets k = k + 1 (step S11) and repeats the comparison process of step S6a.
On the other hand, when the condition of k ≧ p is satisfied, as shown in step S12, the first determination unit 47 determines whether the parameter j is n or more a predetermined number of times, that is, whether j ≧ n. If j ≧ n is not satisfied, the first determination unit 47 sets j = j + 1 (step S13) and returns to the process of step S5. In this case, the same processing is repeated by changing the parameters in the cross-sectional direction.

Further, when the condition of step S12 is satisfied, as shown in step S14, the first determination unit 47 determines whether or not the parameter i is a predetermined number m or more, that is, whether i ≧ m. If i ≧ m is not satisfied, the first determination unit 47 sets i = i + 1 (step S15) and returns to the process of step S4. In this case, the processing parameter i is changed and the same processing is repeated.
Further, in the determination process of step S14, when the condition of i ≧ m is satisfied, the process for adjusting the size and superimposing is terminated. This case corresponds to a case where, for example, the conditions suitable for the conditions for superimposing the CT image and the fluorescence image could not be set under the set parameter range and conditions. Therefore, in this case, the surgeon can be solved by changing the parameter range or the like.

In this embodiment, which operates in this way, the size and position of the CT lymph nodes and lymph vessels in the CT image obtained by the CT apparatus 9 are determined based on the fluorescence lymph nodes and lymph vessels in the fluorescence image obtained by the endoscope 5. Adjustment processing is performed to match the size and position. In the present embodiment, the CT blood vessel image is superimposed on the fluorescence image obtained by the endoscope 5 and displayed on the monitor 8 in the adjusted state.
Therefore, according to the present embodiment, a blood vessel image can be displayed simultaneously with the fluorescent lymph node and the fluorescent lymph vessel even in the fluorescence observation state by the endoscope 5. In other words, an image that can estimate or grasp the running state of the blood vessel is synthesized and displayed in the fluorescence image, so that the operator can easily perform a surgical operation or the like.

In the above description, the adjustment process for adjusting the size and position of the CT lymph node and the CT lymph vessel is performed on the fluorescent lymph node and the fluorescent lymph vessel. Processing may be performed. For example, an adjustment process for adjusting the size and position of a CT lymph node that is a corresponding image may be performed on the fluorescent lymph node.
A fluorescent image obtained by the endoscope 5 and a composite image obtained by superimposing a fluorescent image obtained by the CT apparatus 9 and a CT image including at least two-dimensional blood vessels aligned with the size of the fluorescent image are displayed adjacently. You may do it.
In this case, on the two-dimensional CT image side in the synthesized image, lymph nodes other than blood vessels and lymph vessels may be superimposed and synthesized at the same time.

Next, Embodiment 2 of the present invention will be described with reference to FIGS.
FIG. 6 shows an overall configuration of an observation system 1B according to the second embodiment of the present invention. The observation system 1B includes an endoscope 5B including an optical endoscope 3 and a camera head 4B, a light source device 6B, a CCU 7B, a monitor 8, and a CT device 9. In addition, the same code | symbol is attached | subjected to the same component as Example 1, and the description is abbreviate | omitted.
In the present embodiment, the observation system 1 of the first embodiment can perform normal observation. Therefore, in the observation system 1B shown in FIG. 6, instead of the excitation light cut filter 29 provided in the camera head 4 in FIG. 1, for example, a switching filter 62 that switches a filter arranged in the optical path by a motor 61 is provided. The camera head 4B is employed.

As shown in FIG. 7, the switching filter 62 includes an excitation light cut filter 29 and a normal filter 63 provided in the circumferential direction of the rotating shaft of the motor 61 (see FIG. 6). And the filter arrange | positioned in an optical path can be switched so that it may become one of these filters 29 and 63 by switching operation of the observation mode switch 64 provided in the camera head 4B. The normal filter 63 is a filter that transmits light in the visible region.
Further, the light source device 6B in the present embodiment is provided with a switching filter 66 in which the filter arranged in the illumination optical path is switched by the motor 65 instead of the excitation filter 19 in the light source device 6 of FIG.
The switching filter 66 is provided with a normal filter 63 and an excitation filter 19 as shown in FIG. That is, the switching filter 66 is configured by providing the excitation filter 19 in place of the excitation light cut filter 29 in the switching filter 62 shown in FIG.

As shown in FIG. 6, when a switching operation is performed by the observation mode switching switch 64, the switching instruction signal drives the motor 61 and also drives the motor 65 in the light source device 6B via the CCU 7B.
That is, when a user such as an operator switches the observation mode changeover switch 64 and selects the normal observation mode, the normal filter 63 is used as the optical filter in the switching filters 66 and 62 in the light source device 6B and the camera head 4B, respectively. It will be placed inside.
On the other hand, when the user selects the fluorescence observation mode, the switching filter 66 in the light source device 6B has the excitation filter 19 in the optical path, and the switching filter 62 in the camera head 4B has the excitation light cut filter 29 in the optical path. It becomes. In this case, the configuration is substantially the same as in the first embodiment.

In addition, the CCU 7B in the present embodiment is configured to further include a normal image processing function for a normal image in the CCU 7 of FIG. Specifically, in the image processing circuit 32 shown in FIG. 2, an image (normal image) when captured by the CCD 28 in the normal observation mode in parallel with the fluorescent image memory 43 a provided at the output terminal of the preprocessing circuit 42 is temporarily stored. An image processing circuit 32B provided with a normal image memory for storing is provided.
When the normal observation mode is selected, the normal image captured by the CCD 28 is stored in the normal image memory, the normal image stored in the normal image memory is read at a predetermined period, and the D / A converter 51 is operated. Then, it outputs to the monitor 8.

In the present embodiment, a keyboard 68 is connected to the image processing circuit 32B, and a user such as an operator can use various values of parameters when performing the processing shown in FIG. The range can be specified.
Further, as described in the first embodiment, when the processing of FIG. 5 is performed, the images of the fluorescent lymph nodes and the fluorescent lymph vessels used as the reference images of the portions to be compared in the size and alignment from the fluorescent images. When extracting a part, the user performs marking from the keyboard 68 so that the marked part can be extracted. In addition, when extracting the CT lymph node and CT lymph vessel portion as a corresponding image when aligning the size and position to the reference image also on the CT image side, marking the portion corresponding to the reference image, The part can be extracted.

In addition to the keyboard 68, a pointing device such as a mouse or a joystick may be provided, and the user may set parameters and the like by operating the mouse.
When the user performs manual input of various parameters to perform adjustment processing for aligning the size and position of a CT image with a fluorescent image, the processing is reduced with unnecessary parameter setting conditions, and processing for size and alignment is performed. Can be performed in a short time.
In addition, by specifying (marking) the reference image and the corresponding image when the user performs size and position alignment as described above, the process of size and position alignment can be smoothly performed even for fluorescent images in temporally different frames. It can be carried out.
In this case, the parameters such as the size and position at the time of initial alignment are similarly set to the size and position in a temporally subsequent frame (not limited to a frame after 1/30 sec but a frame with an appropriate period). Adjusting the size and alignment by setting the initial value for the alignment (for example, the CPU 44 stores the information in, for example, an EEPROM as a non-volatile memory, and reads and uses the information in the subsequent frame) By performing the processing, the size and position can be adjusted in a short time.

In addition, when performing size and alignment in such temporally different frames, for example, the first marked part on the reference image side where the position may be temporally moved is changed in temporally different frames. For example, the CPU 44 may perform the process of correcting the position of the corresponding part.
In this way, if the portion marked first on the reference image side can be basically retained as the reference image even if the position of the fluorescent image that is later in time moves, the CT image side Thus, the adjustment process for making the size and position coincide with this becomes simple.

In other words, when aligning the size and position on the CT image side, if the reference image portion in the fluorescence image is first marked with the corresponding image on the CT image side, then the fluorescence image moves and the reference image also moves. Even so, the reference image is corrected and held. For this reason, the same image portion can be used as the corresponding image portion on the CT image side (without determining whether the image corresponds to the reference image), and the adjustment processing for matching the size and position is simplified.
The position correction processing method can be realized by, for example, a method for detecting a motion vector of an area in MPEG2 or 4 or the like. That is, position correction can be performed by detecting to which block the block portion of the predetermined area including the marked portion in the reference image portion has moved in a different frame by, for example, the block matching method. In this case, when the size changes, the position can be corrected with higher accuracy by further performing a process of slightly changing the size.

Therefore, according to the present embodiment, a normal image can be observed, and when the fluorescence observation mode is set, a blood vessel image obtained by the CT apparatus 9 can be synthesized and displayed on the fluorescence image as in the first embodiment. it can. Therefore, the running state of the blood vessel can be estimated or grasped, and the operation under endoscopic observation can be performed smoothly.
In the case of the present embodiment, on the user side, parameters at the time of adjustment processing, that is, conditions and ranges at the time of performing adjustment processing can be made appropriate, and unnecessary adjustment processing can be omitted.
Note that although the case of the CT apparatus 9 has been described as the three-dimensional image generation means, the present invention is not limited to this, and for example, a three-dimensional image by an MRI apparatus may be adopted.
In the above-described embodiment, the excitation light is generated on the light source device 6 side. For example, a light emitting diode (LED) for excitation light is provided at the distal end portion of the endoscope 5 and the excitation light LED is provided. The excitation light may be irradiated to the target site 21 side. In this case, when there is a mode for observing with visible light, a visible light LED that emits light with visible light may be provided, and the LED that emits light may be switched by a mode changeover switch.

  Since the blood vessel image can be superimposed and displayed in the same size and position as the fluorescence image in the fluorescence image obtained by the endoscope, when performing surgery or the like under endoscopic observation, The running state can be grasped and smooth surgery can be performed.

1 is an overall configuration diagram of an observation system according to a first embodiment of the present invention. FIG. 2 is a block diagram showing a configuration of an image processing circuit by the camera control unit. FIG. 3 is a characteristic diagram showing transmittance characteristics and the like of the excitation filter and the excitation light cut filter. FIG. 4 is a diagram showing an image obtained by a CT apparatus and an endoscope. FIG. 5 is a flowchart showing processing contents according to the first embodiment. FIG. 6 is an overall configuration diagram of the observation system according to the second embodiment of the present invention. FIG. 7 is a diagram illustrating a configuration of a switching filter in the camera head. FIG. 8 is a diagram illustrating a configuration of a switching filter in the light source device.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Observation system 2 ... Patient 3 ... Optical endoscope 4 ... Camera head 5 ... Endoscope 6 ... Light source device 7 ... CCU
8 ... Monitor 9 ... CT device 19 ... Excitation filter 28 ... CCD
29 ... Excitation light cut filter 32 ... Image processing circuit 43a ... Fluorescence image memory 43b ... CT image memory 44 ... CPU
45a ... Fluorescent lymph node detection circuit 45b ... CT lymph node detection circuit 45c ... CT blood vessel detection circuit 46 ... XYZ processing unit 47 ... First determination unit 48 ... Scale processing unit 49 ... Second determination unit 50 ... Superimposition processing unit

Claims (5)

The fluorescent substance is generated in a state in which excitation light is irradiated to a target site including at least one of a lymph node and a lymph vessel having a low blood flow rate and a blood vessel having a high blood flow rate in a living body to which the fluorescent substance is administered. Fluorescence is received by the fluorescence imaging means of the endoscope, an imaging signal that is photoelectrically converted and output from the fluorescence imaging means is input, signal processing is performed on the imaging signal, and at least one of the lymph nodes and lymph vessels Video processing means for generating a video signal for generating a fluorescent image including a reference image as a first image;
The three-dimensional image generation means inputs a three-dimensional image including at least one of the lymph nodes and lymph vessels and the blood vessels in the target region, and generates a two-dimensional second image from the three-dimensional image. And an adjustment processing means for performing an adjustment process for matching a corresponding image corresponding to the reference image in the second image to the position and size of the reference image;
Synthesizing means for synthesizing at least the image of the blood vessel portion in the two-dimensional image with the fluorescence image in the adjusted state after being adjusted by the adjustment processing means;
An observation system comprising: Further, excitation light irradiation means for irradiating the target site in the living body to which the fluorescent material is administered with excitation light of a predetermined wavelength band for selectively exciting the fluorescent material;
Receiving the fluorescence generated by the fluorescent material, and transmitting the fluorescence imaging means comprising a filter means that transmits a wavelength band in which the intensity of the fluorescence reaches a peak, and suppresses transmission of the wavelength band of the excitation light;
The observation system according to claim 1, comprising:   The observation according to claim 1, wherein the adjustment processing unit performs a direction adjustment process for adjusting a direction of the corresponding image when the adjustment process is performed to match the corresponding image with a position and a size of the reference image. system. A light generating means for generating light to irradiate a target site including blood vessels, lymph nodes, lymphatic vessels;
A first filter disposed between the light generating means and the target site and transmitting excitation light for exciting the fluorescent material;
A second filter that includes a vicinity of a fluorescence peak of the fluorescent material and transmits only a specific wavelength region that cuts the excitation light;
Imaging means for receiving light transmitted through the second filter and acquiring a first image in which lymphatic vessels and lymph nodes are imaged;
Adjustment processing means for superimposing a second image in which the positions of blood vessels, lymphatic vessels, and lymph nodes of the target region are stored in advance, in accordance with the size and position of the first image;
An observation system comprising:   Further, the marking means for marking a predetermined lymph node or lymph vessel in the first image, and the position of the lymph node or lymph vessel marked by the marking means in the first image of a temporally different frame The observation system according to claim 4, further comprising position correction means for performing position correction.

Images