JP5753105B2 - Electronic endoscope system, image processing apparatus, and method of operating image processing apparatus - Google Patents

Description

The present invention relates to an electronic endoscope system, an image processing apparatus, and an operation method of the image processing apparatus that acquire information related to blood vessels using an electronic endoscope.

  In the medical field, endoscopic diagnosis using an electronic endoscope has become widespread. In recent endoscopic diagnosis, special light observation using special light limited to a specific wavelength is performed in addition to normal observation of observing the overall properties of the surface of living tissue under white light. It has come to be.

  There are various types of special light observations. For example, in the endoscope system of Patent Document 1 by the present applicant, narrowband light in a wavelength region having a difference in the magnitude of the extinction coefficient between oxyhemoglobin and reduced hemoglobin is used. The oxygen saturation of blood hemoglobin is calculated and imaged based on a spectral image obtained by imaging the reflected light.

  In the endoscope system of Patent Document 1, in order to increase the measurement accuracy of oxygen saturation, a plurality of narrowband light spectral images are acquired and the luminance values of corresponding pixels of the spectral images are compared. This eliminates the effects of noise caused by differences in blood vessel depth.

  The amount of reflected light from a living tissue is influenced mainly by scattering in mucous membranes other than blood vessels, in addition to absorption by blood hemoglobin. The light incident on the surface of the mucosa is scattered in the mucosa other than the blood vessels and travels toward the deep mucosa while being attenuated. Part of the light that reaches the blood vessels is absorbed by blood hemoglobin. The scattered light exits from the surface of the mucosa and is observed as reflected light. The longer the distance from the mucosal surface layer to the blood vessel, that is, the deeper the blood vessel depth, the greater the amount of reflected components due to scattering, so even with the same oxygen saturation, the deeper the blood vessel depth, the greater the amount of reflected light. growing. As described above, the reflected light amount of the living tissue is affected by absorption and scattering, and the influence of the scattered light amount on the reflected light amount varies depending on the depth of the blood vessel. Furthermore, the scattering coefficient of a living tissue has a wavelength dependency that varies depending on the wavelength of light, such that the scattering coefficient of the living tissue becomes larger on the short wavelength side than on the long wavelength side.

  On the premise of such light absorption and scattering characteristics of biological tissue, the endoscope system of Patent Document 1 has two narrow wavelength bands having different absorption coefficients and different scattering coefficients of reduced hemoglobin and oxidized hemoglobin. By using the band light, two pieces of information of oxygen saturation and blood vessel depth are acquired. Thereby, highly accurate oxygen saturation information from which the influence of noise due to the difference in blood vessel depth is removed is obtained.

  Of course, the comparison of the luminance values of a plurality of spectral images must be performed between pixels of the same blood vessel region depicted in two spectral images. Since the endoscope system of Patent Document 1 employs a sequential method of sequentially irradiating and imaging a biological tissue with two narrow-band lights as an acquisition method of a plurality of spectral images, a time difference between imaging timings. The positional deviation of a plurality of spectral images caused by the above becomes a problem. The endoscope system of Patent Literature 1 performs frequency filtering processing according to the thickness of a blood vessel on a plurality of spectral images to generate an emphasized image in which a blood vessel region is emphasized, and the generated emphasized image Based on the alignment. By performing the emphasis process, it is possible to perform alignment with higher accuracy than when the emphasis process is not performed.

JP 2011-194151 A

  As described in Patent Document 1, the present applicant is developing a technique for imaging accurate oxygen saturation, but there is room for technical improvement as shown below in the process. I understand that.

  In Patent Document 1, in order to increase the accuracy of alignment, the emphasis image is generated by performing frequency filtering according to the thickness of the blood vessel. However, if the contrast between the blood vessel region and other regions changes between the plurality of spectral images that are the original images, even if the same enhancement processing is performed on each spectral image, it is emphasized in one spectral image. However, there is a case where the blood vessel region to be compared cannot be emphasized in the same manner, for example, there is a blood vessel region that is not emphasized on the other side. This causes a decrease in the accuracy of positional deviation.

  Possible causes of the difference in contrast include differences in the amount of light of a plurality of narrow-band lights, and the fact that biological tissue has wavelength dependency with respect to absorption and scattering. As a measure to eliminate the difference in light quantity, it is conceivable to adjust the light quantity of the light source device. However, the light source device is a combination of a white light source such as a xenon lamp and a color separation filter, or a semiconductor such as a laser or LED. There are various types such as those using a light source, and depending on the type of the light source device, it is difficult to adjust the amount of light. In addition, there is no change in the wavelength dependence of living tissue. Therefore, on the premise of a plurality of spectral images in which a difference in contrast occurs, a technique for improving the alignment accuracy between the spectral images is being sought.

  In addition, as in Patent Document 1, when performing frequency filtering according to the thickness of a blood vessel, it is necessary to accurately grasp the thickness of the blood vessel to be emphasized. If the photographing distance is substantially constant, there is no problem because the thickness of the blood vessel in the spectral image can be grasped in advance. However, when the shooting distance varies due to the movement of the endoscope or when the shooting magnification is changed by the zoom lens, the display magnification of the subject in the shot spectral image changes, so the thickness of the blood vessel drawn in the spectral image is changed. It is also conceivable that the temperature will change greatly. In such a case, since it is necessary to take measures such as changing the processing contents of the frequency filtering according to the change in display magnification, there is a concern that the structure may be complicated.

  The present invention has been made in view of the above problems, and an object thereof is to accurately align a plurality of spectral images with a simple configuration when oxygen saturation is calculated from a plurality of spectral images having different imaging timings. Is to make it possible.

  The electronic endoscope system according to the present invention sequentially irradiates the observation region with at least first and second illumination lights having different wavelength ranges from the electronic endoscope that images the observation region including the blood vessel in the subject. Corresponding to the first and second illumination lights, respectively, from a plurality of imaging signals sequentially output by the electronic endoscope according to the illumination means for performing and the irradiation timing of the first and second illumination lights In order to align the first and second spectral images according to a difference in contrast between the blood vessels in the first and second spectral images, and an image acquisition unit that acquires the first and second spectral images Pre-processing means for performing the pre-processing, and after the pre-processing, an emphasis processing for emphasizing the shape of the blood vessel in each of the first and second spectral images is performed, and the first spectral image and the second spectral image are processed. Alignment corresponding to each Based on the enhancement processing means for generating an image and the first and second alignment images, the first and second blood vessels in the first and second spectral images are overlapped so that the positions of the blood vessels overlap. And oxygen for calculating oxygen saturation of blood hemoglobin in the blood vessel based on the first and second spectral images on which the alignment is performed, and an alignment unit that performs alignment between the two spectral images And a saturation degree calculating means.

  The preprocessing means preferably performs the preprocessing by examining the contrast based on density histograms of the first and second spectral images.

  Preferably, the preprocessing means performs the preprocessing on a lower one of the first and second spectral images with a higher contrast as a reference.

  It is preferable that the preprocessing is a contrast adjustment process for matching the contrast between the first and second spectral images.

  The enhancement processing compares a threshold value set in advance with the pixel values of the first and second images, and distributes the pixel values of the first and second spectral images based on the threshold value. A gradation reduction process for generating the first and second alignment images by reducing the number of gradations of the first and second spectral images, and the pre-processing is performed on the difference in contrast. It is preferable that the threshold value determination process determines the threshold value accordingly.

  Preferably, the gradation reduction process is a binarization process, and the first and second alignment images are binary images.

  It is preferable that at least one of the first and second illumination lights is blue narrow-band light limited to a specific wavelength region in the blue region.

  The illumination means sequentially irradiates three illumination lights obtained by adding third illumination lights having different wavelength ranges to the first and second illumination lights, and the image acquisition means comprises the first to third illuminations. It is preferable that first to third spectral images corresponding to light are acquired, and the oxygen saturation calculating unit calculates the oxygen saturation based on the first to third spectral images.

  For example, the first illumination light is a first blue narrowband light whose wavelength range is limited to 440 ± 10 nm, and the second illumination light is a second blue narrowband whose wavelength range is limited to 470 ± 10 nm. The third illumination light is a third blue narrowband light whose wavelength range is limited to 410 ± 10 nm.

  The illumination means preferably includes first to third semiconductor light sources that emit first to third blue narrow-band light.

  An image processing apparatus according to the present invention is an image processing apparatus that processes an image acquired by using an electronic endoscope that images an observation site including a blood vessel in a subject. Illumination light is sequentially applied to the observation site, and the first and second imaging signals are output from a plurality of imaging signals output from the electronic endoscope according to the irradiation timing of the first and second illumination lights, respectively. Image acquisition means for acquiring first and second spectral images corresponding to the illumination light, and the first and second spectroscopic images in accordance with a difference in contrast between the blood vessels in the first and second spectral images. Pre-processing means for performing pre-processing for aligning images, and after the pre-processing, an emphasis process for emphasizing the shape of the blood vessels in each of the first and second spectral images is performed, and the first and second That of the second spectral image The positions of the blood vessels in the first and second spectral images overlap based on the enhancement processing means for generating the alignment image corresponding to this and the first and second alignment images. As described above, on the basis of the first and second spectral images on which the alignment has been performed, alignment means for performing alignment between the first and second spectral images, and blood hemoglobin in the blood vessel And oxygen saturation calculating means for calculating oxygen saturation.

The operation method of the image processing apparatus of the present invention is acquired by an electronic endoscope imaging an observation site including a blood vessel in a subject that is sequentially irradiated with at least first and second illumination lights having different wavelength ranges. In the operating method of the image processing apparatus for processing an image to be processed , the first and second image acquisition means are configured to output the first and second images from a plurality of imaging signals output by the electronic endoscope in accordance with the irradiation timings of the first and second illumination lights. and Luz step to obtain the first and second spectral images corresponding to the illumination light, the preprocessing means, in accordance with the difference in the contrast of blood vessels in the first and second spectral image, first and second and row mortar step pre-processing to the spectral image alignment position, enhancement processing means, before or after treatment, subjected to emphasizing emphasis processing the shape of the blood vessels in the respective first and second spectral image, Each of the first and second spectral images And Luz step generates an alignment image corresponding to, the alignment means, based on the first and second positioning image, the position of each of the vessels in the first and second spectral image and Luz steps to perform first and second positioning of the spectral image with each other so as to overlap, the oxygen saturation calculating unit, based on the first and second spectral image registration is performed, the vessel characterized in that it comprises a answering step to calculate the blood oxygen saturation of hemoglobin.

  According to the present invention, when the oxygen saturation is calculated based on the first and second spectral images having different imaging timings, the first and second spectral images are subjected to a difference in contrast between the two. Since preprocessing is performed, accurate alignment of a plurality of spectral images can be performed with a simple configuration.

It is an external view of the electronic endoscope system of the present invention. It is a front view of the front-end | tip part of an electronic endoscope. It is a block diagram which shows the electric constitution of an electronic endoscope system. It is a graph which shows the spectrum of illumination light. It is a graph which shows the spectral characteristic of the color microfilter of an image pick-up element. It is explanatory drawing which shows the irradiation timing and imaging timing of illumination light. It is a block diagram of a functional image processing unit. It is a graph which shows the absorption spectrum of hemoglobin. It is a graph which shows the scattering coefficient of a biological tissue. It is explanatory drawing which shows a response | compatibility with the absorption spectrum of hemoglobin, and the wavelength of illumination light. It is a graph which shows the relationship between the luminance value of each narrow-band light, and oxygen saturation. It is a graph which shows the relationship between the luminance value of each narrow-band light, and the blood vessel depth. It is a graph which shows the correlation with 1st and 2 brightness | luminance ratio S1 / S3, S2 / S3, blood vessel depth, and oxygen saturation. (A) is a method for obtaining the coordinates (X ^* , Y ^* ) in the luminance coordinate system from the first and second luminance ratios S1 ^* / S3 ^* , S2 ^* / S3 ^* , and (B) is the coordinates (X ^* , Y blood vessel information coordinate system coordinates corresponding to ^*) (U ^*, it is an explanatory diagram for explaining a method of obtaining the ^{V *).} It is a graph which shows the correlation of the aspect different from FIG. It is explanatory drawing which shows the display form of the blood vessel depth image and an oxygen saturation image. It is explanatory drawing which shows the procedure of the alignment process. It is explanatory drawing of the density histogram of a some spectral image. It is explanatory drawing of a contrast adjustment process. It is explanatory drawing of a positional offset amount detection process. FIG. 18 is an explanatory diagram showing an alignment processing procedure different from FIG. 17. It is explanatory drawing of a threshold value determination process.

[First Embodiment]
As shown in FIG. 1, an endoscope system 10 according to a first embodiment of the present invention includes an electronic endoscope 11 that images an observation site in a subject, and an observation site based on a signal obtained by imaging. A processor device 12 that generates an observation image, a light source device 13 that supplies light for irradiating the observation site, and a monitor 14 that displays the observation image are provided. The processor device 12 is provided with a console 15 that is an operation input unit such as a keyboard and a mouse.

  The electronic endoscope system 10 has two operation modes: a normal observation mode in which an observation site is observed under white light, and a function information observation mode. The function information observation mode is a mode in which special light is used to acquire information related to oxygen saturation and blood vessel depth of blood hemoglobin, which is biological function information related to a living tissue, and image these for observation. .

  The electronic endoscope 11 includes a flexible insertion portion 16 to be inserted into a subject, an operation portion 17 provided at a proximal end portion of the insertion portion 16, an operation portion 17, a processor device 12, and a light source device 13. And a universal cord 18 for connecting the two.

  The insertion portion 16 includes a distal end portion 19, a bending portion 20, and a flexible tube portion 21 that are continuously provided from the distal end. As shown in FIG. 2, on the distal end surface of the distal end portion 19, the illumination window 22 that irradiates the observation site with illumination light, the observation window 23 that receives the image light reflected by the observation site, and the observation window 23 are washed. An air supply / water supply nozzle 24 for performing air supply / water supply, a forceps outlet 25 for projecting a treatment instrument such as forceps and an electric knife, and the like are provided. An imaging element 44 (see FIG. 3) and an imaging optical system are built in the back of the observation window 23.

  The bending portion 20 includes a plurality of connected bending pieces, and is bent in the vertical and horizontal directions by operating the angle knob 26 of the operation portion 17. By bending the bending portion 20, the direction of the distal end portion 19 is directed in a desired direction. The flexible tube portion 21 is flexible so that it can be inserted into a tortuous duct such as the esophagus or the intestine. The insertion unit 16 includes a communication cable that communicates a drive signal for driving the image sensor 44 and an image signal output by the image sensor 44, and a light guide 43 that guides illumination light supplied from the light source device 13 to the illumination window 22. (See FIG. 3) is inserted.

  In addition to the amble knob 26, the operation unit 17 includes a forceps port 27 for inserting a treatment instrument, an air / water supply button for performing air / water supply operation, a release button for taking a still image, and the like. .

  A communication cable and a light guide 43 extending from the insertion portion 16 are inserted into the universal cord 18, and a connector 28 is attached to one end of the processor unit 12 and the light source device 13. The connector 28 is a composite type connector including a communication connector and a light source connector, and one end of a communication cable is disposed on the communication connector, and one end of the light guide 43 is disposed on the light source connector. The electronic endoscope 11 is detachably connected to the processor device 12 and the light source device 13 through the connector 28.

  As shown in FIG. 3, the light source device 13 includes a semiconductor light source unit 31 and a light source control unit 32 that drives and controls them. The light source control unit 32 controls drive timing and synchronization timing of each unit of the light source device 13.

  The semiconductor light source unit 31 has three laser light sources LD1 to LD3 that respectively emit narrowband light limited to a specific wavelength region in the blue region. As shown in FIG. 4, the laser light source LD1 emits narrowband light N1 whose wavelength range is limited to 440 ± 10 nm, preferably 445 nm. The laser light source LD2 emits narrow band light N2 whose wavelength range is limited to 470 ± 10 nm, preferably 473 nm. The laser light source LD3 emits narrow band light N3 whose wavelength range is limited to 400 ± 10 nm, preferably 405 nm. As the laser light sources LD1 to LD3, InGaN-based, InGaNAs-based, and GaNAs-based laser diodes can be used. Further, as the laser light sources LD1 to LD3, a broad area type laser diode having a wide stripe width (waveguide width) capable of increasing output is preferable.

  The light source controller 32 controls turning on / off of the laser light sources LD <b> 1 to LD <b> 3 and the amount of light through the driver 33. The light emitted from the laser light sources LD1 to LD3 is guided to the combiner 36 by the optical fiber 34. The combiner 36 is an optical member having a function of multiplexing the light from each optical fiber 34 and couples the optical axes of the light from each optical fiber 34 that selectively enters into one. A phosphor 37 is provided on the downstream side of the combiner 36.

  As shown in FIG. 4, the phosphor 37 is excited by a narrowband light N1 having a wavelength of 445 nm, and emits fluorescence FL1 in a wavelength region extending from the green region to the red region. The phosphor 37 absorbs a part of the narrowband light N1 to emit fluorescence FL1, and transmits the remaining narrowband light N1. The narrow band light N <b> 1 that passes through the phosphor 37 is diffused by the phosphor 37. White light is generated by the transmitted narrowband light N1 and excited fluorescence FL1, and the generated white light is used as illumination light in the normal observation mode.

As the phosphor 37, for example, a YAG-based or BAM (BgMgAl ₁₀ O ₁₇ ) -based phosphor is used. Further, as shown in FIG. 4, the phosphor 37 is also excited by the narrow-band light N2 having a wavelength of 473 nm and emits the fluorescence FL2. The conversion efficiency to fluorescence is higher in the narrowband light N1 of 445 nm, and the fluorescence FL2 excited by the narrowband light N2 of 473 nm has a smaller amount of light than the fluorescence FL1. The phosphor 37 is hardly excited by the narrow-band light N3 having a wavelength of 405 nm.

  In FIG. 3, a condenser lens 38 and a rod integrator 39 are disposed on the downstream side of the phosphor 37. The condensing lens 38 condenses the light emitted from the phosphor 37 and makes it incident on the rod integrator 39. The rod integrator 39 makes the in-plane light quantity distribution uniform by internally reflecting the incident light, and makes the light enter the incident end of the light guide 43 of the electronic endoscope 11 connected to the light source device 13.

  The electronic endoscope 11 includes a light guide 43, an imaging device 44, an analog processing circuit 45 (AFE: Analog Front End), and an imaging control unit 46. The light guide 43 is a large-diameter optical fiber, a bundle fiber or the like. When the connector 28 on which the incident end of the light guide 43 is disposed is connected to the light source device 13, the incident end is emitted from the rod integrator 39 of the light source device 13. Opposite the edge.

  An illumination lens 48 that adjusts the light distribution angle of illumination light is disposed behind the illumination window 22. The light supplied from the light source device 13 is guided to the irradiation lens 48 by the light guide 43 and irradiated from the illumination window 22 toward the observation site. In the back of the observation window 23, an objective optical system 51 and an image sensor 44 are arranged. The image light reflected by the observation site enters the objective optical system 51 through the observation window 23 and is imaged on the imaging surface 44 a of the imaging element 44 by the objective optical system 51.

  The imaging element 44 is composed of a CCD image sensor, a CMOS image sensor, or the like, and has an imaging surface 44a in which a plurality of photoelectric conversion elements constituting pixels such as photodiodes are arranged in a matrix. The image sensor 44 photoelectrically converts the light received by the imaging surface 44a and accumulates signal charges corresponding to the amount of received light in each pixel. The signal charge is converted into a voltage signal by an amplifier and read out. The voltage signal is output as an imaging signal from the imaging device 44, and the imaging signal is sent to the AFE 45.

  The image pickup device 44 is a color image pickup device, and micro-color filters of three colors B, G, and R having spectral characteristics as shown in FIG. 5 are assigned to each pixel on the image pickup surface 44a. The arrangement of the micro color filter is, for example, a Bayer arrangement.

  As shown in FIG. 6, in the normal observation mode, the image sensor 44 performs an accumulation operation for accumulating signal charges and a read operation for reading the accumulated signal charges within an acquisition period of one frame. As shown in FIG. 6A, in the normal observation mode, the laser light source LD1 is turned on in accordance with the accumulation timing, and the observation site is irradiated with white light including the narrowband light N1 and the fluorescence FL1, The reflected light enters the image sensor 44. The white light is color-separated by a micro color filter, the B pixel receives the reflected light corresponding to the narrowband light N1, the G pixel is the G component in the fluorescence FL1, and the R pixel is the R component in the fluorescence FL1. The reflected light corresponding to is received. The imaging element 44 sequentially outputs the imaging signals B, G, and R for one frame in which the luminance values of the B, G, and R pixels are mixed according to the frame rate. Such an imaging operation is repeated while the normal observation mode is set.

  In the function information observation mode, as shown in FIG. 6B, the laser light sources LD1, LD2, and LD3 are sequentially turned on in accordance with the accumulation timing. When the laser light source LD1 is turned on, as in the normal observation mode, the observation site is irradiated with white light composed of the narrowband light N1 and the fluorescence FL1. When the laser light source LD2 is turned on, the observation site is irradiated with white light composed of the narrowband light N2 and the fluorescence FL2 as illumination light. When the laser light source LD3 is turned on, the narrow band light N3 is turned on as illumination light, and the observation site is irradiated with the narrow band light N3.

  The white light generated by the narrow band light N1 and the fluorescent light FL1, or the white light generated by the narrow band light N2 and the fluorescent light FL2, is color-separated by the micro color filter, and the B pixel is reflected light corresponding to the narrow band light N1 or the narrow band light N2. The G pixel receives the G component in the fluorescence FL1 or fluorescence FL2, and the R pixel receives the reflected light corresponding to the R component in the fluorescence FL1 or fluorescence FL2. The reflected light of the narrowband light N3 is incident only on the B pixel. In the functional information observation mode, as in the normal observation mode, the image sensor 44 sequentially outputs the image signals B, G, and R for one frame in which the luminance values of the B, G, and R pixels are mixed according to the frame rate. To do. Such an imaging operation is repeated while the function information observation mode is set.

  The AFE 45 includes a correlated double sampling circuit (CDS), an automatic gain control circuit (AGC), and an analog / digital converter (A / D) (all not shown). The CDS performs correlated double sampling processing on the image signal from the image sensor 44 to remove noise caused by signal charge reset. The AGC amplifies the imaging signal from which noise has been removed by CDS. The A / D converts the imaging signal amplified by AGC into a digital imaging signal having a gradation value corresponding to a predetermined number of bits and inputs the digital imaging signal to the processor device 12.

  The imaging control unit 46 is connected to a controller 56 in the processor device 12 and inputs a drive signal to the imaging element 44 in synchronization with a base clock signal input from the controller 56. The imaging element 44 outputs an imaging signal to the AFE 45 at a predetermined frame rate based on the drive signal from the imaging control unit 46.

  In addition to the controller 56, the processor device 12 includes a DSP (Digital Signal Processor) 57, an image processing unit 58, a frame memory 59, and a display control circuit 60. The controller 56 includes a CPU, a ROM that stores a control program and setting data necessary for control, a RAM that loads the program and functions as a work memory, and the like. To control.

  The DSP 57 acquires an image signal output from the image sensor 44. The DSP 57 separates an imaging signal in which signals corresponding to B, G, and R pixels are mixed into imaging signals of three colors, performs pixel interpolation processing on the imaging signals of each color, and performs B, G, and R A spectral image of each color is generated. In addition, the DSP 57 performs signal processing such as gamma correction and white balance correction on the imaging signals of the B, G, and R spectral images.

  The frame memory 59 stores image data output by the DSP 57 and processed data processed by the image processing unit 58. The display control circuit 60 reads the image processed image data from the frame memory 59, converts it into a video signal such as a composite signal or a component signal, and outputs it to the monitor 14.

  In the normal observation mode, the image processing unit 58 generates a normal observation image based on the B, G, and R spectral images B, G, and R color-separated by the DSP 57. The image processing unit 58 generates a normal observation image every time the spectral images B, G, and R in the frame memory 59 are updated.

  The image processing unit 58 is provided with a functional image processing unit 61. In the functional information observation mode, the functional image processing unit 61 performs blood based on the three spectral images PB1, PB2, and PB3 corresponding to the imaging signals sequentially acquired in accordance with the irradiation timings of the narrowband light N1, N2, and N3. Information on oxygen saturation StO2 of medium hemoglobin and blood vessel depth D is calculated. Then, an oxygen saturation image in which the calculated oxygen saturation is imaged in a pseudo color is generated. The spectral images PB1, PB2, and PB3 are generated by separating the B pixel signals representing the reflected light amounts of the narrowband light N1, N2, and N3 from the imaging signal output by the imaging device 44.

  As shown in FIGS. 4 and 5, the spectral characteristics of the micro color filter of the B pixel of the image sensor 44 are excited not only by the narrowband light N1, N2, N3 in the blue region but also by the narrowband light N1, N2. Since it overlaps with a part of the fluorescent light FL1, FL2, the luminance value of each spectral image PB1, PB2 includes the reflected light quantity of the fluorescent light FL1, FL3 in addition to the reflected light quantity of the narrowband light N1, N2. It is. The luminance value of the spectral image PB3 represents the amount of reflected light of the narrowband light N3.

  The functional image processing unit 61 includes an alignment processing unit 63, a luminance ratio calculation unit 64, a correlation storage unit 65, a blood vessel depth-oxygen saturation calculation unit 66, a blood vessel depth image generation unit 67, and an oxygen saturation image generation. A portion 68 is provided. As will be described later, the alignment processing unit 63 performs alignment processing of the three spectral images of the spectral images PB1, PB2, and PB3.

  The luminance ratio calculation unit 64 reads the spectral images PB1, PB2, and PB3 that have been subjected to the alignment processing from the frame memory 59, collates the spectral images PB1, PB2, and PB3, and compares the spectral images PB1 and PB3. The first luminance ratio S1 / S3 is obtained, and the second luminance ratio S2 / S3 between the spectral image PB1 and the spectral image PB3 is obtained.

  Here, S1 represents the luminance value of the pixel of the spectral image PB1, S2 represents the luminance value of the pixel of the spectral image PB2, and S3 represents the luminance value of the pixel of the spectral image PB3. The luminance value S3 represents the brightness level of the observation region, and is a reference signal for normalizing the luminance values S1 and S2 in order to compare the luminance values S1 and S2.

  The first luminance ratio S1 / S3 and the second luminance ratio S2 / S3 are calculated for all corresponding pixels between the spectral images PB1, PB2, and PB3. Since the spectral images PB1, PB2, and PB3 are aligned, the luminance ratio between the pixels in the blood vessel region can be obtained. Of course, the blood vessel region may be specified, and the first luminance ratio S1 / S3 and the second luminance ratio S2 / S3 may be obtained for only that portion.

  The correlation storage unit 65 stores the correlation between the first and second luminance ratios S1 / S3 and S2 / S3, the oxygen saturation level in the blood vessel, and the blood vessel depth. This correlation is obtained by analyzing the light reflection characteristics of the living tissue and a large number of spectral images accumulated in the diagnosis so far. Hereinafter, the above correlation and a method for calculating the oxygen saturation and the blood vessel depth based on the above correlation will be described.

  Specifically, the light reflection characteristics of the biological tissue are a light absorption characteristic represented by an absorption spectrum of hemoglobin in blood shown in FIG. 8 and a light scattering characteristic of the biological tissue shown in FIG.

As shown in FIG. 8, hemoglobin has a light absorption characteristic in which the extinction coefficient μa changes depending on the wavelength of light to be irradiated. The extinction coefficient μa represents an absorbance that is the magnitude of light absorption of hemoglobin, and is a coefficient of an expression of I ₀ exp (−μa × x) representing the attenuation state of light irradiated to hemoglobin. Here, I ₀ is the intensity of light irradiated from the light source device to the living tissue such as the surface mucous membrane of the observation site, and x (cm) is the depth to the blood vessel in the living tissue.

  The absorption spectrum of hemoglobin is low in the red region on the long wavelength side and has a peak in the green region and blue region. The peak in the blue region with a wavelength of 450 nm or less is particularly high. For this reason, under white light, the blood appears red because the green and blue regions are absorbed. Thus, the absorption spectrum of hemoglobin has wavelength dependency, and the absorption coefficient μa changes between different wavelengths, so that the luminance value changes as the wavelength changes. For example, even when narrow-band light N1 (445 nm) and narrow-band light N2 (473 nm) are irradiated to blood vessels having the same light intensity and the same thickness (the same blood volume), the absorption coefficient μa is different. The luminance values S1 and S2 corresponding to the spectral images PB1 and PB2 are different.

  Further, in FIG. 10 in which the vertical axis of the graph of FIG. 8 is logarithmically expressed, reduced hemoglobin 70 that is not bonded to oxygen and oxidized hemoglobin 71 that is bonded to oxygen have different light absorption characteristics, and have the same absorption coefficient μa. Except for the isosbestic point (the intersection of the hemoglobins 70 and 71 in FIG. 10). If there is a difference in the extinction coefficient μa, the luminance value changes if the oxygen saturation changes even when the light having the same light intensity and the same wavelength is irradiated. For example, in the narrowband lights N1 and N2, which are light having wavelengths around 445 nm and 473 nm, there is a difference in the extinction coefficient μa in both wavelength ranges, so that when the oxygen saturation changes, the luminance values of the spectral images PB1 and PB2 S1 and S2 change.

  However, in the vicinity of 445 nm, the extinction coefficient μa of reduced hemoglobin 70 is higher than the extinction coefficient μa of oxidized hemoglobin 71, while in the vicinity of 473 nm, oxidized hemoglobin is higher than reduced hemoglobin 70. The magnitude relationship between the two extinction coefficients μa is reversed. For this reason, the luminance value S1 corresponding to 445 nm increases as the oxygen saturation increases, whereas the luminance value S2 corresponding to 473 nm decreases as the oxygen saturation increases.

  On the other hand, since the wavelength 405 nm is an isosbestic point in the absorption spectrum of hemoglobin, the luminance value S3 corresponding to 405 nm is constant if the light intensity is the same even if the oxygen saturation is changed.

  In addition, as shown in FIG. 9, the scattering coefficient μS of the scattering characteristics of living tissue has wavelength dependency. Scattering occurs mainly in mucous membranes other than blood vessels. The scattering coefficient μS is high on the short wavelength side, and rapidly increases particularly in the region of 450 nm or less. And it decreases monotonously toward the long wavelength.

  The reflectance R when the light of wavelength λ is irradiated to the mucous membrane in the digestive tract increases as the scattering coefficient μS increases and the blood vessel depth D increases. This is because if the blood vessel is deep, the distance from the mucosal surface layer to the blood vessel is long, and light incident on the mucosa is reflected by scattering before reaching the blood vessel. The deeper the blood vessel, the smaller the amount of light that reaches the blood vessel, and the smaller the contrast between the blood vessel and the mucous membrane without the surrounding blood vessels. Regarding light absorption by blood hemoglobin (blood), the light absorption coefficient μa is high, and the larger the blood vessel (the larger the blood volume), the more light is absorbed, and thus the reflectance R becomes smaller.

  The luminance value of the spectral image is a value determined by the reflectance R, and is higher as the reflectance R is higher and smaller as the reflectance R is lower. Assuming the above, when the blood vessels have the same thickness (the same blood volume), the two narrow-band light N1 (445 nm) and the narrow-band light N2 (473 nm) have different scattering coefficients μS. The amount of change in reflectance R with respect to the change in thickness D is different. In addition, the narrow band light N1 and the narrow band light N2 are different in the magnitude of the difference between the extinction coefficients μa of the reduced hemoglobin 70 and the oxidized hemoglobin 71, and therefore the amount of change in the reflectance R with respect to the change in the oxygen saturation StO2 is also different. By comparing the brightness values (determined by the reflectance R) of light of two types of wavelengths, which have a difference in the scattering coefficient μS and the difference in the absorption coefficient μa of the hemoglobins 70 and 71 is different. Two types of information, oxygen saturation StO2 and blood vessel depth D, can be acquired. In other words, it is possible to acquire highly accurate information for each of the oxygen saturation StO2 and the blood vessel depth D by eliminating the influence between each other.

  The relationship between the brightness values S1, S2, and S3 of the three light beams of the narrow-band light N1 (445 nm), N2 (473 nm), and N3 (405 nm), and the oxygen saturation StO2 and the blood vessel depth D is specifically described. Has the following relationship:

  First, regarding the relationship between the oxygen saturation StO2 and the luminance value S1 and the oxygen saturation StO2, the reduced hemoglobin 70 is more preferable than the oxidized hemoglobin 71 at the wavelength of 445 nm as shown in the absorption spectrum of hemoglobin in FIG. Since the extinction coefficient μa is high, as shown in the graph of <S1-S> in FIG. 11, the graph rises to the right. The higher the oxygen saturation StO2, the higher the luminance value S1. At the wavelength 473, since the magnitude relationship between the respective extinction coefficients μa of the reduced hemoglobin 70 and the oxidized hemoglobin 71 is reversed, the relationship between the luminance value S2 and the oxygen saturation StO2 is represented by <S2-S> in FIG. As shown in the graph, the graph is a downward sloping graph. The higher the oxygen saturation StO2, the lower the luminance value S2. Since the wavelength 405 nm is an isosbestic point, the luminance value S3 is constant even when the oxygen saturation StO2 changes, as shown in the graph of <S3-S> in FIG.

  From the graphs of <S1-S>, <S2-S>, and <S3-S>, the relationship between each of the first luminance ratio S1 / S3 and the second luminance ratio S2 / S3 and the oxygen saturation StO2 is shown in FIG. <S1 / S3-S>, <S2 / S3-S>. As shown in the graph of <S1 / S3-S>, the relationship between the first luminance ratio S1 / S3 and the oxygen saturation StO2 is such that the higher the oxygen saturation StO2, the higher the first luminance ratio S1 / S3. On the other hand, as shown in the graph <S1 / S3-S>, the relationship between the second luminance ratio S2 / S3 and the oxygen saturation StO2 is such that the higher the oxygen saturation StO2, the higher the second luminance ratio S2 / S3.

  Next, the relationship between the luminance values S1, S2, S3 of the three light beams N1 (445 nm), N2 (473 nm), and N3 (405 nm) and the blood vessel depth D is as shown in FIG. .

  Regarding the relationship between each of the luminance values S1, S2, and S3 and the blood vessel depth D, as shown in the graphs of <S1-D>, <S2-D>, and <S3-D> in FIG. Both S2 and S3 rise to the right, and increase as the blood vessel depth D increases. This is because, as described above, the deeper the blood vessel is, the longer the distance from the mucosal surface layer to the blood vessel is, and the amount of light reflected by scattering before the light that has entered the mucosa reaches the blood vessel increases. This is because of the increase.

  However, since the luminance values S1, S2, and S3 have different scattering coefficients μS, the slopes of the graphs are different. The slope of the graph increases in descending order of the scattering coefficient μS. That is, the gradient of the luminance value S3 (405 nm) is the largest, the next is the luminance value S1 (445 nm), and the luminance value S3 (473 nm) is the smallest. Further, when comparing the luminance values when the blood vessel depth D is the same, the luminance value increases as the scattering coefficient μS increases. As a result, as the scattering coefficient μS is higher, the blood vessel depth D at which the luminance value is saturated is shallower, and the luminance value S3 is saturated at the shallowest position.

  From the graphs of <S1-D>, <S2-D>, and <S3-D>, the relationship between the first luminance ratio S1 / S3 and the second luminance ratio S2 / S3 and the blood vessel depth D is shown in FIG. As shown in the graphs of <S1 / S3-D> and <S2 / S3-D>. As shown in the graph of <S1 / S3-D>, the relationship between the first luminance ratio S1 / S3 and the blood vessel depth D is such that the deeper the blood vessel depth D, the lower the first luminance ratio S1 / S3. Similarly, as shown in the graph of <S1 / S3-D>, the relationship between the second luminance ratio S2 / S3 and the blood vessel depth D is also similar to the first luminance ratio S1 / S3. The second luminance ratio S2 / S3 is lower.

  As shown in FIG. 12, the relationship between each of the first luminance ratio S1 / S3 and the second luminance ratio S2 / S3 and the oxygen saturation StO2 (<S1 / S3-S>, <S1 / S3- in FIG. 12). (S> graph), the relationship between the first luminance ratio S1 / S3 and the second luminance ratio S2 / S3, and the blood vessel depth D (<S1 / S3-D>, <S1 / S3- in FIG. 12). (D> graph) is integrated, the correspondence relationship between the luminance coordinate system 66 representing the first and second luminance ratios S1 / S3, S2 / S3 and the blood vessel information coordinate system 67 representing the oxygen saturation and the blood vessel depth is obtained. can get.

  The luminance coordinate system 66 is an XY coordinate system having two axes XY, and the first luminance ratio S1 / S3 is assigned to the X axis, and the second luminance ratio S2 / S3 is assigned to the Y axis.

  The blood vessel information coordinate system 67 is a UV coordinate system having two UV axes provided on the luminance coordinate system 66. The U axis is assigned to the blood vessel depth D and the V axis is assigned to the oxygen saturation StO2. The U-axis has a positive inclination because the blood vessel depth D has a positive correlation with the luminance coordinate system 66. Regarding the U-axis, the blood vessel is shallower as it goes diagonally upward to the right, and the blood vessel is deeper as it goes diagonally downward to the left. On the other hand, since the oxygen saturation StO2 has a negative correlation with the luminance coordinate system 66, the V-axis has a negative slope. With respect to the V-axis, the oxygen saturation StO2 is lower as it goes diagonally upward to the left, and the oxygen saturation StO2 is higher as it goes diagonally downward to the right. In the blood vessel information coordinate system 67, the U axis and the V axis intersect at an intersection P.

  The correlation storage unit 61 (see FIG. 7) stores correlation data representing the correspondence between the luminance coordinate system 66 and the blood vessel information coordinate system 67 as shown in FIG.

  The luminance ratio calculation unit 60 reads out pixels corresponding to positions in the screen from the spectral images PB1, PB2, and PB3, and calculates the first luminance ratio S1 / S3 and the second luminance ratio S2 / S3 for each pixel. To do. The luminance ratio calculation unit 60 outputs the data of a pair of luminance ratios of the calculated first luminance ratio S1 / S3 and second luminance ratio S2 / S3 to the blood vessel depth-oxygen saturation calculation unit 62 for each pixel. Output. When the first and second luminance ratios S1 / S3 and S2 / S3 are input, the blood vessel depth-oxygen saturation calculation unit 62 is input with reference to the correlation stored in the correlation storage unit 61. The oxygen saturation StO2 and the blood vessel depth D corresponding to the first and second luminance ratios S1 / S3, S2 / S3 are specified.

The first luminance ratio S1 / S3 and the second luminance ratio S2 / S3 input to the blood vessel depth-oxygen saturation calculating unit 62 are respectively represented by a first luminance ratio of S1 ^* / S3 ^* and a second luminance ratio of S2. Assuming ^* / S3 ^* , the blood vessel depth-oxygen saturation calculating unit 62 specifies the oxygen saturation StO2 and the blood vessel depth D as follows.

As shown in FIG. 14A, the blood vessel depth-oxygen saturation calculation unit 62 uses coordinates corresponding to the first and second luminance ratios S1 ^* / S3 ^* , S2 ^* / S3 ^* in the luminance coordinate system 66. Specify (X ^* , Y ^* ). When the coordinates (X ^* , Y ^* ) are specified, as shown in FIG. 14 (B), the specified coordinates (X ^* , Y ^* ) in the blood vessel information coordinate system 67 are set as Vs which are the coordinate axes of the oxygen saturation. The coordinates (U ^* , V ^* ) are specified by projecting on the axis and the U axis which is the coordinate axis of the blood vessel depth. Thereby, blood vessel depth information U ^* and oxygen saturation information V ^* are obtained for one pixel. The blood vessel depth-oxygen saturation calculation unit 62 repeats such processing for all pixels for one screen to obtain blood vessel depth information U ^* and oxygen saturation information V ^* for all pixels.

  As described above, the luminance values S1 and S2 of the light (N1, N2) of two types of wavelengths that have a difference in the scattering coefficient μS and differ in the difference in the absorption coefficient μa between the hemoglobins 70 and 71, and By using the luminance value S3 of the reference light (N3) for comparing the luminance values S1 and S2, highly accurate information on the oxygen saturation StO2 and the blood vessel depth D is obtained without mutual influence. can do. This is nothing other than comparing the reflectances R of the two types of light. Therefore, the greater the difference in the amount of change with respect to changes in the oxygen saturation StO2 and the blood vessel depth D, the greater the difference between the compared luminance values S1 and S2. It becomes large, S / N ratio becomes high, and highly accurate information can be obtained. For the two types of light for obtaining the brightness values S1 and S2 to be compared, like the narrow band lights N1 and N2 of this example, the difference in the scattering coefficient μS is large, and the extinction coefficients of the hemoglobins 70 and 71 are the same. It is preferable to select a wavelength having a large μa difference.

  In the above example, an example has been described in which two lights having a wavelength range in which the magnitude relationship between the extinction coefficients μa of the reduced hemoglobin 70 and the oxidized hemoglobin 71 is reversed are used, such as the narrow-band lights N1 and N2. As long as there is a difference in the coefficient μa, the magnitude relationship may not be reversed. For example, when the two wavelengths for obtaining the brightness values S1 and S2 to be compared are both light having a wavelength higher than the oxygenated hemoglobin 71 by the extinction coefficient μa of the reduced hemoglobin 70, the oxygen saturation StO2 is used. The relationship between the brightness value S and the brightness value S rises to the right in the same manner as the narrow-band light N1 of 445 nm for both wavelengths. In such a case, as shown in FIG. 15, the inclination of the blood vessel coordinate system 67 changes and rises to the right.

  In the example shown in FIG. 15, since the difference in signal value (luminance ratio) is smaller than when the magnitude relationship of the light absorption coefficient μa is reversed, the S / N ratio is lowered accordingly. For this reason, from the viewpoint of increasing the S / N ratio, it is preferable to select two wavelengths whose magnitude relationship of the absorption coefficient μa is reversed, such as the narrowband lights N1 and N2 in the above example.

  In addition, as the reference light for obtaining the reference signal for standardizing the two luminance values S1 and S2, narrowband light N3 that is light having the wavelength of the isosbestic point is used. Therefore, it is not necessary to use the wavelength of the isosbestic point. Further, white light may be used, or G color light or R color light that is color-separated from white light may be used.

In FIG. 7, the blood vessel depth image generating unit 67 has an internal memory, and a color table 67a to which color information is assigned according to the degree of the blood vessel depth is stored in the internal memory. The color table 67a can be clearly distinguished according to the degree of the blood vessel depth, for example, blue when the blood vessel depth is a surface layer, green when the blood vessel is deep, and red when the blood vessel is deep. A color is assigned. The blood vessel depth here is a depth within the range of the depth of the narrow-band lights N1 and N2. Since the narrow-band lights N1 and N2 are in the blue region, the depth of penetration from the surface of the observation site is shorter than the light in the red region having a longer wavelength. The degree of the inner depth can be determined. The blood vessel depth image generation unit 67 specifies color information corresponding to the blood vessel depth information U ^* calculated by the blood vessel depth-oxygen saturation calculation unit 67 from the color table 67a.

  When the color information is specified for all the pixels in the blood vessel region, the blood vessel depth image generation unit 67 combines the color information with the normal observation image to reflect it in the normal observation image. As the normal observation image, for example, image data acquired by white light composed of narrowband light N1 and fluorescence FL1 excited thereby is used. Thereby, the blood vessel depth image in which the information of the blood vessel depth is reflected on the normal observation image is generated. The generated blood vessel depth image is stored in the frame memory 59 again. Note that the color information may be reflected not on the normal observation image but on any of the spectral images PB1, PB2, and PB3 obtained in the function information observation mode, or a synthesized image obtained by synthesizing them.

The oxygen saturation image generation unit 68 has an internal memory, and a color table 68a to which color information is assigned according to the degree of oxygen saturation is stored in the internal memory. The color table 68a clearly distinguishes depending on the degree of oxygen saturation, for example, cyan when low oxygen saturation, magenta when medium oxygen saturation, and yellow when high oxygen saturation. Colors that can be assigned. Similar to the blood vessel depth image generating unit, the oxygen saturation image generating unit 64 specifies color information corresponding to the oxygen saturation information V ^* calculated by the blood vessel depth-oxygen saturation calculating unit from the color table 68a. . Then, by reflecting this color information on the normal observation image, an oxygen saturation image is generated. The generated oxygen saturation image is stored in the frame memory 59 similarly to the blood vessel depth image. Similar to the blood vessel depth image, the oxygen saturation image may be one in which color information is reflected based on the spectral images PB1, PB2, and PB3 or a composite image thereof instead of the normal observation image.

  The display control circuit 60 reads a blood vessel depth image and an oxygen saturation image from the frame memory 59 in the same manner as the normal observation image, converts it into a video signal, and outputs it to the monitor 14. Various patterns are conceivable as image display forms in the function information observation mode.

  For example, as shown in FIG. 16, a display image 72 of a normal observation image is displayed on one side of the monitor 14, and a display image 73 of a blood vessel depth image or a display image 74 of an oxygen saturation image is displayed on the other side. Either of them may be displayed. Switching between the display image 73 and the display image 74 is performed by operating the console 15, for example. Of course, the display images 73 and 74 may be displayed simultaneously. In the display image 73, the surface blood vessel 75 is displayed in blue, the middle blood vessel 76 is displayed in green, and the deep blood vessel 77 is displayed in red. In the oxygen saturation image 74, the blood vessel 80 indicating low oxygen saturation is displayed in cyan, the blood vessel 81 indicating medium oxygen saturation is displayed in magenta, and the blood vessel 82 indicating high oxygen saturation is displayed in yellow.

  The blood vessel depth information and the oxygen saturation information may be displayed as character information instead of or in addition to the image. In addition, oxygen saturation is imaged, but the oxygen saturation image may be expressed as “blood volume (sum of oxidized hemoglobin and reduced hemoglobin) × oxygen saturation in place of or in addition to the form shown in the above example. (%) "Which includes an image of an oxygenated hemoglobin index obtained from"% ".

  In this example, in the calculation of the oxygen saturation, narrow band light in the blue region is used as the narrow band light N1 and N2. The reason for using light in the blue region is that it is often more important to understand the properties of the superficial blood vessels than the mid-deep layer in the diagnosis of lesions such as tumor benign / bad discrimination. This is because an observation method that can be grasped is desired. In order to meet such a demand, the present invention employs light in a blue region that has a low depth of penetration and can acquire information on the superficial blood vessels well.

  The reason why the narrow band light is used is as follows. As is clear from the absorption spectra of hemoglobin shown in FIGS. 8 and 10, the change in absorbance is sharper in the blue region than in the green and red regions, and the absorbance changes greatly when the wavelength is slightly shifted. To do. Further, the interval between the isosbestic points where the magnitude relationship between the absorbances of the hemoglobins 70 and 71 is reversed is narrow. If the wavelength range is wide, signals in two areas where the magnitude relationship is reversed are mixed and the luminance values are averaged, so that highly accurate information cannot be obtained. Therefore, in order to obtain the blood vessel information of the superficial blood vessel using the light in the blue region, a wavelength region having a width close to the interval between the two isosbestic points, preferably a wavelength region within the interval between the two isosbestic points is selected. It is necessary to use narrow narrow band light.

  Further, since the superficial blood vessel is thinner than the middle-deep blood vessel, the amount of light to be irradiated tends to be insufficient, and a light source having a large light amount is required when observing the superficial blood vessel.

  As described above, in order to improve the measurement accuracy of blood vessel information related to the surface blood vessel, a light source that emits a light with a narrow band light in a blue region and a high light amount is suitable. Laser light sources LD1 and LD2 capable of emitting narrowband light N1 and N2 are employed.

  Next, an alignment process executed by the alignment processing unit 63 before calculating the oxygen saturation and the blood vessel depth in the function information observation mode will be described. As shown in FIG. 6 (B), since the irradiation timing of the narrowband light N1, N2, and N3 and the imaging timing of the reflected light are different, the acquired spectral images PB1, PB2, and PB3 are shakes, movement of the visual field, Positional misalignment occurs due to patient movement caused by heartbeat and respiration. In calculating the oxygen saturation, in order to compare the luminance values of the corresponding pixels of the spectral images PB1 and PB2, the alignment processing unit 63 performs alignment and positions between the spectral images PB1, PB2, and PB3. Eliminate the gap.

  As shown in FIG. 17, the alignment processing unit 63 first reads the spectral images PB1, PB2, and PB3 from the frame memory 59, and performs contrast adjustment processing. The contrast adjustment process is a preprocess for generating an alignment image. Then, after adjusting the contrast, binarization processing is performed on the processed spectral images PB1, PB2, and PB3 that have been preprocessed to generate binary images EPB1, EPB2, and EPB3 that are alignment images. . Then, based on the binary images EPB1, EPB2, and EPB3, a positional deviation amount detection process is performed. Finally, alignment processing of the spectral images PB1, PB2, and PB3, which are the original images, is performed based on the positional deviation amount detected in the alignment image.

  As shown in FIG. 18, the alignment processing unit 63 obtains density histograms for the spectral images PB1, PB2, and PB3 when performing the contrast adjustment processing. As is well known, the density histogram is a graph with pixel values (luminance values) on the horizontal axis and frequency on the vertical axis. In each of the spectral images PB1, PB2, and PB3, a superficial blood vessel 86 and a mid-deep blood vessel 87 are depicted.

  The narrowband lights N1, N2, and N3 have a blue wavelength region of 445 nm, 473 nm, and 405 nm, respectively. Since the absorption coefficient of hemoglobin is high in these wavelength regions, the amount of reflected light of the narrowband light N1, N2, and N3 is small and the luminance value is low in the region of the superficial blood vessel 86 and the mid-deep blood vessel 87. On the other hand, the region where the blood vessel does not exist is not absorbed by hemoglobin, and therefore the amount of reflected light is large and the luminance value is high. Therefore, in the spectroscopic images PB1, PB2, and PB3, the superficial blood vessel 86 and the middle-deep blood vessel 87 are higher in concentration than the peripheral blood vessels.

  Further, since the narrow-band lights N1, N2, and N3 have a wavelength region in the blue region, the scattering is strong and the depth of penetration is low. Therefore, in the spectral images PB1, PB2, and PB3, the density of the superficial blood vessel 86 is rendered higher than that of the intermediate deep blood vessel 87. Therefore, the density histograms of the spectral images PB1, PB2, and PB3 show the peak a of the superficial blood vessel 86, the peak b of the middle-deep blood vessel 87, and the peak c of the mucous membrane where no blood vessel exists, in descending order of the density (in order of decreasing pixel value) It has three peaks.

  Further, as is apparent from the absorption spectra of hemoglobin shown in FIGS. 8 and 10, the narrowband light N2 (473 nm) has a lower absorption coefficient μa of hemoglobin than the narrowband lights N1 and N2, so that Less absorbed by hemoglobin. Therefore, the spectral image PB2 has a low contrast between the blood vessel and the mucous membrane where no blood vessel exists compared to the spectral images PB1 and PB3. This appears as a difference in the dynamic range DR which is the width of the maximum value (max) and the minimum value (min) of the pixel value in the density histogram. The dynamic range DR2 of the spectral image PB2 is narrower than the dynamic ranges DR1 and DR3 of the spectral images PB1 and PB3.

  When the spectral images PB1, PB2, and PB3 are aligned, they are binarized into white and black. If there is a difference in contrast, for example, the binary images EPB1 and PB3 of the spectral images PB1 and PB3 with high contrast are displayed. In EPB3, the blood vessel region assigned to black may be assigned to white in the binary image EPB2 of the spectral image PB2 having a low contrast. In this case, since the region which should show the same value will show a different value, the precision of alignment will fall.

  Therefore, as shown in FIG. 19, the alignment processing unit 63 performs contrast adjustment processing between the spectral images PB1, PB2, and PB3, and adjusts the contrast of the spectral images PB1, PB2, and PB3. FIG. 19 shows an example of matching the contrast between the spectral image PB1 and the spectral image PB2.

  The alignment processing unit 63 has an internal memory, and a gradation correction table 63a is stored in the internal memory. The gradation correction table 63a has a gradation correction curve indicating the input value on the horizontal axis and the output value on the vertical axis, and indicating the correspondence between the input value and the output value. In order to increase the contrast, the low density portion of the original image before processing should have a lower density, and the higher density portion should be higher. Therefore, the substantially S-shaped gradation shown in FIG. If gradation correction is performed based on the correction curve, the contrast can be increased (DR increases). On the other hand, if an inverse S-shaped gradation correction curve is used, the contrast is lowered (DR becomes narrower).

  In the contrast adjustment process, the alignment processing unit 63 uses the spectral image PB1 having a higher contrast than the spectral image PB2 having the lowest contrast as a reference image, and adjusts the contrast level so that the contrast of the spectral image PB2 matches the contrast of the reference image. To decide. The alignment processing unit 63 determines the respective dynamic ranges DR1 and DR2 based on the density histograms of the spectral image PB1 and the spectral image PB2, so that the dynamic range DR2 of the spectral image PB2 substantially matches the dynamic range DR1. A gradation correction curve of the gradation correction table 63a is generated. By applying the generated gradation correction curve and performing contrast adjustment processing on the spectral image PB2, the dynamic range DR2 of the spectral image PB2 is matched with the dynamic range DR1 of the spectral image PB1. As a result, in the spectroscopic image PB2, the density difference between the blood vessel region and the region where no blood vessel exists spreads, and clear discrimination becomes possible.

  In FIG. 19, the contrast adjustment process between the spectral images PB1 and PB2 has been described, but the alignment processing unit 63 also performs the contrast adjustment process on the spectral images PB1 and PB3. In the contrast adjustment process between the spectral images PB1 and PB3, the contrast of the spectral image PB3 is adjusted using the spectral image PB1 as a reference image. Thereby, the contrast of the two spectral images of the spectral images PB1, PB2, and PB3 can be substantially matched. The alignment processing unit 63 determines whether or not the difference in contrast between the two spectral images is within a predetermined allowable range based on the density histogram. If the difference is within the allowable range, the contrast adjustment process is performed. Instead, the contrast adjustment process may be executed only when it is outside the allowable range. For example, when the contrast difference between the spectral images PB1 and PB3 is within an allowable range, the contrast adjustment process may not be performed on the spectral images PB1 and PB3.

  As shown in FIG. 17, the alignment processing unit 63 performs binarization processing on the spectral images PB1, PB2, and PB3 that have undergone contrast adjustment processing, and sets the pixel values to white and black with reference to the threshold value. Binary images EPB1, EPB2, and EPB3 are generated. The threshold value is set such that the blood vessel region is black and the region where no blood vessel exists is white. For example, a valley between the peak b of the middle depth blood vessel and the peak c of the mucous membrane is set as the threshold value. In the narrow-band light in the blue region, the surface blood vessel 86 is more clearly depicted as compared to the mid-deep blood vessel 87. Therefore, the valley between the peak a of the surface blood vessel 86 and the peak b of the mid-deep blood vessel 87 is shown. It may be set as a threshold value.

  After the binarization process, the alignment processing unit 63 performs a positional deviation amount detection process based on the binary images EPB1, EPB2, and EPB3. In the positional deviation amount detection process, for example, a motion vector between images is obtained by a block matching technique, and the obtained motion vector is detected as a positional deviation amount.

  Specifically, as shown in FIG. 20, a representative point 91 is set in the screen of the reference image using, for example, the binary image EPB1 as one of the two images of the binary image EPB1 and the binary image EPB2. For example, a small area of about 9 × 9 pixels is set as a block 92 around each representative point. For example, nine representative points 91 are set in the screen, and nine blocks 92 are set according to the number of representative points 91. On the other hand, in the binary image EPB2, a search area 93 larger than the block 92 is set based on the same coordinate position as each representative point 91. Then, in the search area 93 of the binary image EPB2, an area having a high correlation with the integrated value of the pixel values of the block 92 of the binary image EPB1 is searched. As a search method, for example, a square error SSD (Sum of Squared Difference) is used.

  The square error SSD is obtained by squaring the integrated value in the block 92 and the integrated value of the block having the same size as the block 92 in the search area 93, and taking the difference between them. It is determined that the smaller the square error SSD is, the higher the correlation between the two is. In FIG. 20, when the region having the highest correlation with the block 92 is determined to be the block 94 in the search area 93, it is based on the coordinates of the representative point 91 of the block 92 and the coordinates of the representative point 96 of the block 94. A motion vector 97 is obtained. The alignment processing unit 63 performs such processing for each block, and detects a value obtained by averaging the obtained motion vectors 97 as a positional deviation amount of the entire image. Although the square error SSD is used in this example, the absolute value SAD (Sum of Absolute Difference) of the error of each block may be used. Further, in this example, only the positional deviation amount relating to the linear movement direction is obtained, but the positional deviation amount relating to the rotation direction may be obtained.

  The alignment processing unit 63 performs alignment by applying the detected positional deviation amount to the spectral images PB1 and PB2. Specifically, the origin O of each of the spectral images PB1 and PB2 is adjusted based on the amount of positional deviation, and the coordinates of each pixel are converted. The alignment processing unit 63 performs similar processing on the binary images EB1 and EB3 to align the spectral images PB1 and PB3.

  The functional image processing unit 61 calculates and calculates the luminance ratio (S1 / S3, S2 / S3) according to the procedure described in FIGS. 13 and 14 based on the spectral images PB1, PB2, and PB3 that have been aligned. Imaging is performed by calculating the oxygen saturation and the blood vessel depth based on the brightness ratio.

  In the present invention, since contrast adjustment processing is performed as preprocessing for alignment, contrast differences between the spectral images PB1, PB2, and PB3 are eliminated. When the binary images EB1, EB2, and EB3, which are alignment images, are created, the same blood vessel region is not emphasized on one side and is not enhanced on the other side, and the same is true in the binary images EB1, EB2, and EB3. Thus, the blood vessel region is emphasized. Therefore, as compared with the case where the contrast adjustment process is not performed, the detection accuracy of the positional deviation amount is improved, and the positions of the respective pixels of the spectral images PB1, PB2, and PB3 can be accurately associated. This makes it possible to accurately compare the luminance values of the blood vessel regions, so that the oxygen saturation and blood vessel depth calculation accuracy is improved.

  When calculating biological function information such as oxygen saturation and blood vessel depth, the positioning accuracy is required to be very high compared to the case where a composite image is generated by simply combining two images. Therefore, the contrast adjustment process that contributes to high accuracy of alignment is particularly useful when calculating biological function information.

  Further, in this example, when the oxygen saturation is calculated, narrow band lights N1, N2, and N3 in the blue region are used. As described above, in the blue region, changes in the absorption coefficient μa and scattering coefficient μS of hemoglobin with respect to the change in wavelength are steep. For this reason, when light of different wavelengths is used in the blue region, such as narrowband light N1, N2, and N3, the contrast difference between the spectral images PB1, PB2, and PB3 tends to be large. Therefore, the present invention is particularly effective when a plurality of narrow-band lights in the blue region are used to observe the surface blood vessels.

  Furthermore, in this example, the laser light sources LD1, LD2, and LD3 are used as the light sources of the narrow-band lights N1, N2, and N3. Compared with the laser light sources LD1 and LD3, the output is relatively low. This is also one of the causes that the contrast of the spectral image PB2 becomes lower than the other spectral images PB1 and PB3. Therefore, the present invention is particularly effective when using a plurality of light sources having output differences such as the laser light source LD1 (405 nm), LD2 (473 nm), and LD3 (405 nm) of this example.

  In addition, according to the present invention, it is not necessary to grasp the thickness of the blood vessel as in Patent Document 1 that performs frequency filtering processing according to the thickness of the blood vessel. Can also be supported. Since it is not necessary to change the contents of processing when the display magnification changes, the configuration is not complicated.

[Second Embodiment]
In the above embodiment, the contrast adjustment process for adjusting the contrast by expanding the dynamic range has been described as the preprocess according to the contrast difference between the images. However, instead of the contrast adjustment process, the binarization process is performed as the preprocess. You may perform the process which determines a threshold value.

  As shown in FIG. 21, the alignment processing unit 63 determines a threshold value for binarization processing according to the contrast difference between the spectral images PB1, PB2, and PB3. For example, as illustrated in FIG. 22, the threshold value Th <b> 2 for binarization processing of the spectral image PB <b> 2 with low contrast is determined based on the density histogram of the spectral image PB <b> 1 with high contrast.

  When the contrast of the blood vessel is low as in the spectral image PB2, the density histogram is compared with the density histogram of the spectral image PB1 having a high contrast, and the peaks a and b of the blood vessel regions of the surface blood vessel 86 and the mid-deep blood vessel 87 are compared. And the boundary (valley) between the peak c of the mucous membrane where no blood vessel exists may be unclear. If the valley is unclear, the set threshold Th2 may be set to an inappropriate value for distinguishing between a blood vessel region and a region where no blood vessel exists. When the threshold value Th2 is set to an inappropriate value, the pixels assigned to the blood vessel region (black) in the binary image EPB1 are assigned to the region (white) where no blood vessel exists in the binary image EPB2. End up.

  Therefore, the alignment processing unit 63 sets, for example, the difference Δd1 between the pixel values of the peak a of the superficial blood vessel 86 and the peak c of the mucous membrane and the spectral image PB1 based on the density histogram of the spectral image PB1 with clear peaks. Based on the respective threshold values Th1, the threshold value Th2 of the spectral image PB2 is determined. For example, the ratio between the pixel value difference Δd1 of the spectral image PB1 and the threshold Th1 is obtained, and the threshold Th2 with respect to Δd2 of the spectral image PB2 is obtained so as to be the same as the ratio. Expressed by the equation, Th2 = Th1 × Δd2 / Δd1.

  The alignment processing unit 63 performs binarization processing on the spectral image PB2 based on the obtained threshold value Th2, and generates a binary image EPB1. According to this, even when the contrast is low and the peak in the density histogram is unclear as in the spectral image PB2, the appropriate threshold value Th2 can be obtained. Since the threshold value Th2 is determined based on the threshold value Th1 of the spectral image PB1, it is possible to match the reference for distributing the blood vessel region and the other in the spectral images PB1 and PB2. For this reason, the same effect as the case where the contrast adjustment process is performed can be obtained.

  As a method of determining the threshold Th2, for example, the following method can be considered as a method other than the above. First, after appropriately determining the threshold value Th2, a binary image EPB2 is generated, and pattern matching is performed with the binary image EPB1. The pattern matching may be performed by extracting a part of the blood vessel shape or the like in the image, or may be performed on the entire image.

  In this case, the alignment processing unit 63 evaluates the degree of coincidence between the two based on the pattern matching result. The smaller the difference in pixel value between the images EPB1 and EPB2, the higher the matching degree. The alignment processing unit 63 determines whether or not the degree of coincidence is within a predetermined allowable range. If the degree of coincidence is within the allowable range, the binarized threshold value Th2 is determined as a final value. If the value is outside the allowable range, the threshold Th2 is reset to perform binarization, and pattern matching is performed to evaluate the degree of coincidence. Such processing is repeated until the degree of coincidence falls within an allowable range.

  Note that the method of determining the threshold Th2 by evaluating the degree of coincidence of pattern matching can be processed in a short time in the case of a relatively simple pattern such as a character or a mark. In the case of a complicated shape, there is a concern that the processing takes a very long time. Therefore, from the viewpoint of shortening the processing time, a method of determining the threshold value Th2 of the spectral image PB2 based on the density histogram of the spectral image PB1 as in the above example is preferable. In this example, the alignment of the spectral image PB1 and the spectral image PB2 has been described as an example, but the description is omitted because it is the same between the spectral image PB1 and the spectral image PB3. Note that the second embodiment and the first embodiment may be combined.

  In the said 1st and 2nd embodiment, although the example which produces | generates a binary image as an image for position alignment demonstrated, it may be other than a binary image. In the present invention, the binarization process is a process for enhancing the accuracy when performing alignment using block matching or the like by emphasizing feature quantities such as a contour or shape in an image such as a blood vessel shape. As long as the feature amount such as the shape or the shape is emphasized, it is sufficient. Therefore, for example, a ternary image may be generated by a ternarization process in which pixel values are distributed into ternary values of gray in addition to white and black, and this may be used as an alignment image. If the outline and shape are clear compared to the original image, it may be three or more, and if gradation reduction processing is performed to reduce the number of gradations of the original image (for example, 256 gradations for 8 bits). Good. When distributing the threshold value to three or more values, there are a plurality of threshold values. For all of the threshold values, the threshold value may be determined by the above procedure, or one of them may be determined. The above procedure may be applied.

  In the first embodiment, edge enhancement processing may be executed instead of gradation reduction processing. As is well known, the edge emphasis process is a process for emphasizing a portion having a large density change such as a contour portion in an image. For example, there is a filter processing using a Laplacian filter. In this case, the alignment processing unit 63 detects the amount of displacement using an image that has been subjected to edge enhancement processing instead of the binary image as the alignment image.

  In the said embodiment, although demonstrated in the example which used the light of the blue area | region as illumination light, as described in patent document 1 (Unexamined-Japanese-Patent No. 2011-194151), suitable for observation of the middle-deep-layer blood vessel, The present invention can also be applied when light in the green region having a wavelength of the order of 500 nm is used.

  In the above embodiment, the laser light source including the laser diode is exemplified as the semiconductor light source, but an LED light source using an LED instead of the laser diode may be used. In addition, instead of the semiconductor light source, the illumination unit includes a white light source such as a xenon lamp or a halogen lamp, and a rotary filter having a spectral filter that splits white light from the white light source into narrowband light N1, N2, and N3. It may be a combination. In this case, a monochrome image sensor is used as the image sensor. The narrowband lights N1, N2, and N3 are sequentially irradiated in synchronization with the rotation of the rotary filter, and the reflected light corresponding to the narrowband lights N1, N2, and N3 is sequentially captured by the monochrome image sensor. .

  In the above embodiment, the example in which the light source device and the processor device are configured separately has been described, but the two devices may be configured integrally. The present invention also relates to a system comprising an ultrasonic endoscope in which an image sensor and an ultrasonic transducer are built in the tip and a processor device for performing image processing, a system comprising a capsule endoscope and a processor device for performing image processing, etc. The present invention can also be applied to other types of endoscope systems.

DESCRIPTION OF SYMBOLS 10 Electronic endoscope system 14 Monitor 31 Semiconductor light source unit 32 Light source control part 44 Image sensor 57 DSP
58 Image processing unit 61 Functional image processing unit 63 Positioning processing unit 63a Gradation correction tables LD1 to LD3 Laser light sources PB1 to PB3 Spectral images EPB1 to EPB1 Binary image

Claims (12)

An electronic endoscope for imaging an observation site including a blood vessel in a subject;
Illumination means for sequentially irradiating the observation site with at least first and second illumination lights having different wavelength ranges;
First and second spectrums corresponding to the first and second illumination lights, respectively, from a plurality of imaging signals sequentially output by the electronic endoscope according to the irradiation timings of the first and second illumination lights. Image acquisition means for acquiring images;
Preprocessing means for performing preprocessing for aligning the first and second spectral images according to a difference in contrast between the blood vessels in the first and second spectral images;
After the preprocessing, enhancement processing is performed to enhance the shape of the blood vessel in each of the first and second spectral images, and an alignment image corresponding to each of the first and second spectral images is generated. Emphasis processing means to
Based on the first and second alignment images, the first and second spectral images are aligned so that the positions of the blood vessels in the first and second spectral images overlap each other. Alignment means to perform;
Oxygen saturation calculation means for calculating oxygen saturation of blood hemoglobin in the blood vessel based on the first and second spectral images on which the alignment has been performed. Endoscopic system.   The electronic endoscope system according to claim 1, wherein the preprocessing unit checks the contrast based on density histograms of the first and second spectral images and executes the preprocessing.   3. The electronic endoscope system according to claim 2, wherein the preprocessing means performs the preprocessing on a lower one of the first and second spectral images with a higher contrast as a reference. .   The electronic endoscope system according to any one of claims 1 to 3, wherein the preprocessing is a contrast adjustment process for adjusting a contrast between the first and second spectral images. The enhancement processing compares a threshold value set in advance with the pixel values of the first and second images, and distributes the pixel values of the first and second spectral images based on the threshold value. A gradation reduction process for generating the first and second alignment images by reducing the number of gradations of the first and second spectral images,
The electronic endoscope system according to any one of claims 1 to 4, wherein the preprocessing is threshold determination processing that determines the threshold according to the contrast difference.   6. The electronic endoscope system according to claim 5, wherein the gradation reduction process is a binarization process, and the first and second alignment images are binary images.   7. The inside of an electron according to claim 1, wherein at least one of the first illumination light and the second illumination light is blue narrow band light limited to a specific wavelength region in a blue region. Endoscopic system. The illumination means sequentially irradiates three illumination lights obtained by adding a third illumination light having a different wavelength range to the first and second illumination lights,
The image acquisition means acquires first to third spectral images corresponding to the first to third illumination lights,
The electronic endoscope system according to claim 7, wherein the oxygen saturation calculation unit calculates the oxygen saturation based on the first to third spectral images.   The first illumination light is first blue narrowband light whose wavelength range is limited to 440 ± 10 nm, and the second illumination light is second blue narrowband light whose wavelength range is limited to 470 ± 10 nm. The electronic endoscope system according to claim 8, wherein the third illumination light is third blue narrowband light whose wavelength range is limited to 410 ± 10 nm.   10. The electronic endoscope system according to claim 9, wherein the illumination unit includes first to third semiconductor light sources that emit first to third blue narrow-band light. In an image processing apparatus that processes an image acquired using an electronic endoscope that images an observation site including a blood vessel in a subject,
Sequentially illuminating the observation site with at least first and second illumination lights having different wavelength ranges;
First and second spectral images corresponding to the first and second illumination lights, respectively, from a plurality of imaging signals output by the electronic endoscope according to the irradiation timings of the first and second illumination lights. Image acquisition means for acquiring
Preprocessing means for performing preprocessing for aligning the first and second spectral images according to a difference in contrast between the blood vessels in the first and second spectral images;
After the preprocessing, enhancement processing is performed to enhance the shape of the blood vessel in each of the first and second spectral images, and an alignment image corresponding to each of the first and second spectral images is generated. Emphasis processing means to
Based on the first and second alignment images, the first and second spectral images are aligned so that the positions of the blood vessels in the first and second spectral images overlap each other. Alignment means to perform;
Image processing comprising: oxygen saturation calculating means for calculating oxygen saturation of blood hemoglobin in the blood vessel based on the first and second spectral images subjected to the alignment apparatus. Method of operating an image processing apparatus for processing an image wavelength band is obtained by different at least first and second imaging electronic endoscope observation site where illumination light including vascular sequentially irradiated within the subject of In
First and second image acquisition means corresponding to the first and second illumination lights from a plurality of imaging signals output by the electronic endoscope according to the irradiation timing of the first and second illumination lights. and Luz step to obtain the second spectral image,
Preprocessing means, in accordance with the difference in the contrast of the blood vessels in the first and second spectral image, and row mortar step pretreatment for aligning said first and second spectral image,
Enhancement processing means, after the pretreatment, is subjected to emphasizing emphasis processing the shape of the blood vessel in each of the first and second spectral image, corresponding to each of said first and second spectral image position and Luz steps to generate an image for a laminated,
Based on the first and second alignment images, the first and second spectral images are aligned so that the positions of the blood vessels in the first and second spectral images overlap. and Luz steps to run the alignment of each other,
Oxygen saturation calculation means, said alignment is based on the are executed first and second spectral image, and a Luz step to calculate the oxygen saturation in blood hemoglobin in the blood vessel A method for operating an image processing apparatus .

Images