JP5829568B2 - Endoscope system, image processing apparatus, method of operating image processing apparatus, and image processing program - Google Patents

Description

The present invention relates to an endoscope system, an image processing apparatus, an operation method of an image processing apparatus , and an image processing program that acquire information about blood such as oxygen saturation of blood hemoglobin using an endoscope.

  In the medical field, endoscopic diagnosis using an endoscope is widespread. In recent endoscopic diagnosis, special light observation using special light limited to a specific wavelength is performed in addition to normal observation of observing the overall properties of the surface of living tissue under white light. It has come to be.

  There are various types of special light observations. For example, in the endoscope system of Patent Document 1 by the present applicant, narrowband light in a wavelength region having a difference in the magnitude of the extinction coefficient between oxyhemoglobin and reduced hemoglobin is used. The oxygen saturation of blood hemoglobin is calculated and imaged based on a spectral image obtained by imaging the reflected light.

  Here, since the calculated oxygen saturation value is greatly influenced by the depth of the blood vessel, in Patent Document 1, noise generated by the depth of the blood vessel is obtained by using a luminance ratio between predetermined spectral images. To improve the calculation accuracy of oxygen saturation.

  In the endoscope system of Patent Document 1, a sequential method of sequentially irradiating a biological tissue with a plurality of narrow-band lights and taking an image is adopted as a method for acquiring a plurality of spectral images. For this reason, a positional shift between a plurality of spectral images caused by a difference in imaging timing becomes a problem. With respect to this problem of positional deviation, in Patent Document 1, the position of a blood vessel is aligned between spectral images by pattern matching. In Patent Document 1, frequency filtering processing according to the blood vessel depth is performed in order to increase the accuracy of pattern matching.

JP 2011-194151 A

  As described in Patent Document 1, when comparing the luminance values of a plurality of spectral images, it is possible to obtain more accurate oxygen saturation by performing alignment, but on the other hand, the load required for alignment is large. There is a problem. In other words, it is not sufficient to align the spectral images in parallel, and it is necessary to deform the images in order to correct distortion caused between the spectral images due to patient movement. In this case, not only the central portion of the image but also the image processing of the peripheral portion is necessary, which places a heavy load on the processor device of the endoscope system.

  Further, as described above, in order to perform accurate alignment, processing such as frequency filtering processing according to the blood vessel depth is also required. Furthermore, the alignment requires not only many complicated processes as described above, but also a high processing speed. This is because it is premised that the oxygen saturation is displayed as a moving image during the examination, and if a long time is required for alignment, a smooth moving image cannot be displayed. As described above, since a great load is applied to the alignment, it has been studied to abolish or simplify the alignment. However, in this case, the oxygen saturation cannot be accurately obtained, artifacts (incorrect information) are generated, and a problem arises that the screen flickers when the oxygen saturation is displayed.

The present invention has been made in view of the above background, and is an endoscope system, image processing apparatus, and image processing that can reduce the burden of alignment and prevent flickering of the screen when oxygen saturation is displayed. It is an object of the present invention to provide an apparatus operating method and an image processing program.

  In order to achieve the above object, an endoscope system according to the present invention includes an illumination unit that sequentially irradiates a subject with first and second illumination lights having different wavelength components, and the first illumination light. Image acquisition means for acquiring a first image by capturing a reflected image and acquiring a second image by capturing a reflected image of the second illumination light; and feature amounts of the first and second images. The first image is gradation-converted based on the feature value calculation means to be obtained and the feature values of the first and second images, so that there is no positional deviation from the first image and is substantially the same as the second image. Tone conversion means for generating a pseudo second image having the same feature amount, and an oxygen saturation image obtained by imaging oxygen saturation of blood hemoglobin based on the first image and the pseudo second image as a moving image And a moving image display means for displaying It is a symptom.

  The feature amount calculating unit includes a histogram analyzing unit that analyzes a density histogram of the first image and the second image, and the gradation converting unit substantially matches the density histogram of the second image. It is preferable to have a pseudo image generation unit that generates the pseudo second image by performing a grayscale conversion process on the first image.

The histogram analysis unit analyzes the density histogram for each of the plurality of areas of the first and second images, and the pseudo image generation unit performs density histograms between the areas of the first image and the second image. so substantially coincide, it is preferable to perform the dark light conversion processing on respective areas of the first image.

Wherein with concentrated light conversion processing, the tone curve generated by the comparison operation of the density histogram of the first image and the second image is preferably used.

  The illuminating means preferably includes a first semiconductor light source that emits the first illumination light and a second semiconductor light source that emits the second illumination light.

  The illumination means wavelength-separates broadband light from the light source and a light source that emits broadband light including light in the wavelength range of the first illumination light and light in the wavelength range of the second illumination light. A wavelength separation unit that generates the first illumination light and the second illumination light may be provided.

  The illumination means includes a broadband broadband light including a first blue narrowband light whose wavelength range is limited to a specific blue region as the first illumination light, and a wavelength region as a specific blue region as the second illumination light. The limited second blue narrow-band light is sequentially irradiated into the subject, and the image acquisition means captures the first blue image by capturing a reflected image of the first illumination light with a color image sensor. Acquiring a plurality of color images, and acquiring a plurality of color images including a second blue image by capturing a reflected image of the second illumination light with a color imaging element, The feature amounts of the first and second blue images are obtained, and the gradation converting means performs the tone conversion on the first blue image based on the feature amounts of the first and second blue images, whereby the first There is no misalignment with the blue image, and it is almost the same as the second blue image. A pseudo second blue image having the characteristic amount of the image is generated, and the moving image display unit images oxygen saturation of the blood hemoglobin based on the first blue image and the pseudo second blue image. An oxygen saturation image may be displayed as a moving image.

  The first illumination light is white light including the first blue narrowband light whose wavelength range is limited to 440 ± 10 nm and fluorescence obtained by wavelength conversion of the first blue narrowband light by a wavelength conversion member, The second blue narrow band light is preferably light whose wavelength range is limited to 470 ± 10 nm.

  The illumination means sequentially irradiates a subject with third illumination light including third blue narrowband light whose wavelength range is limited to a specific blue region in addition to the first and second illumination light, The image acquisition means acquires a plurality of color images including a third blue image by capturing a reflected image of the third illumination light with a color imaging element in addition to the first and second blue images, The feature amount calculating means obtains the feature amount of the third blue image in addition to the first and second blue images, and the gradation converting means adds the first and third colors to the pseudo second blue image. By performing tone conversion on the first blue image based on the feature amount of the blue image, the pseudo third image which has no positional deviation from the first blue image and has approximately the same feature amount as the third blue image. A blue image is generated, and the moving image display means includes the first blue image and the pseudo second blue In addition to images, the pseudo third on the basis of the blue image, the oxygen saturation level image may be used for displaying videos imaging the oxygen saturation of the blood hemoglobin.

  The third blue narrowband light may be light whose wavelength range is limited to 440 ± 10 nm.

  The third blue narrowband light may be light whose wavelength range is limited to 400 ± 10 nm.

  The illumination means includes a broadband broadband light including a first blue narrowband light whose wavelength range is limited to a specific blue region as the first illumination light, and a wavelength region as a specific blue region as the second illumination light. Broadband light including a limited second blue narrowband light is sequentially irradiated into the subject, and the image acquisition means captures the reflected image of the first illumination light with a color image sensor. To obtain a first blue image, a first green image, and a first red image, and by capturing a reflected image of the second illumination light with a color image sensor, a plurality of color images including the second blue image are obtained. The feature amount calculating means obtains the feature amounts of the first and second blue images, and the gradation converting means obtains the first blue image based on the feature amounts of the first and second blue images. Is converted from the first blue image by the tone conversion. And a pseudo second blue image having substantially the same feature quantity as the second blue image is generated, and the moving image display means is configured to generate the first green image, the first red image, and the pseudo second blue image. An oxygen saturation image obtained by imaging the oxygen saturation of blood hemoglobin based on the image may be displayed as a moving image.

  The first illumination light is first white light including the first blue narrowband light whose wavelength range is limited to 440 ± 10 nm and fluorescence obtained by wavelength-converting the first blue narrowband light with a wavelength conversion member. The second illumination light is a second white light including the second blue narrowband light whose wavelength range is limited to 470 ± 10 nm, and a second white light including fluorescence obtained by wavelength-converting the second blue narrowband light by a wavelength conversion member. Preferably there is.

  The present invention also provides a first image obtained by imaging a subject image irradiated with the first illumination light, and a subject image irradiated with a second illumination light different from the first illumination light. In an image processing apparatus that processes a plurality of images including a second image obtained by capturing an image in a frame different from one image, a feature amount calculating unit that calculates a feature amount of the first and second images, Gradation conversion means for generating a pseudo second image corresponding to a second image in which positional deviation between frames is corrected by performing gradation conversion on the first image based on a feature amount of the second image; Display control means for displaying an oxygen saturation image obtained by imaging oxygen saturation of blood hemoglobin based on one image and the pseudo second image on a display means as a moving image is provided.

Further, the present invention provides a first image obtained by imaging a subject image irradiated with the first illumination light, and a subject image irradiated with a second illumination light different from the first illumination light. In an operation method of an image processing apparatus that processes a plurality of images including a second image obtained by capturing an image with a frame different from one image, a step of calculating a feature amount of the first and second images And the gradation converting means performs gradation conversion on the first image based on the feature amounts of the first and second images, so that there is no positional deviation from the first image and the second image A step of generating a pseudo second image having substantially the same feature quantity; and an oxygen saturation level in which the display control means images the oxygen saturation level of blood hemoglobin based on the first image and the pseudo second image. Table for displaying moving images on the display means Characterized by a step of performing a control.

The present invention also provides a first image obtained by imaging a subject image irradiated with the first illumination light, and a subject image irradiated with a second illumination light different from the first illumination light. In an image processing program for processing a plurality of images including a second image obtained by capturing an image in a frame different from one image, a feature amount calculating means for obtaining a feature amount of the first and second images, the first By performing tone conversion on the first image based on the feature amounts of the first and second images, a pseudo second image that has no positional deviation from the first image and has approximately the same feature amount as the second image. Gradation conversion means for generating an image, and display control for displaying an oxygen saturation image obtained by imaging oxygen saturation of blood hemoglobin based on the first image and the pseudo second image on the display means as a moving image What functions as a means A.

  In the present invention, a pseudo spectral image having a density histogram approximate to the density histogram of the spectral image is generated by performing density conversion processing on the white image, and the oxygen saturation is obtained using the pseudo spectral image, that is, white Since the oxygen saturation is obtained using a pseudo-spectral image having no positional deviation from the image, it is not necessary to perform alignment. In addition, artifacts and flickering when oxygen saturation is displayed can be reduced.

It is an external view of the endoscope system of the present invention. It is a front view of the front-end | tip part of an endoscope. It is a block diagram which shows the electric constitution of an endoscope system. It is a graph which shows the spectrum of illumination light. It is a graph which shows the spectral characteristic of the color microfilter of an image pick-up element. It is explanatory drawing which shows the irradiation timing and imaging timing of illumination light in normal observation mode. It is explanatory drawing which shows the irradiation timing and imaging timing of illumination light in functional observation mode. It is a block diagram of a functional image processing unit. It is explanatory drawing which shows a mode that the density histogram for every area of each spectral image is analyzed. It is explanatory drawing which shows a mode that the pseudo | simulation spectral image PB2 'which has the density histogram similar to the spectral image PB2 is produced | generated by performing a light / dark conversion process to the spectral image PB1. It is explanatory drawing which shows a mode that the pseudo | simulation spectral image PB3 'which has the density histogram similar to the spectral image PB3 is produced | generated by performing a light / dark conversion process to the spectral image PB1. It is explanatory drawing which shows a response | compatibility with the absorption spectrum of hemoglobin, and the wavelength of illumination light. It is a graph which shows the correlation with 1st and 2 brightness | luminance ratio S1 / S3, S2 / S3, blood vessel depth, and oxygen saturation. It is explanatory drawing explaining the method of calculating | requiring the coordinate (X ^* , Y ^* ) in a luminance coordinate system from 1st and 2nd luminance ratio S1 ^* / S3 ^* , S2 ^* / S3 ^* . It is explanatory drawing explaining the method of calculating | requiring the coordinate (U ^* , V ^* ) of the blood-vessel information coordinate system corresponding to coordinate (X ^* , Y ^* ). It is explanatory drawing which shows the RGB gain table used when producing | generating a blood vessel depth image. It is explanatory drawing which shows the blood vessel depth image. It is explanatory drawing which shows the RGB gain table used when producing | generating an oxygen saturation image. It is explanatory drawing which shows an oxygen saturation image. It is explanatory drawing which shows the display mode at the time of the moving image display of the blood vessel depth image and the oxygen saturation image. It is explanatory drawing which shows another display form at the time of the moving image display of the blood vessel depth image and the oxygen saturation image. It is a flowchart which shows the flow in which a blood vessel depth image and an oxygen saturation image are displayed as a moving image. It is a block diagram which shows the electric constitution of an endoscope system. It is a graph which shows correlation with blood volume and signal ratio R1 / G1. It is a graph which shows the correlation with oxygen saturation and signal ratio B2 / G1, R1 / G1. It is explanatory drawing which shows the method of calculating | requiring oxygen saturation from signal ratio B2 / G1, R1 / G1 using the correlation shown to FIG. 21B. It is a block diagram which shows the electric constitution of an endoscope system. It is a top view of a rotation filter. It is a graph which shows the spectral transmittance of each filter part of a rotation filter. It is explanatory drawing which shows the irradiation timing and imaging timing of illumination light in normal observation mode. It is explanatory drawing which shows the irradiation timing and imaging timing of illumination light in functional observation mode.

[First Embodiment]
As shown in FIG. 1, an endoscope system 10 according to a first embodiment of the present invention includes an endoscope 11 that images an observation site in a subject, and observation of an observation site based on a signal obtained by imaging. A processor device 12 that generates an image, a light source device 13 that supplies light for irradiating an observation site, and a monitor 14 that displays an observation image are provided. The processor device 12 is provided with a console 15 that is an operation input unit such as a keyboard and a mouse.

  The endoscope system 10 has two operation modes: a normal observation mode in which an observation site is observed under white light, and a function information observation mode. The function information observation mode is a mode in which special function light is used to acquire biological function information related to a living tissue, image it, and observe it. In the present embodiment, the biological function information is information on oxygen saturation and blood vessel depth of blood hemoglobin, and in the function information observation mode, information on oxygen saturation and blood vessel depth is acquired, and these are imaged and observed. An example will be described.

  The endoscope 11 includes a flexible insertion portion 16 that is inserted into a subject, an operation portion 17 provided at a proximal end portion of the insertion portion 16, an operation portion 17, a processor device 12, and a light source device 13. And a universal cord 18 for connecting the two.

  The insertion portion 16 includes a distal end portion 19, a bending portion 20, and a flexible tube portion 21, which are connected in order from the distal end. As shown in FIG. 2, on the distal end surface of the distal end portion 19, the illumination window 22 that irradiates the observation site with illumination light, the observation window 23 that receives the image light reflected by the observation site, and the observation window 23 are washed. An air supply / water supply nozzle 24 for performing air supply / water supply, a forceps outlet 25 for projecting a treatment instrument such as forceps and an electric knife, and the like are provided. An imaging element 44 (see FIG. 3) and an imaging optical system are built in the back of the observation window 23.

  The bending portion 20 includes a plurality of connected bending pieces, and is bent in the vertical and horizontal directions by operating the angle knob 26 of the operation portion 17. By bending the bending portion 20, the direction of the distal end portion 19 is directed in a desired direction. The flexible tube portion 21 is flexible so that it can be inserted into a tortuous duct such as the esophagus or the intestine. The insertion unit 16 includes a communication cable that communicates a drive signal for driving the image sensor 44 and an image signal output by the image sensor 44, and a light guide 43 that guides illumination light supplied from the light source device 13 to the illumination window 22. (See FIG. 3) is inserted.

  In addition to the amble knob 26, the operation unit 17 includes a forceps port 27 for inserting a treatment instrument, an air / water supply button for performing air / water supply operation, a release button for taking a still image, and the like. .

  A communication cable and a light guide 43 extending from the insertion portion 16 are inserted into the universal cord 18, and a connector 28 is attached to the end of the processor device 12 and the light source device 13. The connector 28 is a composite type connector including a communication connector and a light source connector, and one end of a communication cable is disposed on the communication connector, and one end of the light guide 43 is disposed on the light source connector. The endoscope 11 is detachably connected to the processor device 12 and the light source device 13 through the connector 28.

  As shown in FIG. 3, the light source device 13 includes a semiconductor light source unit 31 and a light source control unit 32 that drives and controls them. The light source control unit 32 controls drive timing and synchronization timing of each unit of the light source device 13.

  The semiconductor light source unit 31 includes three laser light sources LD1 to LD3 that respectively emit narrowband light limited to a specific wavelength region in the blue region, and a phosphor 37. As shown in FIG. 4, the laser light source LD1 emits narrowband light N1 whose wavelength range is limited to 440 ± 10 nm, preferably 445 nm. The laser light source LD2 emits narrow band light N2 whose wavelength range is limited to 470 ± 10 nm, preferably 473 nm. The laser light source LD3 emits narrow band light N3 whose wavelength range is limited to 400 ± 10 nm, preferably 405 nm. As the laser light sources LD1 to LD3, InGaN-based, InGaNAs-based, and GaNAs-based laser diodes can be used. Further, as the laser light sources LD1 to LD3, a broad area type laser diode having a wide stripe width (waveguide width) capable of increasing output is preferable.

The light source control unit 32 controls turning on / off of the laser light sources LD <b> 1 to LD <b> 3 and the amount of light through the driver 33. The narrow band light N1 emitted from the laser light source LD1 enters the phosphor 37. As the phosphor 37, for example, a YAG-based or BAM (BgMgAl ₁₀ O ₁₇ ) -based phosphor is used.

  The narrow-band light N1 incident on the phosphor 37 excites the phosphor 37 and emits fluorescence FL in a wavelength region extending from the green region to the red region. The phosphor 37 absorbs a part of the narrowband light N1 to emit fluorescence FL, and transmits the remaining narrowband light N1. The narrow band light N <b> 1 that passes through the phosphor 37 is diffused by the phosphor 37. White light W is generated by the transmitted narrow band light N1 and excited fluorescence FL, and the generated white light W is guided to the combiner 36.

  On the other hand, the light emitted from the laser light sources LD2 and LD3 is guided to the combiner 36 by the optical fiber 34 without passing through the phosphor 37. The combiner 36 is an optical member having a function of multiplexing the light from each optical fiber 34 and couples the optical axes of the light from each optical fiber 34 that selectively enters into one.

  In FIG. 3, a condenser lens 38 and a rod integrator 39 are disposed on the downstream side of the combiner 36. The condensing lens 38 condenses the light emitted from the combiner 36 and causes the light to enter the rod integrator 39. The rod integrator 39 makes the in-plane light quantity distribution uniform by internally reflecting the incident light, and makes the light incident on the incident end of the light guide 43 of the endoscope 11 connected to the light source device 13.

  The endoscope 11 includes a light guide 43, an imaging element 44, an analog processing circuit 45 (AFE: Analog Front End), and an imaging control unit 46. The light guide 43 is a large-diameter optical fiber, a bundle fiber or the like. When the connector 28 on which the incident end of the light guide 43 is disposed is connected to the light source device 13, the incident end is emitted from the rod integrator 39 of the light source device 13. Opposite the edge.

  An illumination lens 48 that adjusts the light distribution angle of illumination light is disposed behind the illumination window 22. The light supplied from the light source device 13 is guided to the irradiation lens 48 by the light guide 43 and irradiated from the illumination window 22 toward the observation site. In the back of the observation window 23, an objective optical system 51 and an image sensor 44 are arranged. The image light reflected by the observation site enters the objective optical system 51 through the observation window 23 and is imaged on the imaging surface 44 a of the imaging element 44 by the objective optical system 51.

  The imaging element 44 is composed of a CCD image sensor, a CMOS image sensor, or the like, and has an imaging surface 44a in which a plurality of photoelectric conversion elements constituting pixels such as photodiodes are arranged in a matrix. The image sensor 44 photoelectrically converts the light received by the imaging surface 44a and accumulates signal charges corresponding to the amount of received light in each pixel. The signal charge is converted into a voltage signal by an amplifier and read out. The voltage signal is output as an imaging signal from the imaging device 44, and the imaging signal is sent to the AFE 45.

  The image pickup element 44 is a color image pickup element. B, G and R three-color micro color filters having spectral characteristics as shown in FIG. 5 are assigned to each pixel on the image pickup surface 44a. Here, a pixel provided with the B micro color filter is a B pixel, a pixel provided with the G micro color filter is a G pixel, and a pixel provided with the R micro color filter is an R pixel. The arrangement of the micro color filters is, for example, a Bayer arrangement.

  As shown in FIGS. 6A and 6B, in the image sensor 44, an accumulation operation for accumulating signal charges and a read operation for reading the accumulated signal charges are performed within an acquisition period of one frame. As shown in FIG. 6A, in the normal observation mode, the white light W is irradiated onto the observation site in accordance with the accumulation timing, and the reflected light is incident on the image sensor 44. The imaging element 44 sequentially outputs the imaging signals B, G, and R for one frame in which the luminance values of the B, G, and R pixels are mixed according to the frame rate. Such an imaging operation is repeated while the normal observation mode is set.

  As shown in FIG. 6B, in the function information observation mode, the laser light sources LD1, LD2, and LD3 are sequentially turned on in accordance with the accumulation timing. When the laser light source LD1 is turned on, white light W is irradiated as an illumination light to the observation site as in the normal observation mode. When the laser light source LD2 is turned on, the observation region is irradiated with narrowband light N2 as illumination light. When the laser light source LD3 is turned on, the narrow-band light N3 is irradiated to the observation site as illumination light.

  The white light W is reflected at the observation site, is color-separated by the micro color filter, and is incident on all the B, G, and R pixels. On the other hand, the narrowband lights N2 and N3 are incident only on the B pixel after being reflected at the observation site. In the functional information observation mode, as in the normal observation mode, the image sensor 44 sequentially outputs the image signals B, G, and R for one frame in which the luminance values of the B, G, and R pixels are mixed according to the frame rate. To do. Such an imaging operation is repeated while the function information observation mode is set.

  As shown in FIG. 3, the AFE 45 includes a correlated double sampling circuit (CDS), an automatic gain control circuit (AGC), and an analog / digital converter (A / D) (all not shown). The CDS performs correlated double sampling processing on the image signal from the image sensor 44 to remove noise caused by signal charge reset. The AGC amplifies the imaging signal from which noise has been removed by CDS. The A / D converts the imaging signal amplified by AGC into a digital imaging signal having a gradation value corresponding to a predetermined number of bits and inputs the digital imaging signal to the processor device 12.

  The imaging control unit 46 is connected to a controller 56 in the processor device 12 and inputs a drive signal to the imaging element 44 in synchronization with a base clock signal input from the controller 56. The imaging element 44 outputs an imaging signal to the AFE 45 at a predetermined frame rate based on the drive signal from the imaging control unit 46.

  As shown in FIG. 3, the processor device 12 includes a DSP (Digital Signal Processor) 57, an image processing unit 58, a frame memory 59, and a display control circuit 60 in addition to the controller 56. The controller 56 includes a CPU, a ROM that stores a control program and setting data necessary for control, a RAM that loads the program and functions as a work memory, and the like. To control.

  The DSP 57 acquires an image signal output from the image sensor 44. The DSP 57 separates an imaging signal in which signals corresponding to B, G, and R pixels are mixed into imaging signals of three colors, performs pixel interpolation processing on the imaging signals of each color, and performs B, G, and R A spectral image of each color is generated. In addition, the DSP 57 performs signal processing such as gamma correction and white balance correction on the imaging signals of the B, G, and R spectral images.

  The frame memory 59 stores image data output by the DSP 57 and processed data processed by the image processing unit 58. The display control circuit 60 reads the image processed image data from the frame memory 59, converts it into a video signal such as a composite signal or a component signal, and outputs it to the monitor 14.

  The image processing unit 58 is provided with a normal image processing unit 61 and a functional image processing unit 62. The normal image processing unit 61 generates a white image W based on the B, G, and R spectral images B, G, and R color-separated by the DSP 57 in the normal observation mode. The normal image processing unit 61 generates a white image W every time the spectral images B, G, and R in the frame memory 59 are updated. The white image W includes a blue spectral image (having a blue component including narrowband light N1 (445 nm)) based on the B pixel signal of the image sensor 44, a green spectral image based on the G pixel signal, and an R pixel. And a red spectral image based on this signal.

  On the other hand, the functional image processing unit 62 acquires the white image W and the spectral images PB2 and PB3 in accordance with the irradiation timing of the white light W and the narrowband light N2 and N3 and the imaging timing of the reflected light in the functional information observation mode. To do. The spectroscopic image PB2 is a blue spectroscopic image (having a blue component including the narrowband light N2 (473 nm)) based on the signal of the B pixel obtained when the narrowband light N2 is irradiated and imaged. PB3 is a blue spectral image (having a blue component including the narrowband light N3 (405 nm)) based on the B pixel signal obtained when the narrowband light N3 is irradiated and imaged.

  Then, the functional image processing unit 62 performs blood hemoglobin based on the white image W, the luminance value S1 of the blue spectral image PB1 of the white image W, the luminance value S2 of the spectral image PB2, and the luminance value S3 of the spectral image PB3. The oxygen saturation StO2 and the blood vessel depth D are calculated, and these pieces of information are imaged. Note that the luminance value S3 represents the brightness level of the observation region, and is a reference signal for normalizing the luminance values S1 and S2 in order to compare the luminance values S1 and S2.

  However, when obtaining the oxygen saturation StO2 and the depth D, the luminance values S1 to S3 indicate the luminance values of the same part, that is, pixels having the luminance values S1 to S3 are reflected at the same part. It is premised on that the accumulated light is accumulated as a charge. However, since the spectral images PB1 to PB3 are not photographed at the same time, there is a positional deviation between the spectral images PB1 to PB3 due to hand movement, movement of the visual field, body movement of the patient due to heartbeat, respiration, and the like. . For this reason, conventionally, after the spectral images PB1 to PB3 are aligned by pattern matching, the oxygen saturation StO2 and the depth D are obtained. However, the spectral images PB1 to PB3 are not only misaligned in parallel, but also have distortions caused by patient movements and the like. If the accuracy is increased, the alignment processing load increases, leading to an increase in cost and a decrease in processing speed.

  On the other hand, the oxygen saturation StO2 and the depth D are obtained by using the luminance values S1 to S3 obtained from the pixels existing at the same position on the image sensor 44 among the pixels of the spectral images PB1 to PB3 without performing alignment. If this information is obtained, the oxygen saturation cannot be obtained with high accuracy, and when such information is displayed, artifacts occur and the screen flickers.

  For this reason, the functional image processing unit 62 calculates the oxygen saturation and the blood vessel depth without performing alignment and uses a method in which the screen does not flicker even if the information on the oxygen saturation and the blood vessel depth is displayed. The calculated information is displayed. Hereinafter, a specific configuration of the functional image processing unit 62 will be described.

  As shown in FIG. 7, the functional image processing unit 62 includes a pseudo spectral image generation unit 63, a luminance ratio calculation unit 64, a correlation storage unit 65, a blood vessel depth-oxygen saturation calculation unit 66, and a blood vessel depth image generation. A unit 67 and an oxygen saturation image generation unit 68 are provided. The pseudo spectral image generation unit 63 includes a histogram analysis unit 90 and a tone curve generation unit 91.

  The histogram analysis unit 90 analyzes the spectral images PB1 to PB3 and examines these density histograms. As is well known, the density histogram represents the relationship between the pixel value (luminance value) and the frequency of each pixel value (the number of pixels having each pixel value). It is represented by a graph with the axis as the frequency (see FIGS. 8 to 10).

  As shown in FIG. 8, the histogram analysis unit 90 divides the spectral images PB1 to PB3 into, for example, 16 parts, and examines the density histogram for each divided area. In FIG. 8, the upper part shows the position of the area where the density histograms of the spectral images PB1 to PB3 are examined, and the middle part visualizes the image of the upper area. Thus, the image in the middle area shows an image of one area of each of the spectral images PB1 to PB3. In FIG. 8 and FIGS. 9 and 10 to be described later, the same symbols PB1 to PB3 as the corresponding spectral images are used. Is attached. The lower part of FIG. 8 shows density histograms HG1 to HG3 obtained by analyzing the images PB1 to PB3 in the middle part. The density histogram HG1 corresponds to the spectral image PB1, the density histogram HG2 corresponds to the spectral image PB2, and the density histogram HG3 corresponds to the spectral image PB2. The histogram analysis unit 90 sequentially checks the density histograms HG1 to HG3 for each area, and finally checks the density histograms HG1 to HG3 of all areas of all the spectral images PB1 to PB3.

  The tone curve generation unit 91 generates two tone curves TCa and TCb (see FIGS. 9 and 10) used in the light / dark conversion process described later. As is well known, the tone curve represents the relationship between the input pixel value and the output pixel value. For example, the horizontal axis represents the input gradation value, and the vertical axis represents the output gradation value. It is expressed as a line segment composed of a set of points indicating the output gradation value (see FIGS. 9 and 10).

  As shown in FIG. 9, the tone curve TCa is generated based on the comparison result by comparing the density histogram HG1 of the spectral image PB1 (445 nm) with the density histogram HG2 of the spectral image PB2 (473 nm). The tone curve TCa is a tone curve for converting the density histogram HG1 into the density histogram HG2.

  The tone curve TCa is generated for each area. That is, when the spectral image PB1 is divided into 16 areas as in this example, the image of each area of the spectral image PB1 has a density histogram similar to that of the corresponding area of the spectral image PB2. Sixteen types of tone curves TCa are generated.

  For example, the spectral image PB2 of FIG. 9 has pixel values in a wide range from a low gradation (a portion with a low pixel value) to a high gradation (a portion with a high pixel value) with respect to the spectral image PB1, that is, a high contrast. I understand. In this case, the tone curve TCa generated by the tone curve generation unit 91 reduces the output value for the low gradation (low pixel value) portion and increases the high gradation (pixel value of the pixel value) so as to increase the contrast of the spectral image PB1. For the (high) part, the output value will be raised.

  On the other hand, as shown in FIG. 10, the tone curve TCb is generated based on the comparison result by comparing the density histogram HG1 of the spectral image PB1 (445 nm) with the density histogram HG3 of the spectral image PB3 (405 nm). . The tone curve TCb is a tone curve for converting the density histogram HG1 into the density histogram HG3.

  The tone curve generation unit 91 generates a tone curve TCb for each area by the same method as the generation of the tone curve TCa. That is, in this example, 16 types of tone curves TCb are generated so that the image of each area of the spectral image PB1 has the same density histogram as the image of each corresponding area of the spectral image PB3.

  For example, it can be seen that the spectral image PB3 in FIG. 10 is a dark image in which pixel values that are more frequent than the spectral image PB1 are concentrated in a low gradation. In this case, the tone curve TCb generated by the tone curve generation unit 91 has an arcuate shape curved downward so as to lower the pixel value of the spectral image PB1 as a whole.

  As shown in FIG. 9, the pseudo spectral image generation unit 63 performs a grayscale conversion process using the tone curve TCa corresponding to this area on each area of the spectral image PB1, thereby converting the grayscale converted image PB2 for each area. ′ Is generated. The grayscale converted image PB2 ′ has no positional deviation with respect to the spectral image PB1, and has a density histogram similar to that of the spectral image PB2 (high contrast). Note that the grayscale conversion image PB2 ′ shown in FIG. 17 shows an image of one area of a pseudo-spectral image PB2 ′ to be described later, and both are given the same reference numerals.

  The pseudo spectral image generation unit 63 sequentially performs density conversion processing using the tone curve TCa for each area of the spectral image PB1, and finally generates a density conversion image PB2 ′ for all areas of the spectral image PB1. Thereafter, the pseudo spectral image generation unit 63 generates the pseudo spectral image PB2 ′ by connecting the grayscale conversion images PB2 ′ of each area into one. The pseudo spectral image PB2 'generated in this way has no positional deviation with respect to the spectral image PB1, and the density histogram of each area is the same as that of the spectral image PB2.

  Further, the pseudo spectral image generation unit 63 performs density conversion processing for each area of the spectral image PB1 using the tone curve TCb corresponding to this area, as shown in FIG. An image PB3 ′ is generated. The grayscale converted image PB3 ′ has no positional deviation with respect to the spectral image PB1, and has a density histogram similar to that of the spectral image PB3 (the luminance is generally low). Note that the grayscale converted image PB3 ′ shown in FIG. 10 shows an image of one area of a pseudo-spectral image PB3 ′ to be described later, and both are given the same reference numerals.

  The pseudo spectral image generation unit 63 sequentially performs density conversion processing using the tone curve TCb for each area of the spectral image PB1, and finally generates a density conversion image PB3 ′ of all areas of the spectral image PB1. Thereafter, the pseudo spectral image generation unit 63 generates the pseudo spectral image PB3 ′ by connecting the grayscale conversion images PB3 ′ of each area into one. The pseudo spectral image PB3 ′ generated in this way has no positional deviation with respect to the spectral image PB1, and the density histogram of each area is the same as that of the spectral image PB2.

  In this example, the tone curve generation unit generates a curved tone curve. However, a polygonal tone curve may be generated. In addition, the tone curve that minimizes the output value when the input value is the minimum and the maximum output value when the input value is the maximum, that is, the positions of one end and the other end are fixed. Although an example in which a tone curve in which only the shape of the portion connecting the other end and the other end is generated has been described, a tone curve in which the positions of the one end and the other end also change may be generated. Further, a plurality of types of tone curves may be stored in advance, and which tone curve is to be used may be determined based on the comparison result of the density histogram.

  The luminance ratio calculation unit 64 calculates first and second luminance ratios used for calculating oxygen saturation and blood vessel depth. The first luminance ratio is represented by a ratio (S1 / S2 ′) between the luminance value S1 of the spectral image PB1 and the luminance value S2 ′ of the pseudo spectral image PB3 ′, and the second luminance ratio is the luminance of the pseudo spectral image PB2 ′. It is represented by a ratio (S2 ′ / S3 ′) between the value S2 ′ and the luminance value S3 ′ of the pseudo spectral image PB3 ′. The first and second luminance ratios are calculated for all pixels between each image. Here, since the pseudo spectral images PB2 ′ and PB3 ′ are generated by subjecting the spectral image PB1 to density conversion processing, there is no positional deviation between these images, so the first and second luminance ratios are It is an appropriate value. By using such an appropriate first and second luminance ratio, the oxygen saturation and the blood vessel depth can be obtained with high accuracy.

  In the following description, the first luminance ratio is “S1 / S2” obtained by removing “′” from “S1 / S2 ′”, and the second luminance ratio is “S2 ′ / S3 ′” to “′”. And “S2 / S3”.

  The correlation storage unit 65 stores the correlation between the first and second luminance ratios S1 / S3, S2 / S3, the oxygen saturation level in the blood vessel, and the blood vessel depth (see FIG. 12). This correlation is obtained by analyzing the light reflection characteristics of the living tissue shown in FIG. 11 and a large number of image analysis accumulated in the diagnosis so far.

  In FIG. 11, reduced hemoglobin 70 that is not bonded to oxygen and oxidized hemoglobin 71 that is bonded to oxygen have different light absorption characteristics and have the same absorption coefficient μa (each hemoglobin 70, 71 in FIG. 11). Except for the intersection of (2), a difference occurs in the extinction coefficient μa. For example, in the narrowband light N1 (445 nm) and N2 (473 nm), there is a difference in the extinction coefficient μa. Therefore, when the oxygen saturation changes, the luminance values S1 and S2 of the spectral images PB1 and PB2 change. On the other hand, in the narrowband light N3 (405 nm), there is no difference in the extinction coefficient μa, so the luminance value of the spectral image PB3 does not change even if the oxygen saturation changes.

  Specifically, the correlation shown in FIG. 12 is obtained by the light reflection characteristics of the living tissue and the many image analyzes accumulated in the diagnosis so far. This correlation is stored in the correlation storage unit 65.

  In FIG. 12, a luminance coordinate system 96 is an XY coordinate system having two axes of XY, and the first luminance ratio S1 / S3 is assigned to the X axis, and the second luminance ratio S2 / S3 is assigned to the Y axis. Yes. The blood vessel information coordinate system 97 is a UV coordinate system having two UV axes provided on the luminance coordinate system 96. The U axis is assigned to the blood vessel depth D and the V axis is assigned to the oxygen saturation StO2. The U axis has a positive slope because the blood vessel depth has a positive correlation with the luminance coordinate system 96. With respect to this U-axis, it indicates that the blood vessel is in a shallower position as it goes to the upper right, and that the blood vessel is in a deeper position as it goes to the lower left. On the other hand, the V-axis has a negative slope since the oxygen saturation StO2 has a negative correlation with the luminance coordinate system. With respect to the V axis, the oxygen saturation StO2 is lower as it goes diagonally to the left, and the oxygen saturation StO2 is higher as it goes diagonally to the right. In the blood vessel information coordinate system 97, the U axis and the V axis are orthogonal to each other at an intersection point P.

  The blood vessel depth-oxygen saturation calculating unit 66 receives the first luminance ratio S1 / S3 and the second luminance ratio S2 / S3 for each pixel calculated by the luminance ratio calculating unit 64. When the first and second luminance ratios S1 / S3 and S2 / S3 are input, the blood vessel depth-oxygen saturation calculation unit 66 is input with reference to the correlation stored in the correlation storage unit 65. The oxygen saturation StO2 and the blood vessel depth D corresponding to the first and second luminance ratios S1 / S3, S2 / S3 are specified.

The first luminance ratio S1 / S3 and the second luminance ratio S2 / S3 input to the blood vessel depth-oxygen saturation calculation unit 66 are respectively set as the first luminance ratio S1 ^* / S3 ^* and the second luminance ratio S2. Assuming ^* / S3 ^* , the blood vessel depth-oxygen saturation calculating unit 66 specifies the oxygen saturation StO2 and the blood vessel depth D as follows.

As shown in FIG. 13A, the blood vessel depth-oxygen saturation calculating unit 66 uses coordinates (X ^*) corresponding to the first and second luminance ratios S1 ^* / S3 ^* , S2 ^* / S3 ^* in the luminance coordinate system 96 ^. , Y ^* ). Coordinates ^(X *, ^{Y *)} When is identified, as shown in FIG. 13B, in the blood vessel information coordinate system 97, the identified coordinates ^(X *, ^{Y *)} and a V-axis is a coordinate axis of the oxygen saturation, The coordinates (U ^* , V ^* ) are specified by projecting on the U axis which is the coordinate axis of the blood vessel depth. Thereby, blood vessel depth information U ^* and oxygen saturation information V ^* are obtained for one pixel. The blood vessel depth-oxygen saturation calculation unit 66 repeats such processing for all pixels for one screen to obtain blood vessel depth information U ^* and oxygen saturation information V ^* for all pixels.

  In FIG. 7, the blood vessel depth image generation unit 67 stores an RGB gain table 67a. As shown in FIG. 14, in the RGB gain table 67a, the blood vessel depth is abscissa, and the gain values gr, gg, and the blue spectral image PB1, the green spectral image PG1, and the red spectral image PR1 in the white image W are displayed. gb is assigned to the vertical axis. Here, the white image W is captured by irradiating the white light W, that is, acquired by capturing the first frame in the function information observation mode.

  The blood vessel depth image generation unit 67 reflects the blood vessel depth information on the white image W using the blood vessel depth obtained by the blood vessel depth-oxygen saturation calculation unit 66 and the RGB gain table 67a. Specifically, in the RGB gain table 67a, a gain value corresponding to the blood vessel depth obtained by the blood vessel depth-oxygen saturation calculating unit 66 is specified for each pixel of the white image W. Then, a blood vessel depth image is obtained by multiplying the gain value by the pixel value of the white image W for each pixel.

  As shown in FIG. 14, in the RGB gain table 67a, the gain value gr gradually increases and the gain values gg and gb gradually decrease as the blood vessel depth increases. For this reason, as shown in FIG. 15, in the generated blood vessel depth image 73, red becomes stronger as compared with the hue of the white image W from the surface blood vessel 75 toward the middle blood vessel 76 and the deep blood vessel 77.

  In FIG. 7, the oxygen saturation image generation unit 68 stores an RGB gain table 68a. As shown in FIG. 16, in the RGB gain table 68a, the oxygen saturation is on the horizontal axis, the gain values gr, gg, and the blue spectral image PB1, the green spectral image PG1, and the red spectral image PR1 in the white image W. gb is assigned to the vertical axis.

  The oxygen saturation image generation unit 68 reflects the blood vessel depth information on the white image W using the oxygen saturation obtained by the blood vessel depth-oxygen saturation calculation unit 66 and the RGB gain table 68a. Specifically, in the RGB gain table 68a, a gain value corresponding to the oxygen saturation obtained by the blood vessel depth-oxygen saturation calculating unit 66 is specified for each pixel of the white image W. Then, for each pixel, an oxygen saturation image is obtained by multiplying the gain value by the pixel value of the white image W.

  As shown in FIG. 16, in the RGB gain-in table 68a, the gain values gr, gg, and gb are all set to 1 when the oxygen saturation is 100% to 60%. On the other hand, when the oxygen saturation is less than 60%, the gain value gr gradually decreases as the oxygen saturation decreases, and the gain values gg and gb increase gradually as the oxygen saturation decreases. For this reason, as shown in FIG. 17, in the generated blood vessel depth image 74, the region 80 where the oxygen saturation is 100% to 60% is reproduced with the same hue as the white image W, and the oxygen saturation is 60%. The area 81 below the region 81 has a cyan hue stronger than that of the white image W. Furthermore, in the region 81, the cyan hue becomes stronger as the oxygen saturation level is lowered. The oxygen saturation in a normal state is about 100% for arteries and about 70% for veins.

  In the present embodiment, the pixel value of the white image W is changed according to the blood vessel depth and the oxygen saturation, but the color characteristic values of the white image W such as the hue, brightness, and saturation are not the pixel values themselves but the oxygen saturation. You may change according to. At that time, when the hue, lightness, and saturation are changed according to the blood vessel depth and oxygen saturation, the blood vessel depth, oxygen saturation, and pixel value of the white image W are replaced with the hue, lightness instead of the RGB gain table. A hue matrix, a lightness matrix, and a saturation matrix that are associated with conversion values for conversion to saturation are used.

  In the present embodiment, the gain value is changed when the oxygen saturation is less than 60%. However, the present invention is not limited to this, and the gain value is set to a value lower than 60%. It may be emphasized. On the contrary, it may be set to a value slightly higher than 60% to emphasize a region suspected of being hypoxic as much as possible.

  Furthermore, in this embodiment, the color tone is changed by multiplying the gain value by the image signal. However, an offset value corresponding to the gain value may be added to the log-converted image signal.

  The display control circuit 60 sequentially acquires the blood vessel depth image and the oxygen saturation generated as described above, and displays these sequentially acquired images on the monitor 14 as a blood vessel depth moving image and an oxygen saturation moving image. Since these blood vessel depth movies and oxygen saturation movies are generated based on images whose positional deviation has been corrected by the density conversion process, they do not flicker, and the blood vessel depth and oxygen saturation in the movies are not flickered. The information about is also displayed accurately.

  Various patterns can be considered as the display mode of the moving image. For example, as shown in FIG. 17A, a blood vessel depth movie and an oxygen saturation movie may be displayed on the monitor 14 in parallel. Further, as shown in FIG. 17B, only one of the blood vessel depth movie and the oxygen saturation movie is displayed on the monitor 14, and the movie displayed on the monitor 14 is appropriately switched by the operation of the console 15. Also good.

  Next, a procedure until a blood vessel depth image and an oxygen saturation image are displayed will be described based on the flowchart shown in FIG. Since details of the processing performed in each step described later have already been described, the flow until the blood vessel depth image and the oxygen saturation image are displayed will be described below, and the processing in each step will be described. Has simplified the description.

  As shown in FIG. 19, imaging of the first frame is performed by irradiating and imaging white light W, and a white image W and a blue spectral image PB <b> 1 of the white image W are generated. Next, the second frame is imaged by irradiating the narrowband light N2, and a blue spectral image PB2 of the image obtained by this imaging is generated. Subsequently, the narrow-band light N3 is irradiated and the third frame is imaged, and a blue spectral image PB3 of the image obtained by this imaging is generated.

  After the generation of the spectral images PB1 to PB3, these density histograms are analyzed. The density histogram is analyzed for each area of the spectral images PB1 to PB3, and finally the density histogram is analyzed for all areas of all the spectral images.

  After analyzing the density histograms of the spectral images PB1 to PB3, the density histograms of the spectral images PB1 and PB2 are compared, and based on this comparison result, the density histogram of the spectral image PB1 is the same as the density histogram of the spectral image PB2. A tone curve TCa is generated. The tone curve TCa is generated by the number of areas so that the density histogram of each area of the spectral image PB1 is the same as the density histogram of the corresponding area of the spectral image PB2.

  Subsequently, the density histograms of the spectral images PB1 and PB3 are compared, and a tone curve TCb for making the density histogram of the spectral image PB1 the same as the density histogram of the spectral image PB3 is generated based on the comparison result. Similarly to the tone curve TCa, the tone curves TCb are generated by the number of areas.

  When the tone curves TCa and TCb are generated, a pseudo spectral image PB2 ′ obtained by converting the spectral image PB1 using a tone curve TCa and a pseudo spectral image PB3 ′ obtained by converting the spectral image PB1 using a tone curve TCb. The pseudo spectral image PB2 ′ generated in this way has no positional deviation from the spectral image PB1, and has a density histogram similar to that of the spectral image PB2. Similarly, the pseudo spectral image PB3 ′ has no positional deviation from the spectral image PB1, and has a density histogram similar to that of the spectral image PB3.

  When the pseudo spectral images PB2 ′ and PB3 ′ are generated, the first luminance ratio S1 / S3 and the second luminance ratio S2 / S3 are calculated from the spectral image PB1 and the pseudo spectral images PB2 ′ and PB3 ′. When the first luminance ratio S1 / S3 and the second luminance ratio S2 / S3 are calculated, the blood vessel depth and the oxygen saturation are calculated from the correlation between these, the blood vessel depth, and the oxygen saturation. Then, a blood vessel depth image and an oxygen saturation image in which information indicating the blood vessel depth and the oxygen saturation is added to the white image W are generated. The generated image is displayed on the monitor 14 as an oxygen saturation moving image and a blood vessel depth moving image.

  As described above, in the present invention, when calculating the blood vessel depth and the oxygen saturation, the spectral image PB1 and the pseudo spectral image PB2 'and pseudo spectral image PB3' having no positional deviation are used. This saves time and effort for alignment. In addition, since there is no positional deviation in all the images related to the calculation and display of the blood vessel depth and oxygen saturation, the information on the blood vessel depth and oxygen saturation is shifted from the position where it should originally be displayed due to the positional deviation between the images. It is not displayed. For this reason, it is possible to prevent artifacts and screen flicker when displaying information on the blood vessel depth and the oxygen saturation.

  The blood vessel depth information and the oxygen saturation information may be displayed as character information instead of or in addition to the image. In addition, oxygen saturation is imaged, but the oxygen saturation image may be expressed as “blood volume (sum of oxidized hemoglobin and reduced hemoglobin) × oxygen saturation in place of or in addition to the form shown in the above example. (%) "Which includes an image of an oxygenated hemoglobin index obtained from"% ".

  Further, in the present embodiment, an example has been described in which the oxygen saturation and the blood vessel depth are calculated using the luminance value S3 (that is, the luminance value of the image captured by irradiating the narrowband light N3) as the reference signal. However, since the luminance value S3 is a reference signal, instead of the luminance value S3, the luminance value of an image captured by irradiating light having a wavelength range different from that of the narrowband light N3 is used as a reference signal, and the oxygen saturation level (And blood vessel depth) may be calculated.

  As light having a wavelength range different from that of the narrowband light N3, for example, narrowband light N1 (that is, light having a wavelength range of 440 ± 10 nm) can be considered. At this time, the luminance value used as the reference signal (the luminance value used instead of the luminance value S3) is the luminance value of the image captured by irradiating the narrowband light N1 (that is, the luminance value S1).

  As described above, when the luminance value S1 is used instead of the luminance value S3, the light source such as the laser light source LD3 that emits the narrowband light N1 and the optical fiber 34 that guides the narrowband light N1 from the laser light source LD3 is used. The means can be abolished. The luminance value S1 is a luminance value of the spectral image PB1 of the white image W captured by irradiating the white light W. Therefore, the luminance value S1 can be acquired by imaging the first frame in the function information observation mode. For this reason, the oxygen saturation (and blood vessel depth) can be calculated without performing the imaging of the third frame in the function information observation mode. When calculating the oxygen saturation (and blood vessel depth), the correlation between the first and second luminance ratios and the oxygen saturation (and blood vessel depth) is different from that shown in FIG. A different correlation is used.

  Of course, in the case of using the brightness value S1 instead of the brightness value S3, the narrow-band light N1 may be emitted to capture the third frame, and the brightness value S1 may be acquired from the image captured in the third frame. . However, in this case, there is a positional deviation between the image captured in the first frame and the image captured in the third frame. For this reason, when calculating the oxygen saturation (and blood vessel depth), the image captured in the first frame is subjected to grayscale conversion, and a pseudo histogram having a density histogram similar to that of the image captured in the third frame is used. An image is generated, and the luminance value of the pseudo image is used as the luminance value S1.

  When the luminance value S1 is used instead of the luminance value S3 and the third frame is imaged by irradiating the narrowband light N1, the phosphor 37 is provided in the optical path of the narrowband light N1 emitted from the laser light source LD1. An optical path switching mechanism that inserts or retracts is provided, and the phosphor 37 is inserted into the optical path of the narrowband light N1 when the first frame is imaged, and from the optical path of the narrowband light N1 when the third frame is captured. The phosphor 37 may be retracted. Of course, two laser light sources LD1 are provided, and one of the laser light sources LD1 is dedicated to the imaging of the first frame that irradiates the narrowband light N1 through the phosphor 37, and the other laser light source LD1 is not through the phosphor 37. It is good also for exclusive use of the 3rd frame imaging which irradiates narrow band light N1.

[Second Embodiment]
In the first embodiment, imaging is performed by irradiating three types of light: white light W (445 nm to 700 nm), narrowband light N2 having a center wavelength of 473 nm, and narrowband light N3 having a center wavelength of 405 nm. The example in which the present invention is applied to the endoscope system that calculates the blood vessel depth and the oxygen saturation from the obtained image has been described. However, the first white light W1 (440 nm to 700 nm) and the second white light (460 nm to It is also possible to apply the present invention to an endoscope system that images by irradiating two types of light (700 nm) and calculates the blood volume and oxygen saturation from the image obtained by the imaging.

  Hereinafter, an example in which the present invention is applied to an endoscope system for calculating blood volume and oxygen saturation will be described with reference to FIGS. In addition, in the description using drawings after FIG. 20, the same code | symbol is attached | subjected about the member similar to embodiment mentioned above, and description is abbreviate | omitted. In addition, since the illumination light emission, imaging timing, and oxygen saturation calculation method are substantially the same as those in the first embodiment, description thereof is omitted.

  As shown in FIG. 20, in the endoscope system 100, the light source that emits the narrow band light N3 having the center wavelength of 405 nm and the related parts (driver, light guide means, etc.) are abolished from the first embodiment. . Further, the phosphor 37 is provided not between the laser light source LD 1 and the optical fiber 34 but between the combiner 36 and the condenser lens 38. Therefore, not only the narrowband light N1 having the center wavelength of 445 nm but also the narrowband light N2 having the center wavelength of 473 nm is incident on the phosphor 37. Therefore, when the narrow-band light N1 enters the phosphor 37, the observation site is irradiated with the first white light W1 including the narrow-band light N1 and the fluorescence excited by the phosphor 37 by the narrow-band light N1. On the other hand, when the narrow-band light N2 is incident on the phosphor 37, the observation site is irradiated with the narrow-band light N2 and the second white light W2 including the fluorescence excited by the phosphor 37 by the narrow-band light N2.

  In the second embodiment, in the normal observation mode, as in the first embodiment, only the first white light W1 is irradiated to the observation site and imaged. On the other hand, in the function information observation mode, the first white light W1 and the second white light W2 are alternately irradiated, and imaging is performed for each irradiation. Therefore, a white image W1 is obtained in the irradiation frame of the first white light W1, and a white image W2 is obtained in the irradiation frame of the second white light W2. Then, the functional image processing unit 101 calculates and images the blood volume and oxygen saturation based on the white image W1 and the blue spectral image PB2 in the white image W2. The calculation of blood volume and oxygen saturation will be described later.

  Here, since the image capturing timings of the white image W1 and the spectral image PB2 are different, there is a positional deviation between the white image W1 and the spectral image PB2. Therefore, similarly to the first embodiment, a pseudo spectral image in which the positional deviation is corrected is generated by the density conversion process. In the second embodiment, a tone curve is generated by the same method as in the first embodiment based on the density histogram of the blue spectral image PB1 in the white image W1 and the density histogram of the spectral image PB2. Then, by using this generated tone curve, the spectral image PB1 is subjected to density conversion processing to obtain a pseudo spectral image PB2 ′ in which the positional deviation between the images is corrected. In the blood volume image and the oxygen saturation image generated based on the pseudo-spectral image PB2 ′ and the white image W1, the positional deviation is eliminated, so that artifacts and screen flicker can be reduced.

  For the calculation of the blood volume and the oxygen saturation, the signal ratio B2 / G1 between the pseudo spectral image PB2 ′ and the green spectral image PG1 in the white image W1, and the red spectral image in the white image W1. Two signal ratios of signal ratio R1 / G1 between PR1 and spectral image PG1 are used. The signal ratio B2 / G1, the signal ratio R1 / G1, the blood volume, and the oxygen saturation have a correlation as shown in FIGS. As shown in FIG. 21A, the blood volume has a correlation with the signal ratio R1 / G1, and the blood volume increases as R1 / G1 increases. Further, as shown in FIG. 21B, the oxygen saturation has a correlation with the signal ratios B2 / G1 and R1 / G1. In FIG. 21B, the oxygen saturation is represented by contour lines, and the oxygen saturation decreases as the signal ratio B2 / G1 decreases. In addition, according to the correlation shown in FIG. 21B, even if the signal ratio B2 / G1 is the same, the oxygen saturation is different if the signal ratio R1 / G1 is different.

By using the correlation as shown in FIG. 21B, the oxygen saturation in each pixel is obtained from the signal ratios B2 / G1 and B2 / G1. Specifically, as shown in FIG. 22, the corresponding point P corresponding to the signal ratios B2 ^* / G1 ^* , R1 ^* / G1 ^* is specified. When the corresponding point P is between the oxygen saturation = 0% lower limit line 93 and the oxygen saturation = 100% upper limit line 94, the percentage value indicated by the corresponding point P is defined as the oxygen saturation level. To do. In the case of FIG. 22, since the corresponding point P is located on the contour line of 60%, the oxygen saturation is 60%.

  On the other hand, when the corresponding point is out of the range between the lower limit line 93 and the upper limit line 94, the oxygen saturation is 0% when the corresponding point is located above the lower limit line 93, and the corresponding point is higher than the upper limit line 94. When located below, the oxygen saturation is set to 100%. If the corresponding point is out of the range between the lower limit line 93 and the upper limit line 94, the reliability of the oxygen saturation in the pixel may be lowered and not displayed on the monitor 14.

[Third Embodiment]
In the first embodiment, a plurality of light sources that emit illumination light in a desired wavelength range are provided, and an example in which imaging is performed by sequentially irradiating illumination light from each light source has been described. A white light source (such as a xenon lamp) may be provided, and illumination light in a desired wavelength region may be separated from the white light source by, for example, a rotary filter, and imaging may be performed by sequentially irradiating the separated light. Hereinafter, an example in which the present invention is applied to an endoscope system using such a light source and a rotary filter will be described with reference to FIGS.

  In the third embodiment, an endoscope system 200 shown in FIG. 23 is used. In the endoscope system 200, a light source device 202 is different from the light source device 13 of the first embodiment, and a monochrome image pickup device 201 that is not provided with a color filter is used instead of the color image pickup device 44. The point provided in the mirror 11 is different from the first embodiment. Since other than that is substantially the same as 1st Embodiment, description is abbreviate | omitted.

  The light source device 202 includes a white light source unit 203, a rotary filter 204, a motor 205, a shift unit 206, and a light source control unit 207 that drives and controls each unit of the light source device 202.

  The white light source unit 203 includes a light source 203a and a diaphragm 203b. The light source 203a includes a xenon lamp, a halogen lamp, a metal halide, and the like, and is driven by the light source control unit 207 to emit broadband light BB having a wavelength range of 400 nm to 700 nm (see FIG. 25). The diaphragm 203b is driven by the light source control unit 207 to adjust the light amount of the broadband light BB.

  As shown in FIG. 24, the rotary filter 204 is provided so as to be rotatable about the rotary shaft 210. A motor 205 is connected to the rotating shaft 210. The motor 205 is driven by the light source control unit 207 and rotates the rotary filter 204. The rotary filter 204 includes an inner region 211 close to the rotation shaft 210 and an outer region 212 far from the rotation shaft 210. The shift unit 206 is driven by the light source control unit 207 and moves the rotary filter 204 to an inner position where the inner region 211 is inserted on the optical path of the broadband light BB in the normal observation mode, and the outer region 212 in the functional image observation mode. The rotary filter 204 is moved to an outer position to be inserted on the optical path of the light BB.

  In the inner region 211, three filters of an R filter 213, a G filter 214, and a B filter 215 are arranged. In the outer region 212, six filters of an R filter 216, a G filter 217, a B filter 218, an N1 filter 219, an N2 filter 220, and an N3 filter 221 are arranged. As shown in FIG. 25, the R filters 213 and 216 pass R light having a wavelength range of 580 nm to 760 nm among the broadband light BB. The G filters 214 and 217 pass the G light having a wavelength range of 450 nm to 630 nm among the broadband light BB. The B filters 215 and 218 pass B light having a wavelength range of 380 nm to 500 nm among the broadband light BB. The N1 to N3 filters 219 to 221 pass the narrowband light N1 to N3 described in the first embodiment, respectively.

  Thus, in the endoscope system 200, three colors of RGB light are sequentially emitted in the normal observation mode, and in the functional image observation mode, three types of narrowband light N1 to N3 are added in addition to the three colors of RGB. Irradiated sequentially. Then, in the normal observation mode, as shown in FIG. 26, imaging is performed for three frames in accordance with the irradiation of each RGB color, and a white image W is generated by synthesizing the acquired RGB image to generate the monitor 14. To display.

  On the other hand, in the functional image observation mode, as shown in FIG. 27, imaging for six frames is performed, and an oxygen saturation image and a blood vessel depth image are generated from the obtained six images. Specifically, a white image W serving as a base for generating an oxygen saturation image and a blood vessel depth image is generated by synthesizing three images picked up in accordance with the irradiation of RGB colors. Further, by comparing the density histogram of the spectral image PB1 imaged in accordance with the irradiation of the narrow band light N1 with the density histogram of the blue image Bc obtained by the B color irradiation of the white image W, the tone curve T1. Is generated. Similarly, a tone curve T2 is generated by comparing the density histogram of the spectral image PB2 captured in accordance with the irradiation of the narrowband light N2 and the density histogram of the blue image Bc, and the irradiation of the narrowband light N3 is performed. A tone curve T3 is generated by comparing the density histogram of the spectral image PB3 captured together with the density histogram of the blue image Bc. Then, pseudo-spectral images PB1 ′ to PB3 ′ are generated by performing tone conversion processing on the blue image Bc using the tone curves T1 to T3. The tone curve generation method is the same as that in the first embodiment.

  Thereafter, from the luminance values S1 ′ to S3 ′ of the pseudo spectral images PB1 ′ to PB3 ′, the first luminance ratio (S1 ′ / S3 ′ in this example) and the second luminance ratio (S2 ′ / in this example). S3 ′) is calculated, and the blood vessel depth and the oxygen saturation are calculated based on the first and second luminance ratios. Then, by reflecting these information on the white image W, a blood vessel depth image and an oxygen saturation are generated and displayed on the monitor 14.

  In the third embodiment, the example in which the white light source and the rotation filter are used in the endoscope system that calculates the blood vessel depth and the oxygen saturation by irradiating three types of light has been described. As in the embodiment, in an endoscope system that calculates the blood volume and the oxygen saturation level by irradiating two types of light, a white light source and a rotary filter may be used.

  In the above embodiment, an example in which no alignment is performed has been described. However, for example, simple alignment such as simply sliding an image is performed, and a pseudo image is generated based on the image after alignment. Good.

  In the above-described embodiment, an example in which the entire image area is pseudo-imaged has been described. However, only a partial area of the image may be pseudo-imaged. In this case, for example, it is conceivable that the central portion of the image is not subjected to the alignment and pseudo-imaged, but the peripheral portion of the image is pseudo-imaged.

  Furthermore, in the above embodiment, the tone conversion process using the tone curve is performed in order to generate a pseudo-spectral image in which the positional deviation between frames is corrected. It may be a conversion process.

  In the above embodiment, only the low oxygen region (oxygen saturation is 0 to 60%) in the white image W is displayed in pseudo color, and the other high oxygen regions (60 to 100%) are displayed in the same color. That is, only a part of the white image W is displayed in pseudo color. Instead, oxygen saturation may be displayed in pseudo color in all regions from the low oxygen region to the high oxygen region.

  Furthermore, in the said embodiment, although the laser light source which consists of a laser diode was illustrated as a semiconductor light source, the LED light source which uses LED instead of the laser diode may be used.

  In the above embodiment, the example in which the light source device and the processor device are configured separately has been described, but the two devices may be configured integrally. The present invention also relates to a system comprising an ultrasonic endoscope in which an image sensor and an ultrasonic transducer are built in the tip and a processor device for performing image processing, a system comprising a capsule endoscope and a processor device for performing image processing, etc. The present invention can also be applied to other types of endoscope systems.

10, 100, 200 Endoscope system 14 Monitor 31 Semiconductor light source unit 32, 207 Light source controller 44, 201 Image sensor 57 DSP
58 Image processing unit 62, 101 Functional image processing unit 63 Pseudo spectral image generation unit 67 Blood vessel depth image generation unit 68 Oxygen saturation image generation unit 90 Histogram analysis unit 91 Tone curve generation unit 203 White light source unit 204 Rotation filters LD1 to LD3 Laser light sources PB1 to PB3 Spectral images PB1 ', PB2', PB3 'Pseudospectral images TCa, TCb, T1 to T3 Tone curves HG1 to HG3 Density histogram

Claims (16)

Illumination means for sequentially irradiating the subject with first and second illumination lights having different wavelength components;
An image acquisition means for acquiring a first image by capturing a reflected image of the first illumination light and acquiring a second image by capturing a reflected image of the second illumination light;
Feature quantity calculating means for obtaining the feature quantities of the first and second images;
By performing tone conversion on the first image based on the feature amounts of the first and second images, the pseudo image having no feature deviation from the first image and having approximately the same feature amount as the second image. Gradation converting means for generating a second image;
An endoscope system comprising: a moving image display means for displaying a moving image of an oxygen saturation image obtained by imaging oxygen saturation of blood hemoglobin based on the first image and the pseudo second image. The feature amount calculating means includes a histogram analysis unit that analyzes density histograms of the first image and the second image,
The gradation converting unit includes a pseudo image generation unit that generates the pseudo second image by performing a density conversion process on the first image so as to substantially match a density histogram of the second image. The endoscope system according to claim 1. The histogram analysis unit analyzes the density histogram for each of a plurality of areas of the first and second images,
The pseudo-image generating unit is configured so that the concentration histograms among the areas of the first image and the second image substantially coincide, characterized in that the dark light conversion processing on respective areas of the first image The endoscope system according to claim 2. Wherein with concentrated light conversion process, the endoscope system according to claim 2 or 3, wherein said tone curve generated by the comparison operation of the density histogram of the first image and the second image is used.   The said illumination means consists of the 1st semiconductor light source which light-emits the said 1st illumination light, and the 2nd semiconductor light source which light-emits the said 2nd illumination light, The any one of Claim 1 thru | or 4 characterized by the above-mentioned. Endoscope system.   The illumination means wavelength-separates broadband light from the light source and a light source that emits broadband light including light in the wavelength range of the first illumination light and light in the wavelength range of the second illumination light. The endoscope system according to any one of claims 1 to 4, further comprising: a wavelength separation unit that generates the first illumination light and the second illumination light. The illumination means includes a broadband broadband light including a first blue narrowband light whose wavelength range is limited to a specific blue region as the first illumination light, and a wavelength region as a specific blue region as the second illumination light. Irradiate the limited second blue narrowband light sequentially into the subject,
The image acquisition means acquires a plurality of color images including the first blue image by capturing the reflected image of the first illumination light with a color image sensor, and also converts the reflected image of the second illumination light into a color image. By acquiring an image of a plurality of colors including the second blue image by capturing with an image sensor,
The feature amount calculating means obtains feature amounts of the first and second blue images,
The gradation converting means performs gradation conversion on the first blue image based on the feature amount of the first and second blue images, so that there is no positional deviation from the first blue image and the second blue image. Generating a pseudo second blue image having substantially the same feature quantity as the blue image;
The moving image display means displays a moving image of an oxygen saturation image obtained by imaging oxygen saturation of the blood hemoglobin based on the first blue image and the pseudo second blue image. The endoscope system according to any one of Items 1 to 6. The first illumination light is white light including the first blue narrowband light whose wavelength range is limited to 440 ± 10 nm and fluorescence obtained by wavelength conversion of the first blue narrowband light with a wavelength conversion member,
The endoscope system according to claim 7, wherein the second blue narrow-band light is light whose wavelength range is limited to 470 ± 10 nm. The illumination means sequentially irradiates a subject with third illumination light including third blue narrowband light whose wavelength range is limited to a specific blue region in addition to the first and second illumination light,
The image acquisition means acquires a plurality of color images including a third blue image by capturing a reflected image of the third illumination light with a color image sensor in addition to the first and second blue images,
The feature amount calculating means obtains a feature amount of the third blue image in addition to the first and second blue images,
The gradation converting means performs gradation conversion on the first blue image based on the feature amount of the first and third blue images in addition to the pseudo second blue image, thereby positioning the first blue image and the position of the first blue image. Generating a pseudo third blue image having no deviation and having substantially the same feature quantity as the third blue image;
The moving image display means generates an oxygen saturation image obtained by imaging oxygen saturation of blood hemoglobin based on the pseudo third blue image in addition to the first blue image and the pseudo second blue image. The endoscope system according to claim 7 or 8, wherein a moving image is displayed.   The endoscope system according to claim 9, wherein the third blue narrowband light is light having a wavelength range limited to 440 ± 10 nm.   The endoscope system according to claim 9, wherein the third blue narrowband light is light having a wavelength range limited to 400 ± 10 nm. The illumination means includes a broadband broadband light including a first blue narrowband light whose wavelength range is limited to a specific blue region as the first illumination light, and a wavelength region as a specific blue region as the second illumination light. A wide-band broadband light including the limited second blue narrow-band light is sequentially irradiated into the subject,
The image acquisition means acquires a first blue image, a first green image, and a first red image by capturing a reflected image of the first illumination light with a color image sensor, and A plurality of color images including the second blue image are obtained by capturing the reflected image with a color image sensor,
The feature amount calculating means obtains feature amounts of the first and second blue images,
The gradation converting means performs gradation conversion on the first blue image based on the feature amount of the first and second blue images, so that there is no positional deviation from the first blue image and the second blue image. Generating a pseudo second blue image having substantially the same feature quantity as the blue image;
The moving image display means displays a moving image of an oxygen saturation image obtained by imaging oxygen saturation of blood hemoglobin based on the first green image, the first red image, and the pseudo second blue image. The endoscope system according to any one of claims 1 to 6 , wherein the endoscope system is characterized in that: The first illumination light is first white light including the first blue narrowband light whose wavelength range is limited to 440 ± 10 nm and fluorescence obtained by wavelength-converting the first blue narrowband light with a wavelength conversion member. ,
The second illumination light is second white light including the second blue narrowband light whose wavelength range is limited to 470 ± 10 nm and fluorescence obtained by wavelength-converting the second blue narrowband light with a wavelength conversion member. The endoscope system according to claim 12. A first image obtained by imaging a subject image irradiated with the first illumination light, and a subject image irradiated with second illumination light different from the first illumination light, in a frame different from the first image. In an image processing apparatus that processes a plurality of images including a second image obtained by imaging,
Feature quantity calculating means for obtaining the feature quantities of the first and second images;
Gradation conversion means for generating a pseudo second image corresponding to a second image in which positional deviation between frames is corrected by performing gradation conversion on the first image based on the feature amounts of the first and second images. When,
An image comprising: a display control means for displaying an oxygen saturation image obtained by imaging oxygen saturation of blood hemoglobin based on the first image and the pseudo second image on a display means as a moving image. Processing equipment. A first image obtained by imaging a subject image irradiated with the first illumination light, and a subject image irradiated with second illumination light different from the first illumination light, in a frame different from the first image. In an operation method of an image processing apparatus for processing a plurality of images including a second image obtained by imaging,
A step of calculating a feature amount of the first and second images by a feature amount calculating unit ;
The gradation converting means performs gradation conversion on the first image based on the feature amounts of the first and second images, so that there is no positional deviation from the first image and is substantially the same as the second image. Generating a pseudo second image having a feature amount of:
Display control means for performing display control for displaying, on the display means, an oxygen saturation image obtained by imaging oxygen saturation of blood hemoglobin based on the first image and the pseudo second image. A method for operating an image processing apparatus, comprising : A first image obtained by imaging a subject image irradiated with the first illumination light, and a subject image irradiated with second illumination light different from the first illumination light, in a frame different from the first image. In an image processing program for processing a plurality of images including a second image obtained by imaging,
Computer
Feature amount calculating means for determining the feature amounts of the first and second images;
By performing tone conversion on the first image based on the feature amounts of the first and second images, the pseudo image having no feature deviation from the first image and having approximately the same feature amount as the second image. A gradation converting means for generating a second image; and a display means for displaying an oxygen saturation image obtained by imaging oxygen saturation of blood hemoglobin based on the first image and the pseudo second image on a display means. An image processing program for functioning as display control means.

Images