JP2010099172A - Endoscopic system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate display images which do not make a user have the feeling of incompatibility in the color of the display images so much even when using a light source different from a light source normally used in an endoscopic system. <P>SOLUTION: An observation object is irradiated with white light L1 emitted from a white LED 12; an image is captured by an imaging element 22; RGB image signals are acquired and outputted to an estimated spectroscopic data calculation part 33. In the estimated spectral data calculation part 33, a matrix operation is performed using the RGB image signals and estimated spectral matrix data stored in a storage part 41 and estimated spectral data (q1-a61) are calculated. In a pseudo normal image generation part 36, the matrix operation is performed using the estimated spectral data and pseudo correction system matrix data stored in the storage part 41 and pseudo 3-color image signals (R', G', B') are generated. In a display signal generation part 36, pseudo normal image display signals are generated from the pseudo RGB image signals and are outputted to a display device 2. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image acquisition method and apparatus for capturing an image of an observation target with an imaging element by irradiating the observation target with light and acquiring an image signal output from the imaging element.

  Conventionally, endoscope apparatuses for observing tissue in a body cavity are widely known. A normal image is obtained by imaging an observation target in a body cavity illuminated by white light, and the normal image is displayed on a monitor screen. Electronic endoscope devices have been widely put into practical use.

  In such an endoscope apparatus, a xenon lamp is usually used as a light source for emitting white light. On the other hand, in recent years, development of LEDs has been promoted and is being put into practical use as a light source for endoscope apparatuses.

Patent Document 1 describes an endoscope system that obtains an image by forming white light using an LED that emits R light, an LED that emits B light, and an LED that emits R light. Patent Document 2 discloses a capsule-type endoscope system having a light source including an LED that emits white light and an LED that emits a plurality of narrowband lights. In this endoscope system, acquisition of a normal color image and acquisition of a narrowband image are performed in a simultaneous manner by an image sensor provided with an RGB mosaic filter.
JP 2007-29555 A JP 2006-68488 A

  Conventionally, a xenon lamp is often used as a light source of an endoscope system, and users are accustomed to color images acquired by irradiating light emitted from the xenon lamp. FIG. 1 shows the spectral radiation characteristics of light emitted from the xenon lamp. On the other hand, the spectral emission characteristics of white LEDs vary depending on the type of LED. For example, FIG. 2 shows the spectral radiation characteristics of light emitted from a white LED generated by combining a blue LED and a phosphor.

  In this way, when the spectral emission characteristics of the light emitted from the white LED are significantly different from the spectral emission characteristics of the xenon lamp, an image is acquired by irradiating the observation target with the light emitted from the white LED. The display image becomes a bluish display image, and a user who is used to the display image acquired using the light emitted from the xenon lamp may have a sense of incongruity in the color of the display image.

  The present invention has been made in view of the above problems, and even when a light source that is different from the light source that is normally used is used, the display in which the user does not feel discomfort in the color of the display image is small. An object of the present invention is to provide an endoscope system capable of generating an image.

An endoscope system of the present invention includes a light source unit that emits illumination light to be irradiated on an observation target,
An imaging unit including an imaging element that captures an image of the observation target irradiated with the illumination light;
In an endoscope system including a processor unit that generates a display image based on an image signal output from the imaging element,
A first storage unit that stores an estimated spectral matrix based on a spectral radiation characteristic of the illumination light and a spectral characteristic of the imaging means;
A second storage unit for storing a pseudo system matrix based on a spectral radiation characteristic of pseudo illumination light having a spectral radiation characteristic different from that of the illumination light and a spectral characteristic of the imaging unit;
Estimated spectral data calculation means for calculating estimated spectral matrix data using the image signal acquired by the image sensor and the estimated spectral matrix stored in the first storage unit;
A pseudo-normal image generating unit that generates a pseudo-normal image as a display image using the estimated spectral data calculated by the estimated spectral data calculating unit and the pseudo system matrix stored in the second storage unit; It is characterized by this.

  Here, the “illumination light” may be white light or light in a desired wavelength band. Moreover, as long as it is white light, it may be light emitted from a single light source such as a xenon lamp, or light in which light emitted from light sources of three colors of R light, G light, and B light is superimposed. It may be. In the case of light composed of light of three colors of R light, G light, and B light, the light of three colors of R light, G light, and B light may be irradiated simultaneously, Or you may irradiate sequentially. Further, the imaging unit and the processor unit may be directly electrically connected, or may be connected wirelessly or the like.

  The light source unit and the first storage unit may be provided in the imaging unit. Furthermore, the light source unit may be made of an LED.

  According to the endoscope system of the present invention, the estimated spectral matrix data is calculated using the image signal acquired by the image sensor and the stored estimated spectral matrix, and the estimated spectral data, illumination light, and Uses a pseudo-system matrix based on the spectral radiation characteristics of pseudo-illumination light with different spectral radiation characteristics and the spectral characteristics of the imaging means to generate a pseudo-normal image as a display image. It is possible to display a display image based on an image signal obtained when irradiated with simulated illumination light, and even when a light source different from a normally used light source is used, It is possible to display a display image captured using the display image and a recent display image, and to prevent the user from feeling uncomfortable with the color of the display image.

  Hereinafter, an endoscope system using an embodiment of an image acquisition device of the present invention will be described in detail with reference to the drawings. FIG. 3 shows a schematic configuration of an endoscope system 5 using the embodiment of the present invention. This endoscope system performs white light irradiation and narrow band light irradiation in a time-sharing manner, and displays a normal image captured by white light irradiation and a narrow band image captured by narrow band light arranged side by side. Is displayed.

  As shown in FIG. 3, the endoscope system 1 is inserted into a body cavity of a subject and includes a scope unit 10 for observing an observation target, and a processor unit 30 to which the scope unit 10 is detachably connected. It is configured.

  The scope unit 10 includes an illumination optical system 11, LEDs 12 to 14, an LED drive unit 16 that drives the LEDs 12 to 14, an imaging optical system 21, an image sensor 22, an image sensor drive unit 25 that drives the image sensor 22, A CDS / AGC circuit 23 that processes signals output from the image sensor 22, an A / D conversion unit 24 that performs A / D conversion on the signals output from the CDS / AGC circuit 23, and a scope controller that controls the operation of each unit 26.

  The LED 12 is an LED that emits white light L1 for normal image capturing, and the LEDs 13 and 14 are LEDs that emit narrow-band light for narrow-band image capturing. FIG. 2 shows the emission spectrum of the white LED 12. FIG. 2 shows emission spectra of the narrow band LEDs 13 and 14. As shown in FIG. 2, the LED 13 emits L2 narrow band light having a center wavelength of 415 nm and a half value width of 20 to 40 nm. Emits narrow-band light L3 having a center wavelength of 540 nm and a half-value width of 20 to 40 nm.

  The imaging element 22 is made of, for example, a CCD or a CMOS, and obtains image information by photoelectrically converting the observation target image formed by the imaging optical system 21. The image sensor 22 is provided with a color filter 27 composed of minute RGB filters on the front surface of the image pickup surface. The image signal acquired by the image sensor 22 is sampled and amplified by a CDS / AGC (correlated double sampling / automatic gain control) circuit 23, A / D converted by an A / D converter 24, and then sent to the processor unit 30. Is output.

  The processor unit 30 includes a processor unit 31 that generates an image for display from the image signal input from the scope unit 10; a storage unit 41 that is connected to the processor unit 31 and stores an estimated spectral matrix and a pseudo system matrix; And an input unit 42 including a keyboard or a touch panel. Details of the estimated spectral matrix and the pseudo system matrix will be described later.

  The processor unit 31 uses an RGB signal acquisition unit 32 that acquires an R component, a G component, and a B component three-color image signal from the image signal input from the scope unit 10, and the RGB image signal and the estimated spectral matrix data. An estimated spectral data calculation unit 33 that calculates estimated spectral data, a pseudo normal image generation unit 34 that generates a pseudo normal image signal using the estimated spectral data and pseudo system matrix data, narrowband light L2, and narrowband light A narrowband image generation unit 35 that generates a narrowband image signal based on the RGB image signal acquired by the irradiation of the light L3, a corrected normal image signal generated by the correction normal image generation unit 34, and a narrowband image generation unit And a display signal generation unit 36 that performs various processes on the narrowband image signal acquired in 35 to generate a display image signal.

The storage unit 41 stores estimated spectral matrix data for calculating estimated spectral data of an observation target. The estimated spectral matrix data is stored in advance in the storage unit 32 as a table. This estimated spectral matrix data takes into account the spectral emissivity of the white light L1 irradiated to the observation target, and the spectral characteristics of the entire imaging system including the spectral sensitivity characteristics of the imaging element 22, the wavelength transmission characteristics of the color filter 27, and the like. Matrix data, and an RGB image signal picked up by the image pickup device 22 and the estimated spectral matrix data, which are not dependent on the spectral radiation characteristics of the white light L1, the specific spectral characteristics of the imaging system, etc. Spectroscopic data can be obtained. The details of the estimated spectral matrix data are disclosed in Japanese Patent Application Laid-Open No. 2003-93336 or Japanese Patent Application Laid-Open No. 2007-202621. In the present embodiment, an example of estimated spectral matrix data stored in the storage unit 32 is as shown in Table 1 below.

  The matrix data in Table 1 is composed of, for example, 61 wavelength range parameters (coefficient sets) p1 to p61 obtained by dividing a wavelength range of 400 nm to 700 nm at 5 nm intervals.

  The storage unit 41 stores pseudo system matrix data for calculating a pseudo normal image signal from the estimated spectral data of the observation target. The pseudo system matrix data is stored in advance in the storage unit 32 as a table.

  First, normal system matrix data will be described, and then pseudo system matrix data will be described. The system matrix is matrix data that takes into account the spectral emissivity of the white light L1 irradiated to the observation target, and the spectral characteristics of the entire imaging system including the spectral sensitivity characteristics of the imaging element 22, the wavelength transmission characteristics of the color filter 27, and the like. In addition, by calculating the spectral reflectance of the observation target and the system matrix, it is possible to obtain a three-color image signal of the observation target that reflects the spectral radiation characteristics of the white light L1, the spectral characteristics unique to the imaging system, and the like. Note that the system matrix and the above-described estimated spectral matrix have a generalized inverse matrix relationship.

On the other hand, the pseudo system matrix is a pseudo light source used when generating a pseudo normal image, for example, a spectral emissivity of light emitted from a xenon lamp, a spectral sensitivity characteristic of the image sensor 22, a wavelength transmission characteristic of the color filter 27, and the like. System matrix data taking into account the spectral characteristics of the entire imaging system including By calculating the spectral reflectance of the observation target and this pseudo system matrix, a three-color image signal of the observation target reflecting the spectral emission characteristics of the light emitted from the xenon lamp, the spectral characteristics specific to the imaging system, and the like is obtained. Can do. That is, the image signal obtained when the observation target is irradiated with the light emitted from the xenon lamp instead of the white light L1 can be calculated. In the present embodiment, an example of the pseudo system matrix data stored in the storage unit 41 is as shown in Table 2 below.

  The matrix data in Table 2 includes three image signal parameters (coefficient sets) pr, pg, and pb corresponding to the R image signal, the G image signal, and the B image signal.

  The display device 2 is configured by a liquid crystal display device, a CRT, or the like, and displays a pseudo normal image and a narrowband image based on a display image signal output from the processor unit 30.

  Next, the operation of the endoscope system of this embodiment will be described. First, after the insertion portion of the scope unit 10 is inserted into the body cavity, the LED driving unit 16 sets the white LED 12 and the narrow band LEDs 13 and 14 to 1 based on a control signal from the processor unit 31 of the processor unit 30. / Alternate drive at 60 second intervals.

  When the white LED 12 or the narrow band LEDs 13 and 14 are driven, the white light L1 emitted from the white LED 12 or the narrow band lights L2 and L3 emitted from the narrow band LEDs 13 and 14 pass through the illumination optical system 11. To be irradiated. The image sensor 22 driven by the image sensor driving unit 25 captures an image to be observed and outputs an image signal. This image signal is subjected to correlated double sampling and amplification by automatic gain control in the CDS / AGC circuit 23, and then A / D converted in the A / D conversion unit 24, so that the RGB signal acquisition unit of the processor unit 30 is converted into a digital signal. 32.

  The RGB signal acquisition unit 32 generates an RGB image signal composed of an R component, a G component, and a B component from the image signal output from the image sensor 22. The RGB image signal generated by the irradiation of the white light L1 is output to the estimated spectral data calculation unit 33, and the RGB image signal generated by the irradiation of the narrowband light L2 and L3 is the narrowband image signal generation unit 35. Is output. The narrowband light L2 corresponds to the B image signal because it is light having a central wavelength of 415 nm, and the narrowband light L3 corresponds to a G image signal because it is light having a central wavelength of 540 nm. The R image signal is output as a zero value because red light is not irradiated on the observation target.

When the RGB image signal by the irradiation of the white light L1 is input from the RGB signal acquisition unit 32, the estimated spectral data calculation unit 33 outputs the RGB image signal by the white light L1 output from the RGB signal acquisition unit 32 for each pixel. Then, using the estimated spectral matrix data stored in the storage unit 41, matrix calculation represented by the following equation is performed to generate estimated spectral data (q1 to q61), which are output to the pseudo-normal image generation unit 34.

The pseudo normal image generation unit 34 uses the estimated spectral data (q1 to q61) and the pseudo correction system matrix data (pr, pg, pb) stored in the storage unit 41 to perform a matrix calculation represented by the following equation. To generate a corrected three-color image signal (R ′, G ′, B ′), appropriately perform various signal processing, and output the signal to the display signal generator 36 as a pseudo RGB image signal.

  Thereafter, RGB image signals obtained by irradiation with the narrowband light L2 and the narrowband light L3 are input from the RGB signal acquisition unit 32. As described above, the narrowband light L2 is input as a B image signal because it is a light having a center wavelength of 415 nm, and the narrowband light L3 is input as a G image signal because it is a light having a center wavelength of 540 nm. The R image signal is not input because red light is not irradiated on the observation target.

  The narrowband image generation unit 35 assigns the input B image signal to the display B image signal and the R image signal, and assigns the input G image signal to the display G image signal. The display RGB image signal is appropriately subjected to various types of signal processing, and then output to the display signal generation unit 36 as a narrowband RGB image signal.

  The display signal generation unit 36 generates a Y / C signal composed of the luminance signal Y and the color difference signal C from the pseudo RGB image signal input from the pseudo normal image generation unit 34, and further, for this Y / C signal. Various signal processing such as I / P conversion and noise removal is performed to generate a pseudo normal image display signal. Further, a Y / C signal composed of a luminance signal Y and a color difference signal C is generated from the narrowband RGB image signal input from the narrowband image generation unit 35, and further, the I / P Various signal processing such as conversion and noise removal is performed to generate a narrowband image display signal. The narrow-band image display signal and the pseudo normal image display signal are combined to generate a display signal and output it to the display device 2. The display device 2 displays moving images that are updated at 1/30 second intervals.

  The display device 2 displays a pseudo normal image and a narrow band image based on the input display signal. The pseudo-normal image is obtained by calculating and displaying an image signal obtained when the observation target is irradiated with the light emitted from the xenon lamp instead of the white light L1.

  As is clear from the above description, in the present embodiment, the display image based on the image signal obtained when the observation target is irradiated with the light emitted from the xenon lamp instead of the white light L1 is displayed. Even when a light source different from a normally used xenon lamp is used, the user can be prevented from feeling uncomfortable with the color of the display image.

  Next, a second embodiment of the present invention will be described with reference to FIG. FIG. 4 shows a schematic configuration diagram of an endoscope system 6 according to the second embodiment of the present invention. In addition, about the site | part same as the endoscope system 5 which is 1st Embodiment shown in FIG. 1, the same code | symbol is provided and description is abbreviate | omitted.

  As shown in FIG. 4, an endoscope system 6 using an embodiment of the present invention is inserted into a body cavity of a subject, and a scope unit 50 for observing an observation target, and the scope unit 50 are detachable. It is comprised from the processor unit 51 connected.

  In the endoscope system 6, only the pseudo system matrix is stored in the storage unit 52 provided in the processor unit 51, and the estimated spectral matrix is stored in the storage unit 53 provided in the scope unit 50. However, this is different from the endoscope system 5 shown in FIG.

  When the RGB image signal by the irradiation of the white light L1 is input from the RGB signal acquisition unit 32, the estimated spectral data calculation unit 33 outputs the RGB image signal by the white light L1 output from the RGB signal acquisition unit 32 for each pixel. Estimated spectral data (q1 to q61) is created using the estimated spectral matrix data read from the storage unit 53 via the scope controller 26, and is output to the pseudo-normal image generating unit 34. The estimated spectral matrix data stored in the storage unit 53 of the scope unit 50 reflects the spectral emissivity of the white LED 12 provided in the scope unit 50.

  Generally, the processor unit and the scope unit are configured to be detachable. Further, the scope unit needs to be cleaned every time it is used. Therefore, a plurality of scope units are sequentially connected to one processor unit. On the other hand, the spectral emissivity of white LEDs varies greatly from one individual to another. For this reason, when the estimated spectral matrix reflecting the spectral emissivity of the white LED 12 is stored in the storage unit provided in the processor unit, the estimated spectral matrix is used for the white LED 12 mounted in each scope unit. It is difficult to say that the spectral reflectance of each white LED 12 is accurately reflected. On the other hand, in the present embodiment, when the scope unit 50 is manufactured, an estimated spectral matrix corresponding to the white LED 12 mounted on the scope unit 50 is obtained and stored in the storage unit 53. Therefore, the estimated spectral matrix stored in the storage unit 53 is a matrix that more accurately reflects the spectral emissivity of the white LED 12.

  The pseudo normal image generation unit 34 performs a matrix calculation using the estimated spectral data (q1 to q61) and the pseudo correction system matrix data stored in the storage unit 52, thereby correcting the three-color image signal (R ′). , G ′, B ′) are generated, subjected to various signal processing as appropriate, and then output to the display signal generation unit 36 as a pseudo RGB image signal.

  The display signal generation unit 36 generates a pseudo normal image display signal from the pseudo RGB image signal, generates a narrow band image display signal from the narrow band RGB image signal input from the narrow band image generation unit 35, and The image display signal and the pseudo normal image display signal are combined to generate a display signal, which is output to the display device 2.

  As is clear from the above description, in the endoscope system according to the present embodiment, the same effects as those of the first embodiment can be obtained, and the estimated spectral matrix more accurately corrects the spectral reflectance of the white LED 12. Since it is reflected, more accurate estimated spectroscopic data (q1 to q61) can be obtained, and a pseudo-normal image having a higher reliability and closer to an actual normal image can be displayed.

  In the first embodiment and the second embodiment, the white LED 12 is used as the light source for the white light L1, but the present invention is not limited to this, and the LED that emits R light and the G light are used. You may use three light sources of LED which inject | emits and LED which inject | emits B light. Or LED which inject | emits R light, LED13, and LED14 can also be used as a light source for white light.

  Next, a third embodiment of the present invention will be described with reference to FIG. FIG. 5 shows a schematic configuration diagram of an endoscope system 7 according to the fifth embodiment of the present invention. In addition, about the site | part same as the endoscope system 5 which is 1st Embodiment shown in FIG. 1, the same code | symbol is provided and description is abbreviate | omitted.

  As shown in FIG. 5, the endoscope system 7 is inserted into a body cavity of a subject and includes a scope unit 60 for observing an observation target, and a processor unit 61 to which the scope unit 60 is detachably connected. It is configured.

  The scope unit 60 is provided with an LED 62 as a light source. The LED 62 emits light L4 having a spectral emissivity as shown in FIG. The light L4 has emissivity peaks at 430 nm and 540 nm. That is, it is an LED that simultaneously emits white light for normal image capturing and narrow band light for narrow-band image capturing.

  The processor unit 61 includes a processor unit 63 that generates an image for display from the image signal input from the scope unit 60, a storage unit 41 that is connected to the processor unit 63 and stores an estimated spectral matrix and a pseudo system matrix, And an input unit 42.

  The processor unit 63 replaces the RGB signal acquisition unit 32 of the processor unit 31 of the endoscope system 5 according to the first embodiment shown in FIG. 3 with the R component and the G component from the image signal output from the image sensor 22. , An RGB image acquisition unit 64 that generates an RGB image signal composed of the B component and outputs it to the estimated spectral data calculation unit 33 and the narrowband image signal generation unit 35. Other configurations are the same as the configuration of the processor unit 31 of the endoscope system 5.

  After the insertion portion of the scope unit 60 is inserted into the body cavity, the LED drive unit 16 drives the LED 62 based on a control signal from the processor unit 63 of the processor unit 61. The light L4 emitted from the LED 62 is applied to the observation target via the illumination optical system 11. The image sensor 22 driven by the image sensor driving unit 25 captures an image to be observed and outputs an image signal. The image signal is subjected to correlated double sampling and amplification by automatic gain control in the CDS / AGC circuit 23, and then A / D converted in the A / D conversion unit 24, so that the RGB signal acquisition unit of the processor unit 63 is converted into a digital signal. 64.

  The RGB signal acquisition unit 64 generates an RGB image signal composed of an R component, a G component, and a B component from the image signal output from the image sensor 22, and outputs the RGB image signal to the estimated spectral data calculation unit 33 and the narrowband image signal generation unit 35. To do.

  Hereinafter, similarly to the processor unit 31 of the endoscope system 5 according to the first embodiment shown in FIG. 3, estimated spectral data (q1 to q61) are created by the estimated spectral data calculation part 33, and the pseudo normal image generation unit A pseudo RGB image signal is calculated at 34 and output to the display signal generator 36. Note that the estimated system matrix stored in the storage unit 41 reflects the spectral reflectance of the light L4 emitted from the LED 62.

  As is clear from the above description, in the endoscope system of the present embodiment, an effect similar to that of the first embodiment can be obtained, and a simple light source composed of only one LED is used. A narrow-band image and a pseudo-normal image can be displayed.

  In the endoscope system of the above embodiment, the primary color type image pickup device having RGB color filters is used as the image pickup device of the scope unit. However, the present invention is not limited to this. For example, CMY (cyan, magenta, yellow) A complementary color type image pickup device having a color filter may be used.

  Further, the present invention is not limited to the endoscope apparatus including the scope unit as in the present embodiment, but can be applied to a laparoscope, a colposcope, a capsule endoscope, or the like.

Diagram showing the spectral emissivity of light emitted from a xenon lamp The figure which shows the spectral emissivity of the light inject | emitted from white LED The block diagram which shows schematic structure of the endoscope system which is the 1st Embodiment of this invention. The block diagram which shows schematic structure of the endoscope system which is the 2nd Embodiment of this invention. The block diagram which shows schematic structure of the endoscope system which is the 3rd Embodiment of this invention. The figure which shows the spectral emissivity of the light inject | emitted from LED

Explanation of symbols

5, 6, 7 Endoscope system 2 Display device 10, 50, 60 Scope unit 11 Optical system for illumination 12, 13, 14, 62 LED
16 LED drive unit 21 Imaging optical system 22 Image sensor 23 CDS / AGC circuit 24 A / D conversion unit 25 Image sensor drive unit 26 Scope controller 27 Color filters 30, 51, 61 Processor units 31, 63 Processor units 32, 64 RGB Signal acquisition unit 33 Estimated spectral data calculation unit 34 Pseudo normal image generation unit 35 Narrow band image generation unit 36 Display signal generation unit 41 Storage unit 42 Input unit

Claims (3)

A light source unit that emits illumination light to irradiate the observation target;
An imaging unit including an imaging element that captures an image of the observation target irradiated with the illumination light;
In an endoscope system including a processor unit that generates a display image based on an image signal output from the imaging element,
A first storage unit that stores an estimated spectral matrix based on a spectral radiation characteristic of the illumination light and a spectral characteristic of the imaging means;
A second storage unit for storing a pseudo system matrix based on a spectral radiation characteristic of pseudo illumination light having a spectral radiation characteristic different from that of the illumination light and a spectral characteristic of the imaging unit;
Estimated spectral data calculation means for calculating estimated spectral matrix data using the image signal acquired by the image sensor and the estimated spectral matrix stored in the first storage unit;
Pseudo normal image generation for generating a pseudo normal image as a normal image for display, using the estimated spectral data calculated by the estimated spectral data calculation means and the pseudo system matrix stored in the second storage unit And an endoscope system.   The endoscope system according to claim 1, wherein the light source unit and the first storage unit are provided in the imaging unit.   The endoscope system according to claim 2, wherein the light source unit includes an LED.

Images