JP6584147B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus.

  An endoscopic device that calculates the concentration distribution (for example, oxygen saturation or blood volume distribution) of biological substances in biological tissue from imaging data obtained by imaging a biological tissue using a color imaging device and displays it on a monitor is proposed. (Patent Document 1).

  In the endoscope apparatus of Patent Document 1, a biological substance is quantitatively analyzed based on two pieces of imaging data of a biological tissue imaged while switching between two types of illumination light (optical filters) having different wavelength characteristics.

  In order to quantify biological substances with high accuracy, the input range of an ADC (Analogue-to-Digital Converter) is set as narrow as possible at the time of digital conversion of imaging data, and high-gradation digital imaging data is acquired. There is a need.

  Patent Document 2 describes an imaging device that can change the input range of an ADC.

International Publication No. 2014/192781 JP 2001-346106 A

  However, if the ADC input range is changed, the absolute value information included in the imaging data is lost, and the accuracy of quantification is reduced.

  The present invention has been made in view of the above circumstances, and an object thereof is to improve accuracy without reducing the accuracy of spectroscopic analysis based on color imaging data.

  According to an embodiment of the present invention, the image pickup device includes an image pickup device and a processing unit that processes an image pickup signal generated by the image pickup device, and the image pickup device includes a plurality of R pixels sensitive to red light, A plurality of first and second G pixels each sensitive to light and a plurality of B pixels sensitive to blue light, wherein the processing means is generated by the plurality of first G pixels A first initial processing unit that processes the analog imaging signal G1 to generate the digital imaging signal G1, and a second initial processing unit that processes the analog imaging signal G2 generated by the plurality of second G pixels to generate the digital imaging signal G2. Initial processing means, wherein the first initial processing means converts the analog imaging signal G1 into a digital signal G1_dif, and the input level of the first digital conversion means based on the digital imaging signal G2. An input range setting means for generating a di settings, including a imaging device is provided.

  In the imaging apparatus, the input range setting unit may set the input range of the first digital conversion unit in accordance with the fluctuation range of the signal level of the analog imaging signal G2.

  In the imaging apparatus, the input range setting means calculates the statistical value of the digital imaging signal G2, and the input offset amount OF that determines the lower limit value of the input range of the first digital conversion means based on the statistical value. The input offset amount calculation means for calculating the input span and the input span calculation means for calculating the input span SP for determining the width of the input range of the first digital conversion means based on the statistical value may be provided.

  In the imaging apparatus, the statistical processing unit calculates the average value AVG and the standard deviation SD of the digital imaging signal G2, the input offset amount calculation unit calculates the input offset amount OF based on the average value AVG, and the input span. The calculation means may be configured to calculate the input span SP based on the standard deviation SD.

In the above imaging apparatus, the input offset amount calculation unit may calculate the input offset amount OF using Equation 1.
(However, K is a constant.)

In the imaging apparatus described above, the input span calculation unit may calculate the input span SP using Formula 2.
(However, K is a constant.)

  In the imaging apparatus, the processing unit generates a digital imaging signal G1 including absolute value information based on the digital signal G1_dif generated by the first initial processing unit, the input offset amount OF, and the input span SP. It is good also as a structure further provided with a restoring means.

In the above-described imaging apparatus, the restoration unit may be configured to calculate the digital imaging signal G1 using Equation 3.
(However, FSP is a predetermined value.)

  In the above imaging apparatus, the second initial processing means includes second digital conversion means for converting the analog imaging signal G2 into the digital imaging signal G2, and the FSP has an input span that determines the width of the input range of the second digital conversion means. It is good also as composition which is.

  In the imaging apparatus, the first G pixel is a Gr pixel arranged on a line where R pixels are arranged, and a second G pixel is a Gb arranged on a line where B pixels are arranged. It is good also as a structure which is a pixel.

In the imaging apparatus, the processing unit further includes an index calculation unit that calculates an index X indicating a molar concentration ratio of the first and second biological substances included in the biological tissue that is the subject based on the digital imaging signal G1. The imaging device further includes a light source device that generates light by switching between light in the first illumination wavelength range in which the first and second biological materials have absorption and light in the second illumination wavelength range within the first illumination wavelength range. It includes, index calculation means, a first digital image signal G _I obtained by imaging the living body tissue illuminated with light of the first illumination wavelength range, an imaging body tissue under illumination of the second illumination wavelength range of the light a second digital imaging signal G _II obtained by, may be configured to calculate the index X based on.

  In the imaging apparatus, the first illumination wavelength region includes the absorption peak wavelengths of both the first and second biological materials, and the second illumination wavelength region includes the absorption peak wavelength of one of the first and second biological materials. It is good also as a structure including.

  In the imaging apparatus, the first illumination wavelength region is adjacent to the short wavelength side of the second illumination wavelength region, and includes a wavelength region including an absorption peak wavelength of the other biological material of the first and second biological materials, It is good also as a structure containing the wavelength range which adjoins the long wavelength side of an illumination wavelength range, and contains the absorption peak wavelength of the other biological material.

  In the imaging apparatus, the light source device includes a light source that generates broadband light, a first optical filter that selectively extracts light in the first illumination wavelength region from the broadband light, and light in the second illumination wavelength region from the broadband light. It is good also as a structure provided with the 2nd optical filter taken out selectively.

In the imaging apparatus, the index calculating means calculates the biological tissue absorption A _I in the first illumination wavelength range based on the first imaging signal G _I , and the second illumination wavelength range based on the second imaging signal G _II. Get absorption a _II of the biological tissue in and may be configured to calculate an index X on the basis of absorption a _I and absorption a _II.

In the imaging apparatus described above, the index calculation unit may calculate the absorption A _{I according} to any one of Equations 4 and 5, and may calculate the absorption A _{II according} to any one of Equations 6 and 7.

In the imaging apparatus described above, the index calculation unit may calculate the index X according to any one of Formula 8 and Formula 9.
(However, k is a constant.)

  According to an embodiment of the present invention, it is possible to improve the accuracy and accuracy of spectroscopic analysis based on color imaging data.

It is an absorption spectrum of hemoglobin in the Q band. It is a block diagram of an endoscope apparatus. It is a block diagram of a solid-state image sensor. It is a transmission spectrum of the color filter incorporated in an image sensor. It is an external view of a rotation filter. It is a block diagram of Gb-AFE. It is a block diagram of Gr-AFE. It is a figure explaining the definition of input span SP and input offset amount OF. It is a block diagram of an input range setting part. It is a flowchart explaining the image generation process which concerns on embodiment of this invention. It is an example of a display of the image information produced | generated by the endoscope apparatus which concerns on embodiment of this invention. (A) is a two-dimensional display example of an oxygen saturation distribution image, and (b) is a three-dimensional display example of an oxygen saturation distribution image.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
An endoscope apparatus according to an embodiment of the present invention described below quantitatively calculates biological information (for example, oxygen saturation) of a subject based on a plurality of images captured using illumination light having different wavelength ranges. It is an apparatus that analyzes and displays an analysis result as an image. In the quantitative analysis of oxygen saturation described below, the property that the spectral characteristics of blood (that is, the spectral characteristics of hemoglobin) continuously change according to the oxygen saturation is used.

[Calculation principle of spectral characteristics and oxygen saturation of hemoglobin]
Before describing the configuration of the endoscope apparatus according to the embodiment of the present invention, the spectral characteristics of hemoglobin and the calculation principle of oxygen saturation in the present embodiment will be described.

  FIG. 1 shows an absorption spectrum of hemoglobin near 550 nm. Hemoglobin has a strong absorption band called Q band derived from the porphyrin ring in the vicinity of 550 nm. The absorption spectrum of hemoglobin varies depending on the oxygen saturation (the ratio of oxygenated hemoglobin in the total hemoglobin). The solid line waveform in FIG. 1 is an absorption spectrum when the oxygen saturation is 100% (ie, oxygenated hemoglobin HbO), and the long broken line waveform is when the oxygen saturation is 0% (ie, reduction). It is an absorption spectrum of hemoglobin Hb. The short dashed line is an absorption spectrum of hemoglobin (a mixture of oxygenated hemoglobin and reduced hemoglobin) at an intermediate oxygen saturation (10, 20, 30,... 90%).

  As shown in FIG. 1, in the Q band, oxygenated hemoglobin and reduced hemoglobin have different peak wavelengths. Specifically, oxygenated hemoglobin has an absorption peak P1 near a wavelength of 542 nm and an absorption peak P3 near a wavelength of 576 nm. On the other hand, reduced hemoglobin has an absorption peak P2 near 556 nm. FIG. 1 shows a two-component absorption spectrum in which the sum of the concentrations of each component (oxygenated hemoglobin and reduced hemoglobin) is constant. Therefore, the absorption is constant regardless of the concentration of each component (ie, oxygen saturation). The equiabsorption points E1, E2, E3, and E4 appear. In the following description, the wavelength region sandwiched between the isosbestic points E1 and E2 is sandwiched between the wavelength region R1 and the wavelength region sandwiched between the isosbestic points E2 and E3 is sandwiched between the wavelength region R2 and the isosbestic points E3 and E4. This wavelength region is called a wavelength region R3. A wavelength region sandwiched between the isosbestic points E1 and E4 (that is, a combination of the wavelength regions R1, R2, and R3) is referred to as a wavelength region R0.

  As shown in FIG. 1, absorption increases or decreases monotonously with respect to oxygen saturation between adjacent isosbestic points. Further, between adjacent isosbestic points, the absorption of hemoglobin changes almost linearly with respect to the oxygen saturation.

Specifically, the absorption _{A R1, A R3} of hemoglobin in the wavelength range R1, R3 is linearly and monotonously increases with respect to the concentration of oxygenated hemoglobin (oxygen saturation), absorption _{A R2} of hemoglobin in the wavelength range R2 is It increases monotonically linearly with the concentration of reduced hemoglobin (1-oxygen saturation). Therefore, the index X defined by the following formula 10 increases monotonically linearly with respect to the oxygenated hemoglobin concentration (oxygen saturation).

  Therefore, if a quantitative relationship between the oxygen saturation and the index X is acquired experimentally in advance, the oxygen saturation can be calculated from the value of the index X.

[Configuration of endoscope apparatus]
FIG. 2 is a block diagram of the endoscope apparatus 1 according to the embodiment of the present invention. The endoscope apparatus 1 according to the present embodiment includes an electronic scope 100, a processor 200, and a monitor 500. The electronic scope 100 and the monitor 500 are detachably connected to the processor 200. Further, the processor 200 incorporates a light source unit 400.

  The electronic scope 100 has an insertion portion 101 that is inserted into a body cavity. Inside the electronic scope 100, a light guide 102 extending over the entire length is provided. An emission end of the light guide 102 is disposed in a distal end portion of the insertion unit 101, and an incident end of the light guide 102 is connected to the light source unit 400 of the processor 200.

  The light source unit 400 includes a light source lamp 430 that generates white light WL having a large light amount, such as a xenon lamp. The illumination light IL generated by the light source unit 400 is incident on the incident end of the light guide 102. The light incident on the incident end of the light guide 102 propagates through the light guide 102 to the exit end and is emitted from the exit end. A light distribution lens 104 is disposed at the distal end of the insertion portion of the electronic scope 100 so as to face the emission end of the light guide 102, and the illumination light IL emitted from the emission end of the light guide 102 is a light distribution lens. The living body tissue (for example, the inner lining of the digestive tract) around the distal end portion of the insertion portion 101 is illuminated through 104.

  In addition, an objective optical system 106 and a solid-state imaging device 110 including one or more optical lenses are provided at the distal end portion of the insertion portion 101. A part (return light) of the illumination light IL reflected or scattered on the surface of the living tissue is incident on the objective optical system 106 and is condensed and imaged on the light receiving surface of the solid-state imaging device 110.

  A driver signal processing circuit 120 is provided in the connection portion of the electronic scope 100. The driver signal processing circuit 120 receives a digital imaging signal (digital imaging data, RAW data) from the solid-state imaging device 110 in a field cycle. In the following description, “field” may be replaced with “frame”. In the present embodiment, the field period and the frame period are 1/60 seconds and 1/30 seconds, respectively. The driver signal processing circuit 120 performs predetermined processing on the digital imaging signal input from the solid-state imaging device 110 and outputs the processed signal to the image processing circuit 220 of the processor 200.

  The driver signal processing circuit 120 also accesses the memory 121 and reads the unique information of the electronic scope 100. The unique information of the electronic scope 100 recorded in the memory 121 includes, for example, the number and sensitivity of the solid-state image sensor 110, the operable field rate, the model number, and the like. The driver signal processing circuit 120 outputs the unique information read from the memory 121 to the system controller 202.

  The system controller 202 performs various calculations based on the unique information of the electronic scope 100 and generates a control signal. The system controller 202 uses the generated control signal to control the operation and timing of various circuits in the processor 200 so that processing suitable for the electronic scope 100 connected to the processor 200 is performed.

  The timing controller 204 generates a synchronization signal according to the timing control by the system controller 202. The driver signal processing circuit 120 drives and controls the solid-state imaging device 110 at a timing synchronized with the field rate of the video signal generated by the processor 200 in accordance with the synchronization signal supplied from the timing controller 204.

  The image processing circuit 220 generates image data by demosaic processing or the like based on the digital imaging signal output from the electronic scope 100 under the control of the system controller 202. The image processing circuit 220 generates screen data for monitor display using the generated image data, converts the screen data into a video signal of a predetermined video format, and outputs the video signal. The video signal is input to the monitor 500, and a color image of the subject is displayed on the display screen of the monitor 500.

[Schematic configuration of solid-state image sensor]
FIG. 3 is a block diagram of the solid-state image sensor 110. The solid-state imaging device 110 is a single-plate color CMOS (Complementary Metal Oxide Semiconductor) image sensor provided with RGB primary color filters in a Bayer arrangement.

  The color filter (RGB primary color filter) of the solid-state imaging device 110 includes an R filter that transmits red light, a G filter (Gr filter, Gb filter) that transmits green light, and a B filter that transmits blue light. Is a so-called on-chip filter mounted directly on each light receiving element of the solid-state image sensor 110. Each of the R, G, and B filters has spectral characteristics as shown in FIG. That is, the R filter of this embodiment is a filter that transmits light having a wavelength longer than about 570 nm, the G filter is a filter that transmits light having a wavelength of about 470 nm to 620 nm, and the B filter is about 530 nm. It is a filter that transmits light having a shorter wavelength.

  The G filter is divided into a Gr filter arranged on a line where R filters are arranged and a Gb filter arranged on a line where B filters are arranged.

  The solid-state imaging device 110 accumulates an optical image formed by each pixel on the light receiving surface as a charge corresponding to the amount of light, generates R, Gr, Gb, and B color signals, and scan lines (imaging signals). Output as. The solid-state image sensor 110 is not limited to a CMOS image sensor, and a CCD (Charge Coupled Device) image sensor or other types of imaging devices can also be used. The detailed configuration of the solid-state image sensor 110 will be described later.

[Configuration of light source section]
The light source unit 400 includes a collimator lens 440, a rotation filter 410, a filter control unit 420, and a condenser lens 450 in addition to the light source 430 described above. The white light WL emitted from the light source 430 becomes parallel light by the collimator lens 440, passes through the rotary filter 410, and enters the incident end of the light guide 102 by the condenser lens 450. The rotary filter 410 is movable between an application position on the optical path of the white light WL and a retracted position outside the optical path by a moving means (not shown) such as a linear guide way.

  The rotary filter 410 is a disk-type optical unit including a plurality of optical filters, and is configured such that the pass wavelength range is switched according to the rotation angle. The rotation angle of the rotary filter 410 is controlled by a filter control unit 420 connected to the system controller 202. When the system controller 202 controls the rotation angle of the rotary filter 410 via the filter control unit 420, the spectrum of the illumination light that passes through the rotary filter 410 and is supplied to the light guide 102 is switched.

  FIG. 5 is an external view (front view) of the rotary filter 410. The rotary filter 410 includes a substantially disk-shaped frame 411 and three fan-shaped optical filters 415, 416 and 418. Three fan-shaped windows 414a, 414b and 414c are formed at equal intervals around the central axis of the frame 411, and optical filters 415, 416 and 418 are fitted into the windows 414a, 414b and 414c, respectively. Yes. The optical filters of the present embodiment are all dielectric multilayer filters, but other types of optical filters (for example, absorption optical filters and etalon filters using dielectric multilayer films as reflective films). May be used.

  A boss hole 412 is formed on the central axis of the frame 411. In the boss hole 412, an output shaft of a servo motor (not shown) provided in the filter control unit 420 is inserted and fixed, and the rotary filter 410 rotates together with the output shaft of the servo motor.

  FIG. 5 shows a state where the white light WL is incident on the optical filter 415. However, when the rotary filter 410 rotates in the direction indicated by the arrow, the optical filters on which the white light WL is incident are 415, 416, In this order, the spectrum of the illumination light IL passing through the rotary filter 410 is switched.

  The optical filters 415 and 416 are optical bandpass filters that selectively pass light in the 550 nm band. As shown in FIG. 1, the optical filter 415 allows light in the wavelength region from the isoabsorption points E1 to E4 (that is, the wavelength region R0 (referred to as “first illumination wavelength region”)) to pass through with low loss, It is configured to block light in other wavelength regions. The optical filter 416 allows light in the wavelength region from the isosbestic points E2 to E3 (that is, the wavelength region R2 (referred to as “second illumination wavelength region”)) to pass through with low loss, and in other wavelength regions. It is configured to block light.

  As shown in FIG. 1, the wavelength range R1 includes the peak wavelength of the absorption peak P1 derived from oxygenated hemoglobin, and the wavelength range R2 includes the peak wavelength of the absorption peak P2 derived from reduced hemoglobin. The region R3 includes the peak wavelength of the absorption peak P3 derived from oxygenated hemoglobin. The wavelength band R0 includes the peak wavelengths of the absorption peaks P1, P2, and P3.

  The pass wavelength range (FIG. 1) of the optical filters 415 and 416 is included in the pass wavelength range (FIG. 4) of the G (Gr, Gb) filter of the color filter of the solid-state imaging device 110. Accordingly, an image formed by light that has passed through the optical filter 415 or 416 is captured by a light receiving element on which a Gr or Gb filter is mounted, and is obtained as digital imaging data Gr or Gb.

  The optical filter 418 is an ultraviolet cut filter, and the illumination light IL (that is, white light) that has passed through the optical filter 418 is used for capturing a normal observation image. Note that the optical filter 418 may not be used, and the window 414c of the frame 411 may be opened.

  Further, a darkening filter (ND filter) 419 is attached to the window 414c (FIG. 5) so as to overlap the optical filter 418. The neutral density filter 419 has no wavelength dependence over the entire visible light range, and reduces only the amount of light without changing the spectrum of the illumination light IL. By using the neutral density filter 419, the amount of illumination light IL that has passed through the optical filter 418 and the neutral density filter 419 is adjusted to approximately the same level as the amount of illumination light IL that has passed through the optical filters 415 and 416. As a result, even when the illumination light IL that has passed through any of the optical filters 415, 416, and 418 is used, it is possible to capture an image with appropriate exposure with the same exposure time.

  In the present embodiment, a fine metal mesh is used as the neutral density filter 419. Besides the metal mesh, a neutral density filter such as a half mirror may be used. Further, a neutral density filter may be attached to the windows 414a and 414b.

  A through hole 413 is formed at the peripheral edge of the frame 411. The through-hole 413 is formed at the same position as the boundary between the window 414a and the window 414c in the rotation direction of the frame 411. Around the frame 411, a photo interrupter 422 for detecting the through hole 413 is arranged so as to surround a part of the peripheral edge of the frame 411. The photo interrupter 422 is connected to the filter control unit 420.

  The endoscope apparatus 1 of the present embodiment has two operation modes, a normal observation mode and a spectroscopic analysis mode. The normal observation mode is an operation mode for photographing a color image using white light that has passed through the optical filter 418. In the spectroscopic analysis mode, spectroscopic analysis is performed based on digital imaging data (RAW data) captured using illumination light that has passed through the optical filters 415 and 416, respectively, and a distribution image (for example, oxygen saturation level) of a biomolecule in a living tissue. Distribution image).

  In the normal observation mode, the system controller 202 controls the moving unit to move the rotary filter 410 from the application position to the retracted position. In the operation mode other than the normal observation mode, the rotary filter 410 is disposed at the application position. When the rotary filter 410 has no moving means, the system controller 202 controls the filter control unit 420 to stop the rotary filter 410 at a position where the white light WL is incident on the optical filter 418. Then, the digital imaging data captured by the solid-state imaging device 110 is demosaiced to generate image data. The image data is subjected to image processing as necessary, and then converted into a video signal to be displayed on the monitor 500. Display.

  In the spectroscopic analysis mode, the system controller 202 controls the filter control unit 420 to rotate and rotate the rotary filter 410 at a constant rotation speed, and the living tissue by the illumination light that has passed through the optical filters 415, 416, and 418, respectively. Are sequentially picked up. Then, an image showing the distribution of biomolecules in the biological tissue is generated based on the digital imaging data acquired using the optical filters 415 and 416, respectively, and a normal observation image captured using the optical filter 418. The arranged display screen is generated, further converted into a video signal, and displayed on the monitor 500.

  In the spectroscopic analysis mode, the filter control unit 420 detects the phase of the rotation of the rotary filter 410 based on the timing when the photo interrupter 422 detects the through hole 413, and uses this to detect the phase of the synchronization signal supplied from the timing controller 204. The phase of rotation of the rotary filter 410 is adjusted as compared with. The synchronization signal from the timing controller 204 is synchronized with the drive signal for the solid-state image sensor 110. Therefore, the rotary filter 410 is rotationally driven at a substantially constant rotational speed in synchronization with the driving of the solid-state imaging device 110. Specifically, the rotation of the rotation filter 410 is performed by optical filters 415, 416, on which the white light WL is incident every time one image (three frames of R, G, B) is captured by the solid-state imaging device 110. Control is performed so that 418 switches sequentially.

[Detailed configuration of solid-state image sensor]
As described above, in the spectroscopic analysis mode of the present embodiment, the digital imaging data G ₄₁₅ (digital imaging signal G ₄₁₅ ) imaged under irradiation of the illumination light IL ₄₁₅ that has passed through the optical filter 415 and the optical filter 416 have passed. Based on the digital imaging data G ₄₁₆ (digital imaging signal G ₄₁₆ ) imaged under irradiation of the illumination light IL ₄₁₆ , the concentration of the biomolecule in the biological tissue that is the subject is quantified.

In order to determine the concentration of the biomolecule with high accuracy, the width of the input range of the ADC (Analogue-to-Digital Converter) is set as narrow as possible during digital conversion, and the digital imaging signals G ₄₁₅ , G 315 with high gradation are set. ₄₁₆ needs to be obtained. In addition, in order to quantify the concentration of biomolecules with high accuracy, it is necessary to accurately match the zero points (black levels) of the two digital imaging signals G ₄₁₅ and G ₄₁₆ .

When the ADC input range is narrowed in accordance with the fluctuation range of the output levels of the analog imaging signals G ₄₁₅ and G ₄₁₆ of the solid-state imaging device 110, high gradation digital imaging signals G ₄₁₅ and G ₄₁₆ are obtained. However, if the input range of the ADC is simply adjusted, the zero point information is lost. _Therefore , the accurate absolute value of the two digital imaging signals G ₄₁₅ and G ₄₁₆ (or the accuracy between the digital imaging signals G ₄₁₅ and G ₄₁₆ is accurate). A relative value) cannot be obtained, and the accuracy of quantification is reduced.

The endoscope apparatus 1 according to the present embodiment can acquire high-gradation and high-accuracy digital imaging signals G ₄₁₅ and G ₄₁₆ by providing an input range setting unit 113 (FIG. 7) described below. This enables high-precision and high-accuracy quantification of biomolecule concentrations.

Further, the optical filter 415 and the optical filter 416 have different pass bandwidths, and as a result, the amount of light passing therethrough also differs (specifically, the amount of light passing through the optical filter 415 is approximately three times that of the optical filter 416). . Therefore, when the ADC input range of the analog imaging signal G ₄₁₆ is set in accordance with the high signal level of the analog imaging signal G ₄₁₅ , the gradation of the digital imaging signal G ₄₁₆ becomes rough.

The endoscope apparatus 1 of the present embodiment can individually adjust the input range of the ADC with respect to the analog imaging signals G ₄₁₅ and G _{416 having} different output levels, and thereby biomolecules in the biological tissue. Enables more accurate quantification of concentration.

  As shown in FIG. 3, the solid-state imaging device 110 includes a plurality of CDS (Correlated Double Sampling) [R-CDS, Gr-CDS, each of which is provided for each line and each color signal (R, Gr, Gb, B). Gb-CDS, B-CDS].

  Further, the solid-state image sensor 110 has four AFEs (R-AFE, Gr-AFE, Gb-AFE, B-) provided for each color signal (analog image signals R, Gr, Gb, B). AFE]. Digital imaging signals output from these four AFEs are normally used for generating an observation image. The solid-state imaging device 110 further includes Gr-AFEsa that outputs a high-precision digital imaging signal Gr for spectral analysis.

  The output line of Gr-CDS is connected to either Gr-AFE or Gr-AFEsa by switching the switch SW2. Specifically, Gr-AFE is connected to Gr-CDS in the normal observation mode, and Gr-AFEsa is connected to Gr-CDS in the spectroscopic analysis mode.

  The output line of Gb-AFE is connected to either the driver signal processing circuit 120 or Gr-AFEsa by switching the switch SW1. Specifically, in the normal observation mode, the output line of Gb-AFE is connected to the driver signal processing circuit 120, and in the spectroscopic analysis mode, the output line of Gb-AFE is connected to Gr-AFEsa, and Gr-AFEsa Is supplied with a digital imaging signal Gb.

  Further, from Gb-AFE, FSP described later is supplied to Gr-AFEsa.

  In general, a circuit including the CDS is referred to as AFE, but for convenience of explanation, a circuit excluding the CDS is referred to as “AFE”. In this specification, means for performing initial signal processing for generating a digital imaging signal such as CDS or AFE is referred to as initial processing means.

  Further, the CDS may be provided for each analog imaging signal (for example, in the AFE) without providing the CDS for each line. Further, an AFE (including CDS) may be provided for each line or each pixel. In this embodiment, the AFE (including the CDS) is provided in the solid-state imaging device 110. However, the AFE is provided in another circuit of the endoscope apparatus 1 such as the driver signal processing circuit 120, for example. It is good also as a structure.

[Gb-AFE (R-AFE, Gr-AFE, B-AFE)]
FIG. 6 is a block diagram of Gb-AFE. R-AFE, Gr-AFE, Gb-AFE and B-AFE have the same general configuration. The configuration of Gb-AFE will be described on behalf of these. However, in the present embodiment, as indicated by the dashed arrow in FIG. 6, the output of Gb-AFE is supplied to Gr-AFEsa, but the outputs of R-AFE, Gr-AFE, and B-AFE are Gr−. It is not supplied to AFEsa.

  As illustrated in FIG. 6, the Gb-AFE includes an AGC (Automatic Gain Control) 111 that performs automatic gain correction, and an ADC 112 that converts the analog imaging signal Gb into a digital imaging signal Gb. The analog imaging signal Gb output from the Gb-CDS is subjected to gain adjustment by the AGC 111 and then converted to the digital imaging signal Gb by the ADC 112. Specifically, the ADC 112 converts the analog imaging signal Gb into a full-scale digital imaging signal Gb including a zero point (black level). In the normal observation mode, the digital imaging signal Gb output from the Gb-AFE is supplied to the image processing circuit of the processor 200 through the driver signal processing circuit 120 together with the digital imaging signals R, Gr, and B output from other AFEs. 220. In the spectroscopic analysis mode, a digital imaging signal Gb output from the Gb-AFE and an FSP described later are supplied to the Gr-AFEsa and used for input range adjustment of the ADC 112a (FIG. 7) of the Gr-AFEsa.

[Gr-AFEsa]
FIG. 7 is a block diagram of Gr-AFEsa. The Gr-AFEsa includes an input range setting unit 113 and a restoration unit 114 in addition to the AGC 111 and the ADC 112a. The input range setting unit 113 is a functional unit that adjusts and sets the input range of the ADC 112a based on the digital imaging signal Gb output from the Gb-AFE. The input range setting unit 113 calculates and outputs two setting values (input span SP and input offset amount OF) that define the input range of the ADC 112a.

  FIG. 8 is a diagram for explaining the definitions of the input span SP and the input offset amount OF. FIG. 8A is a graph showing a full-scale waveform of an analog input (analog imaging signal Gr), and FIG. 8B is a graph in which the vicinity of the input range of FIG. 8A is enlarged.

  The input span SP is a set value that determines the width of the input range of the ADC 112a (difference between the upper limit value and the lower limit value of the input range). The input offset amount OF is a set value that determines the lower limit value of the input range (level difference from the zero point).

[Input range setting section]
FIG. 9 is a block diagram of the input range setting unit 113. The input range setting unit 113 includes a statistical processing unit 1131, an input span calculation unit 1132, and an input offset amount calculation unit 1133.

[Statistics processing section]
The statistical processing unit 1131 performs statistical processing on the digital imaging signal Gb output from the Gb-AFE, and outputs an average value Gb_AVG and a standard deviation Gb_SD of the digital imaging signal Gb.

[Input span calculation section]
The input span calculator 1132 calculates the input span SP using Equation 11 based on the standard deviation Gb_SD of the digital imaging signal Gb.
K is a preset constant. In this embodiment, K = 1.5 is set.

[Input offset calculation section]
The input offset amount calculation unit 1133 calculates the input offset amount OF using Equation 12 based on the average value Gb_AVG and the standard deviation Gb_SD of the digital imaging signal Gb.

  That is, as shown in FIG. 8B, the center of the input range of the ADC 112a is the average value Gb_AVG of the digital imaging signal Gb, and the width of the input range is 2K times (3 times) the standard deviation Gb_SD of the digital imaging signal Gb. The input offset amount OF and the input span SP are set so that

[Restore unit]
The ADC 112a sets the input range based on the input offset amount OF and the input span SP given from the input range setting unit 113, and converts the analog imaging signal Gr into the differential digital imaging signal Gr_dif. The difference digital imaging signal Gr_dif is a discrete value having zero as a minimum value and a value determined by the resolution of the ADC 112a (for example, 255 in the case of an 8-bit ADC) as a maximum value. The difference digital imaging signal Gr_dif can be said to be data including only information on the relative fluctuation amount of the analog imaging signal Gr from which the scale information (that is, the input offset amount OF and the input span SP) is removed from the analog imaging signal Gr. . Therefore, in order to obtain the digital imaging signal Gr including the absolute value information corresponding to the original analog imaging signal Gr, it is necessary to supplement the information about the input offset amount OF and the input span SP to the differential digital imaging signal Gr_dif.

Based on the difference digital imaging signal Gr_dif output from the ADC 112a and the input offset amount OF and the input span SP given from the input range setting unit 113, the restoration unit 114 uses the original analog imaging signal using Equation 13. A digital imaging signal Gr corresponding to Gr is generated. The digital imaging signal Gr has the same gradation accuracy as the differential digital imaging signal Gr_dif, and has a wider range than the differential digital imaging signal Gr. That is, the digital imaging signal Gr (for example, 16-bit signal) is a digital signal having a larger number of bits than the differential digital imaging signal Gr (for example, 8-bit signal).

  Here, FSP is a set value (constant) that determines the input span of the ADC 112 of Gb-AFE in the digital conversion of the analog image signal Gb, and is set to a value corresponding to the saturation output level of the analog image signal Gb. Further, as shown in FIG. 8A, the sum of the input offset amount OF and the input span SP may be FSP.

  With the Gr-AFEsa configuration described above, a digital imaging signal Gr having a high gradation and an accurate absolute value is generated. In the spectroscopic analysis mode of this embodiment described below, quantitative analysis of biomolecules is performed using a high-gradation and high-accuracy digital imaging signal Gr generated by Gr-AFEsa. Further, the normal observation image is generated by demosaic processing using the digital imaging signals R, Gr, and B output from the solid-state imaging device 110. In the normal observation mode, a normal observation image is generated from the four digital imaging signals R, Gr, Gb, and B.

[Image processing in spectral analysis mode]
Next, image generation processing executed by the image processing circuit 220 in the spectroscopic analysis mode will be described. Since the image processing circuit 220 calculates the index X according to the embodiment of the present invention as described later, it is also referred to as an “index calculation unit”. FIG. 10 is a flowchart for explaining image generation processing (index calculation processing).

When the spectroscopic analysis mode is selected by a user operation, as described above, the filter control unit 420 rotationally drives the rotary filter 410 at a constant rotational speed. Then, the illumination light IL that has passed through the optical filters 415, 416, and 418 is sequentially supplied from the light source unit 400, and imaging using each illumination light IL is sequentially performed (S1). Specifically, digital imaging data Gr ₄₁₅ (x, y) imaged using the illumination light IL that has passed through the optical filter 415, digital imaging data Gr ₄₁₆ (x, y) that was imaged using the illumination light IL that has passed through the optical filter ₄₁₆ ( x, y), and digital imaging data R ₄₁₈ (x, y), Gr ₄₁₈ (x, y), and B ₄₁₈ captured using the illumination light IL (white light) that has passed through the optical filter (ultraviolet cut filter) _418. (X, y) is stored in the internal memory 221 of the image processing circuit 220.

Next, the image processing circuit 220 uses the digital imaging data R ₄₁₈ (x, y), Gr ₄₁₈ (x, y), and B ₄₁₈ (x, y) acquired in the process S 1 to perform the following analysis process ( A pixel selection process S2 for selecting pixels to be processed S3-S5) is performed. For locations that do not contain blood or where the color of the tissue is predominantly affected by substances other than hemoglobin, meaningful values can be obtained even if oxygen saturation and blood flow are calculated from pixel color information. It cannot be obtained and becomes mere noise. If such noise is calculated and provided to the doctor, not only will the doctor make an accurate diagnosis, but there will also be a problem that an unnecessary load is applied to the image processing circuit 220 to reduce the processing speed. Therefore, in the image generation process of the present embodiment, pixels suitable for the analysis process (that is, pixels on which the spectroscopic characteristics of hemoglobin are recorded) are selected, and the analysis process is performed only on the selected pixels. It is configured.

In the pixel selection process S2, only pixels satisfying all the conditions of the following Expressions 14, 15, and 16 are selected as analysis target pixels.
Here, a ₁ , a ₂ , and a ₃ are positive constants.

  The above three conditional expressions are set based on the magnitude relationship of the values of G component <B component <R component in the blood transmission spectrum. Note that the pixel selection process S2 is performed using only one or two of the above three conditional expressions (for example, using Formula 15 and Formula 16 focusing on the red color peculiar to blood). Also good.

Next, the image processing circuit 220 calculates the absorption A ₄₁₅ (x, y) and A ₄₁₆ (x, y) of the living tissue. The absorptions A ₄₁₅ (x, y) and A ₄₁₆ (x, y) are calculated by the following equations 17 and 18.

Further, the absorptions A ₄₁₅ (x, y) and A ₄₁₆ (x, y) can also be approximately calculated by the following equations 19 and 20, respectively.

Further, as is apparent from the relationship between the absorption wavelength ranges R1, R2, and R3 of hemoglobin shown in FIG. 1 and the pass wavelength ranges of the optical filters 415 and 416, the absorption A _{R1 of} living tissue with respect to the wavelength ranges R1, R2, and R3 ( x, y), A _R2 (x, y), A _R3 (x, y), and absorption of living tissue with respect to the illumination light IL that has passed through the optical filters 415, ₄₁₆ A ₄₁₅ (x, y), A ₄₁₆ (x , Y) has a relationship represented by the following formulas 21 and 22.

Therefore, the index X (Formula 10) is expressed by the following Formula 23.

Further, as shown in FIG. 1, in the wavelength region R2, the fluctuation range of the absorbance with respect to the change in oxygen saturation is larger than that in the wavelength regions R1 and R3. Therefore, by setting a large weight for absorption A _R2 in the wavelength range R2, it is possible to improve the sensitivity of the index X for the oxygen saturation changes.

Specifically, for example, the index X can be calculated by Expression 24 in which the weight is doubled for the absorption _AR2 .

The non-volatile memory 222 included in the image processing circuit 220 stores a numerical table indicating a quantitative relationship between the oxygen saturation of hemoglobin and the value of the index X, which are experimentally acquired in advance. The image processing circuit 220 acquires the oxygen saturation SatO ₂ (x, y) corresponding to the value of the index X calculated from Expression 23 or Expression 24 with reference to this numerical table. Then, the image processing circuit 220 sets image data (oxygen saturation distribution image data) having a pixel value of each pixel (x, y) as a value obtained by multiplying the acquired oxygen saturation SatO ₂ (x, y) by a predetermined constant. ) Is generated (S5).

Further, the image processing circuit 220 includes digital imaging data R ₄₁₈ (x, y), Gr ₄₁₈ (x, y) acquired using the illumination light IL (white light) that has passed through the optical filter (ultraviolet cut filter) 418, and Normal observation image data is generated from B ₄₁₈ (x, y) by demosaic processing or the like.

  FIG. 11 shows a display example of image data generated by the image processing circuit 220. FIG. 11A is a display example of the oxygen saturation distribution image data (two-dimensional display) generated by the above-described process S5. FIG. 11B is a display example of oxygen saturation distribution image data (three-dimensional display) generated in the form of a three-dimensional graph with the oxygen saturation as a vertical axis. Note that FIG. 11 is an observation of the right hand in a state where the vicinity of the proximal interphalangeal joint of the middle finger is compressed with a rubber band. It is shown that the oxygen saturation is lowered due to the blood flow being blocked by the compression on the distal side of the compression part of the right middle finger.

  Further, the image processing circuit 220 generates screen data for displaying the normal observation image and the oxygen saturation distribution image side by side on one screen from the generated oxygen saturation distribution image data and normal observation image data. Note that the image processing circuit 220 displays a patient image on a display screen that displays only an oxygen saturation distribution image, a display screen that displays only a normal observation image, an oxygen saturation distribution image and / or a normal observation image, according to a user operation. Various display screens such as a display screen in which incidental information such as ID information and observation conditions are superimposed on each other can be generated.

Focusing on the absorption wavelength ranges R1, R2, and R3 of hemoglobin (that is, the pass wavelength range of the optical filter 415), the absorption A _R1 (x of each wavelength range R1, R2, and R3 according to the change in oxygen saturation) , Y), A _R2 (x, y), and A _R3 (x, y) change, but their sum Y (shown in Equation 25) is substantially constant. The sum Y of absorption is proportional to the total amount of hemoglobin in the living tissue (sum of the concentrations of oxygenated hemoglobin HbO ₂ and reduced hemoglobin Hb), so it is appropriate to use it as an index indicating the total amount of hemoglobin.

  It is known that a tissue of a malignant tumor has a higher total hemoglobin amount than that of a normal tissue due to angiogenesis, and oxygen metabolism is remarkable, so that oxygen saturation is lower than that of a normal tissue. Therefore, the image processing circuit 220 has an index X indicating the total hemoglobin amount calculated by Expression 25 larger than a predetermined reference value (first reference value), and an index X indicating oxygen saturation calculated by Expression 24 or the like. Is extracted from the pixel that is smaller than a predetermined reference value (second reference value), for example, lesion-enhanced image data is generated by performing highlight display processing on the corresponding pixel of the normal observation image data, and the normal observation image In addition, the lesion-emphasized image may be displayed on the monitor 500 together with (or alone) the oxygen saturation distribution image.

  As the highlighting process, for example, a process of increasing the pixel value of the corresponding pixel, a process of changing the hue (for example, a process of increasing the red component by increasing the R component, or a process of rotating the hue by a predetermined angle) ), A process of blinking the corresponding pixel (or changing the hue periodically).

  In addition, the image processing circuit 220, for example, based on the deviation from the average value of the index X (x, y) and the deviation from the average value of the index Y (x, y) instead of the lesion-emphasized image data. The index Z (x, y) indicating the degree of suspicion of malignant tumor may be calculated to generate image data (malignant suspicion image data) having the index Z as a pixel value.

  The above is the description of the embodiment of the present invention and specific examples of the embodiment. However, the present invention is not limited to the above-described configuration, and various modifications are possible within the scope of the technical idea of the present invention. Is possible.

  In the above embodiment, the input span SP is dynamically set based on the standard deviation Gb_SD of the digital imaging signal Gb. However, the input span SP may be a fixed set value (constant). In this case, it is desirable that the term depending on the standard deviation Gb_SD of the input offset amount OF is also a constant. According to this configuration, the calculation of the standard deviation Gb_SD becomes unnecessary, and the necessary calculation amount can be reduced.

  In the above-described embodiment, the processing by the input range setting unit 113 and the restoration unit 114 is performed on all the Gr pixels, but some Gr pixels (for example, the index X having a good imaging state and high accuracy) The input range setting unit 113 and the restoration unit 114 may be provided only in the Gr pixel at the center of the image where the image is obtained.

  In the above-described embodiment, a configuration in which data necessary for setting the input range of Gr-AFEsa is supplied from Gb-AFE is adopted. However, a configuration in which necessary data is supplied from Gr-AFE may be used. . In the above embodiment, in the spectroscopic analysis mode, the imaging signals output from the Gr-AFE and the Gb-AFE are not used for generating the normal observation image, but the imaging output from the Gr-AFE or the Gb-AFE is used. It is good also as a structure which produces | generates a normal observation image using a signal.

  Further, an optical image of a subject (blood digestive tract inner wall) containing blood contains an extremely large amount of R component as compared with G component and B component. For this reason, in the above-described embodiment, a configuration in which the signal processing of the imaging signal Gr by the input range setting unit 113 or the like is performed based on the imaging signal Gb with less leakage of the R component, but conversely, the imaging signal Gr. The signal processing of the imaging signal Gb may be performed based on the above (that is, Gb-AFEsa is provided).

  In addition, the solid-state imaging device 110 of the present embodiment has been described as an imaging device for color image imaging including an on-chip color filter, but is not limited to this configuration. It is good also as a structure provided with the color filter of what is called a field sequential system using this image pick-up element. The color filter is not limited to an on-chip configuration, and can be arranged in the optical path from the light source 430 to the solid-state image sensor 110.

  In the above embodiment, the rotary filter 410 is used. However, the present invention is not limited to this configuration, and other types of wavelength tunable filters whose pass wavelength band can be switched can be used. .

  In the above embodiment, a white light source such as a xenon lamp is used as a light source that generates broadband light for illumination. However, a non-white light source having a sufficient amount of light over the entire pass band of each optical filter to be used. It is also possible to use a light source that generates broadband light.

  The above embodiment is an example in which the present invention is applied to an electronic endoscope apparatus which is a form of a digital camera. However, other types of digital cameras (for example, a digital single lens reflex camera and a digital video camera) are used. The present invention can also be applied to the system used. For example, when the present invention is applied to a digital still camera, it is possible to observe body surface tissue and brain tissue during craniotomy (for example, rapid examination of cerebral blood flow).

DESCRIPTION OF SYMBOLS 1 Endoscope apparatus 100 Electronic scope 110 Solid-state image sensor 113 Input range setting part 114 Restoration part 200 Processor 220 Image processing circuit 400 Light source part 500 Monitor

Claims (17)

An image sensor;
Processing means for processing an imaging signal generated by the imaging element;
With
The image sensor is
A plurality of R pixels sensitive to red light;
A plurality of first and second G pixels each sensitive to green light;
A plurality of B pixels sensitive to blue light,
The processing means is
First initial processing means for processing the analog imaging signal G1 generated by the plurality of first G pixels to generate a digital imaging signal G1,
A second initial processing unit that processes the analog imaging signal G2 generated by the plurality of second G pixels to generate a digital imaging signal G2, and
A first digital conversion means for converting the analog imaging signal G1 into a digital signal G1_dif;
Input range setting means for generating a set value of the input range of the first digital conversion means based on the digital imaging signal G2 ,
The digital signal G1_dif is
The analog imaging signal G1 is a signal obtained by removing the input offset amount OF that defines the lower limit value of the input range of the first digital conversion means and the input span SP that defines the width of the input range of the first digital conversion means .
Imaging device. The input range setting means sets the input range of the first digital conversion means in accordance with the fluctuation range of the signal level of the analog imaging signal G2.
The imaging device according to claim 1. The input range setting means is
Statistical processing means for calculating a statistical value of the digital imaging signal G2,
Based on the statistics, and the input offset amount calculation means for calculating the input offset OF,
Based on the statistical value, with a, and input span calculation means for calculating the input span SP,
The imaging device according to claim 2. The statistical processing means is
Calculating an average value AVG and a standard deviation SD of the digital imaging signal G2,
The input offset amount calculating means is
Calculating the input offset amount OF based on the average value AVG;
The input span calculation means is
Calculating the input span SP based on the standard deviation SD;
The imaging device according to claim 3. The input offset amount calculation means calculates the input offset amount OF according to Equation 1.
Where K is a constant,
The imaging device according to claim 4. The input span calculation means calculates the input span SP according to Equation 2.
Where K is a constant,
The imaging device according to claim 4 or 5. The processing means is
The apparatus further comprises restoration means for generating the digital imaging signal G1 including absolute value information based on the digital signal G1_dif generated by the first initial processing means, the input offset amount OF, and the input span SP.
The imaging device according to any one of claims 3 to 6. The restoration means calculates the digital imaging signal G1 according to Equation 3.
However, FSP is a predetermined value.
The imaging device according to claim 7. The second initial processing means includes second digital conversion means for converting the analog imaging signal G2 into the digital imaging signal G2,
The FSP is an input span that defines the width of the input range of the second digital conversion means;
The imaging device according to claim 8. The first G pixel is a Gr pixel disposed on a line in which the R pixels are arranged;
The second G pixel is a Gb pixel arranged on a line in which the B pixels are arranged.
The imaging device according to any one of claims 1 to 9. The processing means is
Based on the digital imaging signal G1, further comprising index calculation means for calculating an index X indicating a molar concentration ratio of the first and second biological substances contained in the biological tissue as the subject;
The imaging device is
A light source device that generates light by switching between light in a first illumination wavelength region in which the first and second biological materials have absorption and light in a second illumination wavelength region in the first illumination wavelength region;
The indicator calculating means
A first digital imaging signal G _I obtained by imaging the living body tissue under illumination of the first illumination wavelength range of light, obtained by imaging the living body tissue under illumination of the second illumination wavelength range of the light a second digital imaging signal G _II was to calculate the index X based on,
The imaging device according to any one of claims 1 to 10. The first illumination wavelength range includes absorption peak wavelengths of both the first and second biological materials;
The second illumination wavelength region includes an absorption peak wavelength of one of the first and second biological materials;
The imaging device according to claim 11. The first illumination wavelength region is
A wavelength region adjacent to the short wavelength side of the second illumination wavelength region and including an absorption peak wavelength of the other biological material of the first and second biological materials;
A wavelength region adjacent to the long wavelength side of the second illumination wavelength region and including the absorption peak wavelength of the other biological material,
The imaging device according to claim 12. The light source device is
A light source that generates broadband light;
A first optical filter that selectively extracts light in the first illumination wavelength region from the broadband light;
A second optical filter that selectively extracts light in the second illumination wavelength region from the broadband light, and
The imaging device according to any one of claims 11 to 13. The indicator calculating means
Calculating the absorption A _{I of the} living tissue in the first illumination wavelength range based on the first imaging signal G _I ;
Calculating the absorption A _{II of the} living tissue in the second illumination wavelength region based on the second imaging signal G _II ;
Calculating the index X based on the absorption A _I and the absorption A _II ;
The imaging device according to any one of claims 11 to 14. The indicator calculating means
Calculate the absorption A _{I according} to either Equation 4 or Equation 5,
Calculating the absorption A _{II according} to any one of Equation 6 and Equation 7;
The imaging device according to claim 15. The indicator calculating means
Calculating the index X according to any one of Equation 8 and Equation 9;
The imaging device according to claim 15 or 16.