JP2018011796A - Calibration method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem that it is difficult, depending on a lesioned part, to reproduce color information optimal for calibration on a conventional chart for calibration.SOLUTION: A calibration method includes: setting a transmission type chart in an angle of view of an electronic scope in a measurement box; adjusting the transmission type chart to a color serving as a calibration target by irradiating the transmission type chart with light having a color tone corresponding to the set transmission type chart; imaging the transmission type chart adjusted to the color serving as the calibration target by an electronic scope; calculating a calibration value calibrating a value of each pixel constituting an image imaged by the electronic scope on the basis of image data of the imaged transmission type chart; and storing the calculated calibration value.SELECTED DRAWING: Figure 9

Description

  The present invention relates to a calibration method.

  The lesion is generally colored differently from normal mucosal tissue. In order for the surgeon to grasp and diagnose a lesion slightly different in color from normal tissue, an endoscopic image that reproduces accurate color information at least in the color region corresponding to the lesion is displayed on the monitor screen. A configuration that can be displayed on the screen is required.

  For example, Patent Document 1 describes a configuration in which a reference surface carried on a carrier is placed in a field of view and irradiated with irradiation light emitted from a light source. In the configuration described in Patent Document 1, calibration is performed based on a comparison result between an actual color signal obtained by imaging a reference surface irradiated with irradiation light and a target color signal held in advance. . By performing the calibration, an endoscopic image having appropriate color information is displayed on the display screen of the monitor, so that the surgeon can more accurately diagnose the lesioned part.

JP-A-6-46428

  However, in the configuration described in Patent Document 1, there is a problem that it is difficult to faithfully reproduce the color information optimal for calibration on the reference surface depending on the lesion because there are restrictions on the material and the like of the reference surface. The

  The present invention has been made in view of the above circumstances, and an object of the present invention is to perform calibration using a chart such as a reference surface that faithfully reproduces color information optimal for calibration. A calibration method is provided.

  A calibration method according to an embodiment of the present invention is a method for calibrating an electronic scope set in a measurement box, and sets a transmission chart in the measurement box and within the angle of view of the electronic scope. An adjustment step for adjusting the transmissive chart to a calibration target color by irradiating the transmissive chart with light having a hue corresponding to the set transmissive chart, and a target color. An imaging step for imaging the transmission chart adjusted to the above with an electronic scope, and a calculation for calculating a correction value for correcting the value of each pixel constituting the captured image by the electronic scope based on the image data of the captured transmission chart A step and a storing step for storing the calculated correction value.

  The calibration method according to an embodiment of the present invention includes a setting step of setting a ratio of the brightness of each color with respect to a light source capable of independently adjusting the brightness of each of a plurality of colors. Also good. In this case, the ratio corresponding to the transmission chart set in the setting step is set in the setting step. In addition, by causing each color of the light source to emit light at the ratio set in the setting step in the adjustment step, the transmission chart is irradiated with light having a hue corresponding to the transmission chart set in the setting step. The

  In one embodiment of the present invention, the ratio of the brightness of each color may be set using the data stored in advance in the memory in the setting step.

  In one embodiment of the present invention, the transmission chart is adjusted to, for example, a predetermined disease color in the adjustment step.

  The calibration method according to an embodiment of the present invention may execute a set step, an irradiation step, and an imaging step for each of a plurality of transmission charts. In this case, in the calculation step, a correction value for correcting the value of each pixel constituting the captured image by the electronic scope is calculated based on the image data of the plurality of captured transmission charts.

  According to an embodiment of the present invention, there is provided a calibration method capable of performing calibration using a chart such as a reference surface that faithfully reproduces color information optimal for calibration.

It is a block diagram which shows the structure of the electronic endoscope system which concerns on one Embodiment of this invention. It is a figure which shows the flowchart of the special image generation process performed at the time of special mode in one Embodiment of this invention. It is a figure which shows the RG plane on which a pixel corresponding point is plotted in one Embodiment of this invention. It is a figure explaining the reference axis set in an RG plane. It is a figure which shows the example of a display screen displayed on the display screen of a monitor at the time of special mode in one Embodiment of this invention. It is a schematic block diagram which shows the state which set the jig | tool for calibration in one embodiment of this invention in the electronic endoscope system. It is a block diagram which shows the structure of the LED lighting apparatus and LED lighting power supply which concern on one Embodiment of this invention. It is a circuit diagram which shows the structure of each LED row | line | column of the LED lighting apparatus which concerns on one Embodiment of this invention. It is a figure which shows the flowchart of the calibration process performed at the time of calibration mode in one Embodiment of this invention. FIG. 10 is a diagram for assisting in the description of the calibration process of FIG. 9.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following, an electronic endoscope system will be described as an example of an embodiment of the present invention.

[Configuration of Electronic Endoscope System 1]
FIG. 1 is a block diagram showing a configuration of an electronic endoscope system 1 according to an embodiment of the present invention. As shown in FIG. 1, the electronic endoscope system 1 is a system specialized for medical use, and includes an electronic scope 100, a processor 200, and a monitor 300.

  The processor 200 includes a system controller 202 and a timing controller 204. The system controller 202 executes various programs stored in the memory 222 and controls the entire electronic endoscope system 1 in an integrated manner. The system controller 202 is connected to the operation panel 218. The system controller 202 changes each operation of the electronic endoscope system 1 and parameters for each operation in accordance with an instruction from the operator input from the operation panel 218. The input instruction by the operator includes, for example, an instruction to switch the operation mode of the electronic endoscope system 1. In this embodiment, there are a normal mode, a special mode, and a calibration mode as operation modes. The timing controller 204 outputs a clock pulse for adjusting the operation timing of each unit to each circuit in the electronic endoscope system 1.

  The lamp 208 emits white light L after being started by the lamp power igniter 206. The lamp 208 is, for example, a high-intensity lamp such as a xenon lamp, a halogen lamp, a mercury lamp, or a metal halide lamp, and may be a semiconductor light emitting element such as an LD (Laser Diode) or an LED (Light Emitting Diode). The white light L emitted from the lamp 208 is limited to an appropriate amount of light through the diaphragm 212 while being collected by the condenser lens 210.

  A motor 214 is mechanically connected to the diaphragm 212 via a transmission mechanism such as an arm or a gear (not shown). The motor 214 is a DC motor, for example, and is driven under the drive control of the driver 216. The aperture 212 is operated by the motor 214 to change the opening degree so that the image displayed on the display screen of the monitor 300 has an appropriate brightness. The amount of white light L emitted from the lamp 208 is limited according to the opening degree of the diaphragm 212. The appropriate reference for the brightness of the image is changed according to the brightness adjustment operation of the operation panel 218 by the operator. Note that the dimming circuit that controls the brightness by controlling the driver 216 is a well-known circuit and is omitted in this specification.

  The white light L that has passed through the diaphragm 212 is condensed on an incident end face of an LCB (Light Carrying Bundle) 102 and is incident on the LCB 102. White light L incident on the LCB 102 from the incident end face propagates in the LCB 102.

  The white light L that has propagated through the LCB 102 is emitted from the emission end face of the LCB 102 disposed at the tip of the electronic scope 100 and irradiates the living tissue via the light distribution lens 104. The return light from the living tissue irradiated with the white light L forms an optical image on the light receiving surface of the solid-state image sensor 108 via the objective lens 106.

  The solid-state image sensor 108 is a single-plate color CCD (Charge Coupled Device) image sensor equipped with a complementary color checkered filter. The solid-state image sensor 108 accumulates an optical image formed by each pixel on the light receiving surface as a charge corresponding to the amount of light, and generates and outputs pixel data of yellow Ye, cyan Cy, green G, and magenta Mg. The solid-state imaging element 108 is not limited to a CCD image sensor, and may be replaced with a CMOS (Complementary Metal Oxide Semiconductor) image sensor or other types of imaging devices. The solid-state image sensor 108 may also be one equipped with a primary color filter (Bayer array filter).

  A driver signal processing circuit 112 is provided in the connection portion of the electronic scope 100. The driver signal processing circuit 112 receives pixel data of each pixel obtained by imaging the living tissue irradiated with the white light L from the solid-state imaging device 108 in a frame cycle. The driver signal processing circuit 112 outputs pixel data input from the solid-state image sensor 108 to the signal processing circuit 220 of the processor 200. In the following description, “frame” may be replaced with “field”. In the present embodiment, the frame period and the field period are 1/30 seconds and 1/60 seconds, respectively.

  The driver signal processing circuit 112 also accesses the memory 114 to read out the unique information of the electronic scope 100. The unique information of the electronic scope 100 recorded in the memory 114 includes, for example, the number and sensitivity of the solid-state image sensor 108, the operable frame rate, the model number, and the like. The driver signal processing circuit 112 outputs the unique information read from the memory 114 to the system controller 202.

  The system controller 202 performs various calculations based on the unique information of the electronic scope 100 and generates a control signal. The system controller 202 controls the operation and timing of various circuits in the processor 200 using the generated control signal so that processing suitable for the electronic scope connected to the processor 200 is performed.

  The timing controller 204 supplies clock pulses to the driver signal processing circuit 112 in accordance with timing control by the system controller 202. The driver signal processing circuit 112 drives and controls the solid-state imaging device 108 at a timing synchronized with the frame rate of the video processed on the processor 200 side, according to the clock pulse supplied from the timing controller 204.

[Operation in normal mode]
A signal processing operation in the processor 200 in the normal mode will be described.

  The signal processing circuit 220 provided in the processor 200 includes a preprocessing circuit 220A, a processing circuit 220B, an output circuit 220C, a correction circuit 220D, a scoring circuit 220E, and a mapping circuit 220F.

  The pre-processing circuit 220A performs demosaic processing on the RAW pixel data input at a frame period from the driver signal processing circuit 112 to convert it into RGB pixel data, color matrix processing, white balance adjustment, Hue gain adjustment, and the like. And output to the process circuit 220B.

  The process circuit 220B performs normal processing, gamma correction, and the like on the pixel data input from the preprocess circuit 220A to generate normal color image data, and outputs the normal color image data to the output circuit 220C.

  The output circuit 220C performs processing such as Y / C separation and color difference correction on the color image data input from the process circuit 220B and converts it into a predetermined video format signal. The converted video format signal is output to the monitor 300. As a result, a normal color image of the living tissue is displayed on the display screen of the monitor 300.

[Operation in special mode]
Next, a signal processing operation in the processor 200 in the special mode will be described. FIG. 2 shows a flowchart of special image generation processing executed in the special mode. The special image generation process of FIG. 2 is started when the operation mode of the electronic endoscope system 1 is switched to the special mode.

[S11 in FIG. 2 (input of pixel data of current frame)]
In this processing step S11, pixel data of each pixel of the current frame is input to the preprocess circuit 220A. Pixel data of each pixel is input to the process circuit 220B and the correction circuit 220D after signal processing by the preprocess circuit 220A.

[S12 in FIG. 2 (plot on RG plane)]
FIG. 3 is a diagram for conceptually explaining the operation of the correction circuit 220D, and shows an RG plane (two-dimensional color space) defined by an R axis and a G axis orthogonal to each other. The R axis is the axis of the R component (R pixel value), and the G axis is the axis of the G component (G pixel value).

  In this processing step S12, pixel data (three-dimensional data) of each pixel in the RGB space defined by the RGB three primary colors is converted into two-dimensional data of RG, and as shown in FIG. And is plotted in the RG plane. Hereinafter, for the convenience of description, the pixel data points plotted in the RG plane are referred to as “pixel corresponding points”. In FIG. 3, for the sake of clarity, only the pixel corresponding points of some pixels are shown instead of the pixel corresponding points of all the pixels.

[S13 in FIG. 2 (setting of reference axis)]
In this processing step S13, a reference axis in the RG plane necessary for calculating the inflammation intensity of a predetermined disease such as gastritis is set by the correction circuit 220D. FIG. 4 is a diagram for assisting the explanation of the reference axis.

  In the body cavity of a patient to be imaged, the R component is dominant over other components (G component and B component) due to the influence of hemoglobin pigment or the like. Component) becomes strong against other colors (G component and B component). However, the color of the captured image in the body cavity changes according to the imaging condition that affects the brightness (for example, the degree of hitting of the white light L). Illustratively, the shaded portion where the white light L does not reach is black (achromatic color, for example, R, G, B is zero or a value close to zero), and the portion where the white light L strikes strongly and is specularly reflected is White (achromatic color, for example, R, G, B is a value close to 255 or 255). That is, even when the same abnormal site where inflammation has occurred is captured, the pixel value of the abnormal site image increases as the white light L hits stronger. Therefore, depending on how the white light L hits, the pixel value may take a value that has no correlation with the intensity of inflammation.

  In general, normal sites in body cavities that are not inflamed are covered with sufficient mucosa. On the other hand, an abnormal site in a body cavity where inflammation occurs is not covered with sufficient mucosa. Specifically, blood vessels and blood fluids leak from the blood vessels as the blood vessels dilate, so that the mucous membrane becomes relatively thin and the color of blood is easily visible. The mucous membrane is basically a white tone, but has a slightly yellowish color, and the color (yellow color) reflected on the image changes depending on the shade (thickness of the mucous membrane). Therefore, the density of the mucous membrane is considered to be one index for evaluating the intensity of inflammation.

  Therefore, in this processing step S13, as shown in FIG. 4, a straight line passing through (50, 0) and (255, 76) is set as one of the reference axes in the RG plane, and (0, 0) and (255, 192) are set as one of the reference axes. For convenience of explanation, the former reference axis is referred to as “hemoglobin change axis AX1”, and the latter reference axis is referred to as “mucosal change axis AX2”.

  The plot shown in FIG. 4 is obtained as a result of analysis of many sample images in the body cavity by the inventor. Sample images used for analysis include an example of inflammation image with the highest symptom level (example of inflammation image with the most severe level) and an example of inflammation image with the lowest symptom level (image example that is considered to be a substantially normal site) ) And the like are included. In the example of FIG. 4, only a part of the plot obtained as a result of the analysis is shown for the sake of clarity. The number of plots actually obtained as a result of the analysis is much larger than the number of plots shown in FIG.

  As described above, the R component becomes stronger with respect to other components (G component and B component) as the abnormal site is more intensely inflamed. Therefore, it is a boundary line between a region where the plot is distributed and a region where the plot is not distributed, and is an axis on the boundary line closer to the R axis than the G axis, in the example of FIG. 4, (50, 0) and (255, 76). ) Is set as an axis having a high correlation with the lesion having the highest symptom level (the inflammation (abnormal) site having the highest symptom level). This axis is the hemoglobin change axis AX1. On the hemoglobin change axis AX1, a plot corresponding to an inflammatory site with the highest symptom level photographed under various photographing conditions (for example, the degree of hitting of the white light L) is superimposed.

  On the other hand, the closer to the normal site, the stronger the G component (or B component) becomes than the R component. Therefore, it is a boundary line between a region where the plot is distributed and a region where the plot is not distributed, and is an axis on the boundary line closer to the G axis than the R axis, in the example of FIG. 4, (0, 0) and (255, 192). ) On the boundary line passing through) is highly correlated with the lesion with the lowest symptom level (the inflammatory (abnormal) site with the lowest symptom level, which is considered to be a substantially normal (healthy) site) Set as axis. This axis is the mucosa changing axis AX2. On the mucosal axis AX2, a plot corresponding to an inflammatory site having the lowest symptom level (substantially regarded as a normal site) photographed under various imaging conditions (for example, how white light L hits) is superimposed. .

  In addition, the site of inflammation with the highest symptom level is accompanied by bleeding. On the other hand, the inflammatory site with the lowest symptom level is a substantially normal site and is therefore covered with sufficient mucosa. Therefore, the plot in the RG plane shown in FIG. 4 can be understood as being distributed in a region sandwiched between an axis having the highest correlation with blood (hemoglobin pigment) and an axis having the highest correlation with mucous color. . Therefore, of the boundary lines between the region where the plot is distributed and the region where the plot is not distributed, the boundary line closer to the R axis (the R component is strong) indicates the inflammatory site with the highest symptom level (hemoglobin change axis AX1). The boundary line closer to the G axis (stronger G component) corresponds to the axis indicating the inflammatory site with the lowest symptom level (mucosal change axis AX2).

[S14 in FIG. 2 (Selection of Target Pixel)]
In this processing step S14, the correction circuit 220D selects one target pixel from all the pixels in a predetermined order.

[S15 in FIG. 2 (correction of pixel data)]
The correction circuit 220D stores correction matrix coefficients calculated at the time of calibration processing described later. In this processing step S15, the correction circuit 220D performs processing step S14 (attention) in order to suppress variation in score values (in other words, individual differences of electronic scopes) when the same lesioned part is imaged by different electronic endoscope systems. The pixel data (R, G) of the target pixel selected in (Selection of pixel) is corrected using the correction matrix coefficient.

・ Example of correction matrix
R _new : Pixel data of the target pixel after correction (R component)
G _new : Pixel data of the target pixel after correction (G component)
M _{00 to} M ₁₁ : Correction matrix coefficient R: Pixel data of the target pixel before correction (R component)
G: Pixel data of the target pixel before correction (G component)

[S16 in FIG. 2 (calculation of angle)]
When processing steps S14 (selection of target pixel) and S15 (correction of pixel data) are executed for all the pixels in the current frame in the correction circuit 220D, the scoring circuit 220E performs processing step S15 (correction of pixel data). The angle for calculating the inflammation intensity is calculated for the pixel data (R _new , G _new ) of each pixel corrected in (1). Specifically, in this processing step S16, for each pixel, a line segment L connecting the intersection (reference point) O ′ between the hemoglobin change axis AX1 and the mucosa change axis AX2 and the pixel corresponding point (R _new , G _new ). And an angle θ (see FIG. 3) formed by the hemoglobin change axis AX1 is calculated. The reference point O ′ is located at the coordinates (−150, −75).

[S17 in FIG. 2 (normalization processing)]
When the brightness of the captured image in the body cavity changes depending on how the white light L hits, the color of the captured image is affected by individual differences, the captured location, the state of inflammation, etc. The inflammatory site with the highest level changes along the hemoglobin change axis AX1, and the inflammatory site with the lowest symptom level changes along the mucosal change axis AX2. Moreover, it is estimated that the color of the photographed image of the inflammatory site having an intermediate symptom level also changes with the same tendency. That is, when the pixel corresponding point corresponding to the inflamed site changes depending on how the white light L hits, the pixel corresponding point shifts in the azimuth direction starting from the reference point O ′. In other words, when the pixel corresponding point corresponding to the inflamed site changes depending on how the white light L hits, the angle θ remains constant and the distance from the reference point O ′ changes. This means that the angle θ is a parameter that is substantially unaffected by changes in the brightness of the captured image.

  As the angle θ is smaller, the R component is stronger than the G component, indicating that the symptom level at the inflammatory site is high. Further, as the angle θ is larger, the G component becomes stronger than the R component, indicating that the symptom level at the inflammatory site is lower.

Therefore, in this processing step S17, the scoring circuit 220E uses the angle 255 for all the pixels of the current frame so that the value 255 is obtained when the angle θ is zero and the value is zero when the angle θ is θ _MAX. θ is normalized. Note that θ _MAX is equal to the angle formed by the hemoglobin change axis AX1 and the mucosa change axis AX2. Thereby, the inflammation intensity (8-bit information) falling within the range of 0 to 255 is obtained.

[S18 in FIG. 2 (calculation of inflammation evaluation value)]
In this processing step S18, the scoring circuit 220E calculates an average value (or an integrated value of the inflammation intensity of all the pixels) obtained by averaging the inflammation intensity of all the pixels in the current frame as the inflammation evaluation value of the entire captured image. In addition, display data (display data example: Score: OO) of the calculated inflammation evaluation value is generated.

[S19 in FIG. 2 (Determination of Display Color on Color Map Image)]
In this embodiment, it is possible to display a color map image obtained by mosaicing a captured image with a display color corresponding to the inflammation intensity. In order to display the color map image, a table in which the value of the inflammation intensity is associated with a predetermined display color is stored in a predetermined storage area of the scoring circuit 220E. In this table, for example, different display colors are associated with each value in increments of 5. Illustratively, yellow is associated with a value of inflammation intensity in the range of 0 to 5, and each time the value increases by 5, a different display color is associated according to the order of colors in the hue circle, When the value is in the range of 250 to 255, red is associated.

  In this processing step S19, the mapping circuit 220F displays the display color of each pixel of the current frame on the color map image based on the above table, the value of the inflammation intensity obtained in the processing step S17 (normalization processing). Depending on the color.

[S20 in FIG. 2 (Generation of Color Map Image Data)]
In the present processing step S20, the mapping circuit 220F converts the color data of each pixel of the current frame into the display color data determined in the processing step S19 (determining the display color on the color map image). Color map image data composed of pixels displayed in the displayed display color is generated.

[S21 in FIG. 2 (overlay processing)]
In this processing step S21, the output circuit 220C causes a normal color image based on the normal color image data input from the process circuit 220B, and the color map image generated in the processing step S20 (color map image data generation). The former image data (normal color image data) and the latter image data (color map image data) are added using the ratio of overlaying the color map image based on the data as a coefficient.

  It should be noted that the coefficient setting can be appropriately changed by a user operation. For example, when a normal color image is desired to be displayed darker, the color image data coefficient is set higher. When a color map image is desired to be displayed darker, the color map image data coefficient is set higher.

[S22 in FIG. 2 (End Determination)]
In this processing step S22, it is determined whether or not the operation mode of the electronic endoscope system 1 has been switched to a mode different from the special mode. When it is determined that the mode has not been switched to another mode (S22: NO), the special image generation processing in FIG. 2 returns to processing step S11 (input of pixel data of the current frame). On the other hand, when it is determined that the mode has been switched to another mode (S22: YES), the special image generation process in FIG. 2 ends.

[Screen display example]
The output circuit 220C generates display data of an overlay image of a normal color image and a color map image based on the image data added in processing step S21 (overlay processing) in FIG. A masking process for masking the peripheral area (periphery of the image display area) is performed, and screen data for monitor display is generated by superimposing an inflammation evaluation value on the mask area generated by the masking process. The output circuit 220 </ b> C converts the generated monitor display screen data into a predetermined video format signal and outputs it to the monitor 300.

  FIG. 5 shows a screen display example in the special mode. As illustrated in FIG. 5, on the display screen of the monitor 300, a captured image in the body cavity (an overlay image in which a normal image and a color map image are displayed in an overlay manner) is displayed in the center area, and an image display area A screen with a mask around is displayed. In addition, an inflammation evaluation value (score) is displayed in the mask area.

  Note that the display form of the captured image in the special mode is not limited to an overlay display of a normal color image and a color map image. For example, a display form in which a normal color image and a color map image are displayed side by side in a single screen, or only a color map image is displayed. In the former case, both the normal color image and the color map image may be displayed with the same size, or one of the normal color image and the color map image is displayed as the main image and the other is smaller than the main image. It may be displayed as an image.

  As described above, according to the present embodiment, the inflammation evaluation value (here, the region to be imaged) can be obtained only by performing simple calculation processing without performing nonlinear calculation processing such as tone enhancement processing or complicated color space conversion processing. A value correlated with the increase or decrease of the hemoglobin pigment). That is, the hardware resources necessary for calculating the inflammation evaluation value can be greatly reduced. In addition, since the inflammation evaluation value does not substantially vary depending on the imaging conditions that affect the brightness of the captured image in the body cavity (for example, how the irradiated light hits), the surgeon can make a more objective and accurate determination of inflammation. Can be reduced.

[Description of calibration mode]
Next, the calibration mode will be described. In the calibration mode, the calibration jig 400 is set in the electronic endoscope system 1 and the calibration software is activated on a terminal (PC) connected to the processor 200. FIG. 6 is a schematic configuration diagram showing a state where the calibration jig 400 is set in the electronic endoscope system 1.

  In the calibration process executed in the calibration mode, calibration is performed with a specific color related to the target disease as a correction target. That is, in this calibration process, calibration different from white balance is performed. It is assumed that the white balance has already been adjusted using the white balance dedicated jig. Note that white balance adjustment using the calibration jig 400 can be performed by using a white plate, a gray card, or the like having the same specifications as the white balance dedicated jig instead of a transmission chart 408 described later.

  As shown in FIG. 6, the calibration jig 400 includes a measurement box 402. In the measurement box 402, an introduction port 402a is formed.

  The measurement box 402 is provided with a fixed base 404. In executing the calibration process, the distal end portion of the electronic scope 100 is attached to the fixed base 404 by the attachment member 404a.

  The position adjustment controller 406 can move the fixed base 404 forward and backward through the introduction port 402a in and out of the measurement box 402 by an actuator (not shown). Specifically, the position adjustment controller 406 includes a forward key 406a, a backward key 406b, and a display 406c. When the advance key 406a is pressed by the operator, the fixed base 404 advances into the measurement box 402 through the introduction port 402a. When the operator presses the retreat key 406b, the fixed base 404 moves out of the measurement box 402 through the introduction port 402a. On the display 406c, the position of the fixed base 404 designated by the operation on the forward key 406a or the backward key 406b is displayed.

  In executing the calibration process, the operator attaches the distal end portion of the electronic scope 100 to the fixed base 404 in a state where the fixed base 404 is located outside the measurement box 402. Next, the operator presses the advance key 406a to move the fixed base 404, thereby inserting the distal end portion of the electronic scope 100 to a predetermined position in the measurement box 402 through the introduction port 402a. By inserting the distal end portion of the electronic scope 100 into the measurement box 402, the introduction port 402a is substantially blocked. Thereby, external light does not enter into the measurement box 402, and the measurement box 402 functions as a dark box. Since calibration is not affected by external light, the accuracy of calibration is improved.

  In the measurement box 402, a transmission chart 408 is arranged at a position facing the introduction port 402a (in other words, a position facing the front end surface of the electronic scope 100 (an arrangement surface of the light distribution lens, objective lens, etc.)). Yes. The transmission chart 408 is a transmission chart for calibration processing. During the calibration process, a color index imitating a specific color related to the target disease is set as the transmission chart 408. The transmission chart 408 is mechanically coupled to the drive mechanism 416 and is set at a predetermined position in the measurement box 402 by the drive mechanism 416. Further, the transmission chart 408 may be manually set at a predetermined position in the measurement box 402.

  An LED illumination device 410 is disposed on the back surface of the transmission chart 408. The LED illumination device 410 is controlled to be turned on / off and brightness by the power supplied from the LED illumination power source 414.

  By providing the LED illumination device 410 in the measurement box 402, it is not necessary to use the white light L emitted from the lamp 208 during calibration. Therefore, in the calibration process, it is not necessary to consider aged deterioration of the lamp 208 (in other words, the spectral characteristics of the white light L).

  FIG. 7 is a block diagram showing the configuration of the LED illumination device 410 and the LED illumination power supply 414. As shown in FIG. 7, the LED illumination device 410 has a plurality of bullet-type or chip-type LEDs arranged in a square lattice pattern. More specifically, in the LED lighting device 410, LED rows 410rL, 410gL, and 410bL in which a plurality of LEDs are arranged in a row (up and down direction in the figure) are repeatedly arranged at predetermined intervals in the row direction (left and right direction in the figure). Yes. The LED row 410rL is a row of red LEDs arranged in a row, the LED row 410gL is a row of green LEDs, and the LED row 410bL is a row of blue LEDs arranged in a row. .

  In the above, each LED row is arranged so that each LED has a square lattice arrangement, but each adjacent LED row is shifted by a predetermined amount so that the square lattice has an LED arrangement rotated by 45 °. May be arranged. In addition, in the above, each LED column has a configuration in which a plurality of LEDs are arranged in a row (vertical direction in the figure), but a plurality of LEDs may be arranged in a row direction (left and right direction in the diagram). Good.

  FIG. 8 is a circuit diagram showing a configuration of each LED row. As shown in FIG. 8, in the LED rows 410rL, 410gL, and 410bL, a plurality of red LEDs 410r, green LEDs 410g, and blue LEDs 410b are connected in series, and variable resistors VRr, VRg, and VRb are connected in series. Yes. Each variable resistor is built in the LED illumination power supply 414, for example.

  The LED illumination power supply 414 includes a power supply circuit 414a. The power supply circuit 414a converts an AC power supplied from a commercial power source into a DC output power source Vcc having a predetermined voltage, and converts the converted DC output power source Vcc through the variable resistors VRr, VRg, VRb into the LED strings 410rL, 410gL. , 410 bL. Thereby, each LED emits light.

  The light of each color emitted from the LED illumination device 410 is diffused by the diffusion plate 412 disposed in the front stage of the LED illumination device 410, converted into uniform light, and applied to the transmission chart 408.

  The LED illumination power source 414 includes a control unit 414b that controls the brightness of each color light emitted from each LED. The LED illumination power supply 414 includes a brightness adjustment knob 414c, a color adjustment knob 414rd, 414gd, and 414bd. The brightness adjustment knob 414c is a knob for manually adjusting the brightness of the LED illumination device 410 (the brightness of each LED). The color adjustment knobs 414rd, 414gd, and 414bd are knobs for manually adjusting the brightness of the red LED 410r, the green LED 410g, and the blue LED 410b, in other words, the hue (the ratio of the brightness of each color).

  In each LED row 410rL, 410gL, 410bL, each LED 410r, 410g, 410b emits light with brightness according to the input current (in other words, the output power supply Vcc and the resistance values of the variable resistors VRr, VRg, VRb).

  For example, consider a configuration in which the output power supply Vcc is constant. In this configuration, the control unit 414b increases or decreases the resistance values of the variable resistors VRr, VRg, and VRb based on adjustment values by the brightness adjustment knob 414c, the color adjustment knobs 414rd, 414gd, and 414bd or a command from the PC. Thus, the input current to each LED 410r, 410g, 410b is controlled.

  Also, consider a configuration in which the resistance values of the variable resistors VRr, VRg, VRb are constant, for example. In this configuration, the control unit 414b controls the input current to each LED 410r, 410g, 410b by varying the output power supply Vcc based on a command from the PC.

  FIG. 9 shows a flowchart of the calibration process executed in the calibration mode. FIG. 10 is a diagram for assisting the description of the calibration process of FIG. The calibration process shown in FIG. 9 is executed at the time of factory shipment, for example, and is started when the operation mode of the electronic endoscope system 1 is switched to the calibration mode.

  Prior to the execution of the calibration process, a preparatory work is performed by an operator. Specifically, the operator adjusts the white balance of the electronic endoscope system 1 using a white balance dedicated jig. When the white balance adjustment is completed, the operator sets the electronic endoscope system 1 on the calibration jig 400 and activates the calibration software on the PC connected to the processor 200.

[S31 in FIG. 9 (Reset of Variable i)]
In this processing step S31, when a predetermined switch of the LED illumination power source 414 is pressed or the power source is turned on, the variable i held in the internal memory of the control unit 414b is reset to zero.

[S32 in FIG. 9 (Transparent Chart Set)]
In this processing step S 32, the transmission chart 408 is set at a predetermined position in the measurement box 402 by the drive mechanism 416. Thereby, the transmission chart 408 is fixedly disposed in the space where the external light is shielded and in the angle of view of the electronic scope 100. The transmission chart 408 may be set manually.

  In the present embodiment, a plurality of (n) transmissive charts 408 are prepared for the calibration process. Each of the plurality of transmission charts 408 is a color index imitating a color when the living tissue is at a symptom level at a different stage for the target disease.

For the first execution of this processing step S32, for example, as a transmission chart 408, for example, an index simulating the color of the living tissue when the symptom level is the highest for the target disease (hereinafter, “first transmission chart” for convenience of explanation) .) Is set. In the present embodiment, the first transmission chart is a chart simulating the color corresponding to a predetermined point (first target point P _T1 described later) on the hemoglobin change axis AX1 in the RG plane.

  Note that the order of the transmission chart 408 set on the calibration jig 400 during the calibration process is determined in advance. Therefore, the transmission chart 408 set in the calibration jig 400 is known to the control unit 414b.

[S33 in FIG. 9 (Color Adjustment of Transmission Chart)]
With the transmission chart 408 alone, it is difficult to faithfully reproduce the color of the living tissue at the symptom level at each stage because there are restrictions on materials and the like. As an example, with the first transmission chart alone, it is difficult to faithfully reproduce the color of a predetermined point (first target point P _T1 described later) on the hemoglobin change axis AX1 in the RG plane. Therefore, in this processing step S33, the color of the transmission chart 408 is adjusted by adjusting the hue of the light emitted from the LED illumination device 410.

  Specifically, adjustment data corresponding to each of the plurality of transmission charts 408 is stored in advance in the memory 414 e provided in the calibration jig 400. For example, when adjusting the first transmission chart, the control unit 414b reads out adjustment data corresponding to the first transmission chart from the memory 414e, and based on the read adjustment data, the variable resistors VRr, VRg, The resistance value of VRb is adjusted. Thereby, the hue of the light emitted from the LED lighting device 410 is adjusted. The first transmissive chart is irradiated with light after color adjustment to the first transmissive chart so that the first transmissive chart has the color that is the target of calibration (the color of the living tissue when the symptom level is the highest for the target disease). Adjusted. In other words, the first transmission chart faithfully reproduces the color of the living tissue when the symptom level is the highest for the target disease.

  The operator can adjust various shades of light emitted from the LED lighting device 410 by operating various adjustment knobs, and store the adjustment value in the memory 414e as new adjustment data.

[S34 in FIG. 9 (Calculation of Actual Value)]
In this processing step S34, the transmission chart 408 irradiated with the light emitted from the LED illumination device 410 is photographed by the electronic scope 100, and the photographed image data (RAW format, YUV format, etc.) is input to the PC. Next, the actual measurement value of the transmission chart 408 is calculated from the captured image data of the transmission chart 408 by the calibration software. Illustratively, an average value of pixels (for example, 200 × 200 pixels) in the central region in the image of the transmission chart 408 is calculated as an actual measurement value.

[S35 in FIG. 9 (update of variable i)]
In this processing step S35, the variable i held in the internal memory of the control unit 414b is incremented by one.

[S36 in FIG. 9 (completion judgment)]
In this processing step S36, it is determined whether or not the variable i is n (n is a natural number). When it is determined that the variable i is n (S36: YES), all the multiple (n) transmission charts 408 prepared for the target disease are actually measured in the processing step S34 (calculation of measured values). Value calculation is complete. In this case, the flowchart proceeds to processing step S37 (calculation of correction matrix coefficients).

  When it is determined that the variable i is not n (S36: NO), the actual measurement value is calculated by the processing step S34 (calculation of the actual measurement value) out of the n transmission charts 408 prepared for the target disease. There is nothing left. In this case, the flowchart returns to processing step S32 (transmission chart set). Next, the next transmission chart 408 is set at a predetermined position in the measurement box 402 by the drive mechanism 416, and the processing step S33 (color adjustment of the transmission chart) and the subsequent steps are executed. The transparent chart 408 may be switched by the operator pressing the chart switching button 414f.

[S37 in FIG. 9 (Calculation of correction matrix coefficients)]
In this processing step S37, a correction matrix coefficient is calculated.

In this embodiment, for convenience of explanation, it is assumed that the processing steps S32 to S36 have been executed for two transmission charts 408 (first transmission chart and second transmission chart). The second transmission chart is a color index simulating the color of the living tissue when the target disease is healthy, and is a predetermined point on the mucosa change axis AX2 in the RG plane (second target point described later). It is a chart simulating colors corresponding to P _T2 ). In addition, the actual measurement value of the first transmission chart is the first actual imaging data point _PD1, and the actual measurement value of the second transmission chart is the second actual imaging data point _PD2 . For convenience of explanation, as shown in FIG. 10, the first actual imaging data point P _D1 and the second actual imaging data point P _D2 are associated with the target disease (such as gastritis) here. To place.

As conceptually shown in FIG. 10, the first target point P _T1 corresponding to the first actual imaging data point P _D1 is arranged on the hemoglobin change axis AX1, and the second target point P _T1 is arranged on the mucosa change axis AX2. A second target point P _T2 corresponding to the actual photographing data point P _D2 is arranged. In this processing step S37, the calibration software sets the distance between the first actual photographing data point P _D1 and the first target point P _T1 (first distance Δ ₁ ) and the second actual photographing data point P. A correction matrix coefficient that minimizes the total value of the distance (second distance Δ ₂ ) between _D2 and the second target point P _T2 is calculated using a least square method or the like.

[S38 in FIG. 9 (preservation of correction matrix coefficients)]
In this processing step S38, the correction matrix coefficient calculated in processing step S37 (calculation of correction matrix coefficient) is stored in the correction circuit 220D of the processor 200. Thereby, the calibration process shown in FIG. 9 is completed.

By executing the calibration process for each electronic endoscope system, when the transmission chart 408 related to the target disease (such as gastritis) is photographed with each electronic endoscope system (whichever electronic In the endoscope system as well, a value that approximates the first target point P _T1 and the second target point P _T2 is obtained, so that the finally calculated inflammation evaluation value is also substantially equal. Therefore, it can be seen that even when the target disease (such as gastritis) is actually photographed by each electronic endoscope system, variation in the inflammation evaluation value can be suppressed.

  In other words, according to the present embodiment, by limiting the correction target (specifically, a specific color related to the target disease is set as the correction target), it remains in the color data used for the evaluation value calculation of the target disease. Errors (mainly, variations due to individual differences in the optical parts of the electronic scope 100) are removed satisfactorily. Thereby, the calculation accuracy of the evaluation value is improved.

  Further, according to the present embodiment, the light emitted from the LED illumination device 410 is adjusted to an optimum color for performing calibration using the transmission chart 408 set in the calibration jig 400. . By irradiating the transmissive chart 408 with light adjusted to the optimum hue, the electronic scope 100 captures an image of the optimum color (image of the transmissive chart 408) faithfully reproduced for the target disease. can do. Therefore, the accuracy of calibration is improved.

  The above is the description of the exemplary embodiments of the present invention. Embodiments of the present invention are not limited to those described above, and various modifications are possible within the scope of the technical idea of the present invention. For example, the embodiment of the present application also includes an embodiment that is exemplarily specified in the specification or a combination of obvious embodiments and the like as appropriate.

  In the above embodiment, the operator selects the first transmission chart 408 having the first color (the color of the living tissue when the symptom level is the highest for a predetermined disease), and the second As the transmission chart 408, a chart having the second color (the color of the living tissue when healthy for a given disease) is selected. For this reason, in the above-described embodiment, the color closer to the first color or the second color (that is, the correction target) in the color space is calibrated with higher accuracy. In other words, in the color space, a color that is farther from the correction target (for example, a color that cannot be inflamed, such as light blue) has a lower calibration accuracy.

  Therefore, when the operator selects the transmission chart 408 set on the calibration jig 400 in the calibration mode, the electronic chart system 1 corresponding to the symptom level to be scored with high accuracy is selected. Good. For example, when it is desired to improve the scoring accuracy for mild inflammation, the operator may select a transmission chart 408 that simulates the color of biological tissue when mild inflammation occurs.

  In the above embodiment, the inflammation evaluation value is calculated using the R component and G component (RG two-dimensional color space) included in each pixel. In another embodiment, the two-dimensional color of RG is calculated. By using a two-dimensional color space of RB or a three-dimensional color space such as HSI, HSV, or Lab instead of the space, the target disease (stomach atrophy) different from the above-described embodiment corresponding to each color space. And evaluation values for colorectal tumors and the like can also be calculated. In this case, the correction matrix coefficient is calculated using an index and a target point different from those in the above embodiment.

  The correction circuit 220D of the processor 200 may store a plurality of types of correction matrix coefficients corresponding to various target diseases. By changing the correction matrix coefficient according to the disease to be diagnosed, evaluation value calculation that is stable (small variation due to individual differences) is performed for each target disease.

  In the above-described embodiment, variation in evaluation values due to individual differences in the electronic endoscope system 1 (mainly optical components of the electronic scope 100) is suppressed by using the correction matrix coefficient, but in another embodiment, The variation of evaluation values due to individual differences in the electronic endoscope system 1 may be suppressed by the Hue gain instead of the correction matrix coefficient.

  In the above embodiment, the processor 200 and the calibration jig 400 are separate devices, but the calibration jig 400 may be built in the processor 200.

1 Electronic Endoscope System 100 Electronic Scope 102 LCB
104 Light distribution lens 106 Objective lens 108 Solid-state imaging device 112 Driver signal processing circuit 114 Memory 200 Processor 202 System controller 204 Timing controller 206 Lamp power source igniter 208 Lamp 210 Condensing lens 212 Aperture 214 Motor 216 Driver 218 Operation panel 220 Signal processing circuit 220A Pre-processing circuit 220B Process circuit 220C Output circuit 220D Correction circuit 220E Scoring circuit 220F Mapping circuit 222 Memory 400 Calibration jig 402 Measurement box 402a Inlet 404 Fixing base 404a Mounting member 406 Position adjustment controller 406a Advance key 406b Reverse key 406c Display 408 Transmission type chart 410 LED illumination device 410rL, 410gL, 410 bL LED string 410r Red LED
410g Green LED
410b Blue LED
412 Diffuser plate 414 LED illumination power supply 414a Power supply circuit 414b Control unit 414c Brightness adjustment knob 414rd, 414gd, 414bd Color adjustment knob 414e Memory 414f Chart switching button 416 Drive mechanism VRr, VRg, VRb Variable resistor

Claims (5)

A calibration method for calibrating an electronic scope set in a measurement box,
A set step of setting a transmission chart within the measurement box and within the angle of view of the electronic scope;
An adjustment step of adjusting the transmissive chart to a calibration target color by irradiating the transmissive chart with light having a hue corresponding to the set transmissive chart;
An imaging step of imaging a transmission chart adjusted to a target color by the electronic scope;
A calculation step of calculating a correction value for correcting the value of each pixel constituting the image captured by the electronic scope based on the image data of the captured transmission chart;
A storage step for storing the calculated correction value;
including,
Calibration method. Including a setting step of setting a brightness ratio of each color with respect to a light source capable of independently adjusting the brightness of each of the plurality of colors;
In the setting step,
Set the ratio corresponding to the transmission chart set in the setting step,
In the adjustment step,
By illuminating each color of the light source at a ratio set in the setting step, the transmission chart is irradiated with light having a hue corresponding to the transmission chart set in the setting step.
The calibration method according to claim 1. In the setting step,
Setting the ratio of brightness of each color using data stored in advance in memory;
The calibration method according to claim 2. In the adjustment step,
The transmission chart is adjusted to a predetermined disease color,
The calibration method according to any one of claims 1 to 3. For each of the plurality of transmission charts, execute the set step, the irradiation step and the imaging step,
In the calculating step,
Calculating a correction value for correcting the value of each pixel constituting the image captured by the electronic scope based on the image data of the plurality of captured transmission charts;
The calibration method according to any one of claims 1 to 3.