JP2016214637A - Image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that a score of a lesioned part varies, and the accuracy is low since it is not possible to image the lesioned part under the same conditions every time.SOLUTION: An image processing device includes: first scoring means for performing scoring of an image being photographed every time a predetermined time elapses and calculating its average value; second scoring means for performing scoring of an image photographed at the time when predetermined operation input is received; and display means for displaying the average value calculated by the first scoring means and a score calculated by the second scoring means on a predetermined display screen together with the image being photographed.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing apparatus that displays a captured image on a predetermined display screen.

  The lesion is generally colored different from normal mucosal tissue. With the improvement in the performance of color endoscope devices, it has become possible to identify lesions slightly different in color from normal tissues. However, it is necessary for the surgeon to receive long-term training under the guidance of an expert in order to be able to accurately identify a lesion from a normal tissue due to a slight color difference on an image captured by an endoscope. is there. Moreover, even a skilled operator cannot easily identify a lesion from a slight color difference, and requires careful work.

  Therefore, for example, in Patent Document 1 and Patent Document 2, as a supplementary information for scoring an image captured by an endoscope and identifying a lesioned part (inflammation, ulcer, etc.), a display screen of a display device The configuration to be displayed in is described. Specifically, the image processing apparatus described in Patent Document 1 or Patent Document 2 is a pixel of a lesioned part based on hue and saturation information for each pixel constituting an image captured by an endoscope. The lesion index (score) is calculated based on the determined number of pixels and displayed on the display screen of the display device.

JP 2014-18332 A JP 2014-18333 A

  For example, in the case of inflammatory bowel disease, a test using an endoscope is performed once every six months. By performing this examination using the endoscope apparatus described in Patent Literature 1 or Patent Literature 2, changes in inflammatory bowel disease can be left as a score that is objective information. However, the score varies depending on the imaging conditions (for example, the imaging distance to the affected area, the imaging composition, the lighting condition (brightness), the imaging technique by the operator, the performance of the device used, the setting of the device used, etc.) . In other words, the score includes a difference in imaging conditions in each examination as an error.

  In order to inspect a lesion more accurately, it is necessary to photograph the lesion under the same conditions every time. However, it is practically impossible to image a lesioned part under the same conditions every time.

  An image processing apparatus according to an embodiment of the present invention is configured to score a captured image every time a predetermined time elapses, and to calculate a mean value of the first scoring unit, and when a predetermined operation input is received. The second scoring means for scoring the image captured in step 1, the average value calculated by the first scoring means, and the score calculated by the second scoring means together with the image being captured Display means for displaying on a predetermined display screen.

  According to one embodiment of the present invention, the average value calculated by the first scoring means and the score calculated by the second scoring means are calculated within one procedure, for example, Differences in shooting conditions during calculation are minimized. Therefore, the surgeon considers the relative relationship between the score of the image captured at the time of receiving a predetermined operation input and the score calculated accurately by averaging, thereby reducing the difference in imaging conditions in each procedure. It can be used as an inspection material after eliminating the influence.

  In one embodiment of the present invention, the first scoring unit may set a higher score weight for the image periphery than for the image center when scoring the image being shot. .

  In one embodiment of the present invention, the second scoring unit may score the image with respect to the center of the image rather than the periphery of the image when scoring the image taken at the time when the predetermined operation input is received. It is good also as a structure which sets a weight high.

  In one embodiment of the present invention, the display means arranges a mask area around an image to be displayed on the display screen, and the average value calculated by the first scoring means on the mask area, The score calculated by the scoring means may be displayed side by side.

  According to an embodiment of the present invention, there is provided an image processing apparatus that can use the score of a lesion as an examination material after eliminating the influence of a difference in imaging conditions in each procedure.

It is a block diagram which shows the structure of the electronic endoscope system which concerns on one Embodiment of this invention. It is a figure which shows the flowchart of the score display process performed by the processor with which the electronic endoscope system which concerns on one Embodiment of this invention is equipped. It is a figure which illustrates a picked-up image. It is a figure which shows the example of a screen displayed on the display screen of a monitor in one Embodiment of this invention. It is a figure which shows the example of a screen displayed on the display screen of a monitor in one Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following, an electronic endoscope system will be described as an example of an embodiment of the present invention.

[Configuration of Electronic Endoscope System 1]
FIG. 1 is a block diagram showing a configuration of an electronic endoscope system 1 according to an embodiment of the present invention. As shown in FIG. 1, the electronic endoscope system 1 includes an electronic scope 100, a processor 200, and a monitor 300.

  The processor 200 includes a system controller 202 and a timing controller 204. The system controller 202 executes various programs stored in the memory 212 and controls the entire electronic endoscope system 1 in an integrated manner. The system controller 202 is connected to the operation panel 214. The system controller 202 changes each operation of the electronic endoscope system 1 and parameters for each operation in accordance with an instruction from the operator input from the operation panel 214. The timing controller 204 outputs a clock pulse for adjusting the operation timing of each unit to each circuit in the electronic endoscope system 1.

  The lamp 208 emits the irradiation light L after being started by the lamp power igniter 206. The lamp 208 is, for example, a high-intensity lamp such as a xenon lamp, a halogen lamp, a mercury lamp, or a metal halide lamp, or an LED (Light Emitting Diode). The irradiation light L is light having a spectrum that spreads mainly from the visible light region to the invisible infrared light region (or white light including at least the visible light region).

  The irradiation light L emitted from the lamp 208 is condensed on the incident end face of an LCB (Light Carrying Bundle) 102 by the condenser lens 210 and is incident on the LCB 102.

  Irradiation light L incident on the LCB 102 propagates through the LCB 102, exits from the exit end face of the LCB 102 disposed at the tip of the electronic scope 100, and irradiates the subject via the light distribution lens 104. The return light from the subject irradiated with the irradiation light L forms an optical image on the light receiving surface of the solid-state image sensor 108 via the objective lens 106.

  The solid-state image sensor 108 is a single-plate color CCD (Charge Coupled Device) image sensor having a Bayer pixel arrangement. The solid-state image sensor 108 accumulates an optical image formed by each pixel on the light receiving surface as a charge corresponding to the amount of light, and generates R (Red), G (Green), and B (Blue) image signals. Output. The solid-state imaging element 108 is not limited to a CCD image sensor, and may be replaced with a CMOS (Complementary Metal Oxide Semiconductor) image sensor or other types of imaging devices. The solid-state image sensor 108 may also be one equipped with a complementary color filter.

  A driver signal processing circuit 110 is provided in the connection portion of the electronic scope 100. An image signal is input to the driver signal processing circuit 110 from the solid-state imaging device 108 in a frame cycle. In the following description, “frame” may be replaced with “field”. In the present embodiment, the frame period and the field period are 1/30 seconds and 1/60 seconds, respectively. The driver signal processing circuit 110 performs predetermined processing on the image signal input from the solid-state image sensor 108 and outputs the processed image signal to the front-stage signal processing circuit 220 and the TE (Tone Enhance) circuit 222 of the processor 200.

  The driver signal processing circuit 110 also accesses the memory 112 and reads the unique information of the electronic scope 100. The unique information of the electronic scope 100 recorded in the memory 112 includes, for example, the number and sensitivity of the solid-state image sensor 108, the operable frame rate, the model number, and the like. The driver signal processing circuit 110 outputs the unique information read from the memory 112 to the system controller 202.

  The system controller 202 performs various calculations based on the unique information of the electronic scope 100 and generates a control signal. The system controller 202 controls the operation and timing of various circuits in the processor 200 using the generated control signal so that processing suitable for the electronic scope connected to the processor 200 is performed.

  The timing controller 204 supplies clock pulses to the driver signal processing circuit 110 according to timing control by the system controller 202. The driver signal processing circuit 110 drives and controls the solid-state imaging device 108 at a timing synchronized with the frame rate of the video processed on the processor 200 side, according to the clock pulse supplied from the timing controller 204.

  The pre-stage signal processing circuit 220 performs predetermined signal processing such as color interpolation, matrix calculation, and Y / C separation on the image signal input from the driver signal processing circuit 110 in one frame period, and the post-stage signal processing circuit 230. Output to.

  The TE circuit 222 performs tone enhancement processing on the image signal input from the driver signal processing circuit 110 in one frame period, and outputs it to the scoring circuit 224. Detailed examples of tone enhancement processing can be referred to in Patent Document 1 and Patent Document 2.

  The tone enhancement process is a process for improving the determination accuracy of the lesion, and roughly gives a non-linear gain (gain) to each primary color signal R, G, B input from the driver signal processing circuit 110. By performing gain adjustment, the dynamic range in the vicinity of the color gamut (particularly the boundary) peculiar to the lesion to be determined is substantially expanded, and the effective resolution of color expression is increased.

  The scoring circuit 224 scores the captured image made up of the image signal based on the image signal after the tone enhancement process input from the TE circuit 222 in one frame period. The scoring process involves objective and quantitative information about the lesion in the captured image (for example, the severity of a non-localized disease such as inflammatory bowel disease or a lesion of a localized disease such as a polyp). This is a process of calculating a lesion index (score) for allowing the surgeon to grasp whether or not he is approaching the part.

  In general, the scoring circuit 224 performs effective pixel determination on each pixel data constituting the image signal of each frame input from the TE circuit 222, and performs a predetermined value such as HSI (Heu-Saturation-Intensity). Percentage of pixels determined to be lesions out of pixels determined to be valid pixels after conversion to color space pixel data, based on the converted pixel data to determine whether the pixel is a lesion (Score) is calculated. Detailed examples of the scoring process can be referred to in Patent Document 1 and Patent Document 2.

  The scoring circuit 224 stores the calculated score of the captured image of each frame in the memory 226 and also the average value of scores within a predetermined period stored in the memory 226 (hereinafter referred to as “average score”). Is output to the subsequent signal processing circuit 230.

  The post-stage signal processing circuit 230 processes the image signal input from the pre-stage signal processing circuit 220 to generate display data of the captured image, and masks the peripheral area of the display screen of the monitor 300 (around the display area of the captured image). Further, the screen data for monitor display is generated by superimposing the average score input from the scoring circuit 224 on the mask area generated by the masking process. The post-stage signal processing circuit 230 converts the generated screen data for monitor display into a predetermined video format signal, and outputs it to the monitor 300. As a result, a captured image of the subject is displayed in the center area of the display screen of the monitor 300, and a screen in which the periphery of the display area of the captured image is masked is displayed. An average score is displayed in the mask area. That is, the average score is displayed at a position that does not overlap the captured image in order to maintain the visibility of the captured image.

  In response to an instruction from the system controller 202, the saved data control circuit 232 saves the screen data output from the subsequent signal processing circuit 230 or the score saved in the memory 226 in the external memory 400. The external memory 400 is, for example, a USB (Universal Serial Bus) memory.

[Score display processing]
FIG. 2 shows a flowchart of a score display process executed by the processor 200 in an embodiment of the present invention. The score display process of FIG. 2 is started when, for example, a predetermined operation instructing the start of execution of this process is performed, and is ended when a predetermined operation instructing the end of the execution is performed. 2 may be executed when the electronic endoscope system 1 is activated, and may be terminated when the electronic endoscope system 1 is stopped, for example.

[S11 in FIG. 2 (calculation of average score)]
When the execution of the score display process of FIG. 2 is started, the score of the captured image of each frame is sequentially stored in the memory 226, and when the execution of the score display process of FIG. Are all erased. In processing step S11, an average value of scores stored in the memory 226, that is, an average score is calculated. Note that the average score is not deleted even after the execution of the score display process of FIG.

  Here, FIG. 3A and FIG. 3B exemplify captured images. FIG. 3A shows a photographed image of a lesioned part, and FIG. 3B shows a photographed image when the electronic scope 100 is inserted into a body cavity and advanced and retracted in the tubular part.

  As shown in FIG. 3A, it is considered that the surgeon normally performs imaging while placing a lesion in the center of the imaging range. That is, imaging by the operator is performed with a composition in which a lesioned part is arranged in the center of the imaging range and a normal part is arranged in the peripheral part of the imaging range. In addition, while the electronic scope 100 is inserted into the body cavity and advanced or retracted in the tubular part (for example, while searching for a lesioned part or while the examination is completed and the electronic scope 100 is returned to the outside of the body cavity), A tube wall portion located around the electronic scope 100 is photographed and appears on the entire circumference of the peripheral portion of the photographing range as shown in FIG. Most of the tube wall portion that appears on the entire circumference of the periphery of the imaging range is a normal portion.

  Therefore, in this processing step S11, a higher weighting coefficient is set for the pixel data in the peripheral part than the pixel data in the central part of the photographing range, and the score of each frame is calculated. For example, a zero weighting factor may be set in the pixel data in the center of the shooting range. As a result, the average score becomes a value that more accurately reflects the state of the normal part, and a value in which sudden errors are eliminated by averaging.

[S12 in FIG. 2 (Display of Average Score)]
FIG. 4 shows a screen example displayed on the display screen of the monitor 300. In this processing step S12, as illustrated in FIG. 4, the average score calculated in processing step S11 (calculation of average score) is displayed on a mask area arranged around the display area of the photographed image. .

[S13 in FIG. 2 (operation determination)]
In this processing step S13, it is determined whether or not a still image shooting operation or a freeze operation has been performed. The score display process of FIG. 2 loops the processing steps S11 to S13 as long as it is determined that the still image shooting operation or the freeze operation is not performed (S13: NO).

[S14 in FIG. 2 (score calculation)]
This processing step S14 is executed when it is determined in processing step S13 (operation determination) that a still image shooting operation or a freeze operation has been performed (S13: YES). In this processing step S 14, the captured image at the time when the input of the still image capturing operation or the freeze operation is received is scored separately from the average score, and the score is stored in the memory 226. Similarly to the average score, the score calculated here is not deleted even after the execution of the score display process of FIG. 2, and remains as the history of the current procedure.

  As described above, imaging by an operator is generally performed with a composition in which a lesion is arranged at the center of the imaging range and a normal part is arranged around the imaging range (see FIG. 3A). Therefore, in this processing step S14, a higher weighting coefficient is set for the pixel data in the central portion than the pixel data in the peripheral portion of the shooting range, and when a still image shooting operation or a freeze operation is received. The score of the captured image is calculated. As a result, the score calculated in this processing step S14 is a value that more accurately reflects the state of the lesioned part by eliminating the influence of the normal part. Hereinafter, for convenience of explanation, the score calculated in this processing step S14 is referred to as “lesion site score”.

[S15 in FIG. 2 (Calculation of Statistical Values)]
In this processing step S15, a predetermined statistical numerical value is calculated. Illustratively, a standard deviation with a population of all scores stored in the memory 226 is calculated, or some scores stored in the memory 226 are extracted as samples, and based on the extracted samples The standard deviation is calculated, and the deviation of the lesion score from the average score is calculated.

[S16 in FIG. 2 (display of lesion score)]
FIG. 5 shows a screen example displayed on the display screen of the monitor 300. In the present processing step S16, as exemplified in FIG. 5, the lesion part score calculated in the processing step S14 (score calculation) is calculated on the mask area arranged around the display area of the captured image and the average score. They are displayed side by side.

  In addition, the surgeon can display the numerical values calculated in the processing step S15 (calculation of statistical numerical values) side by side along with the average score and the lesion part score on the mask area by performing a predetermined setting operation. .

  In addition, the surgeon performs a predetermined operation so that the numerical values calculated in the processing step S15 (calculation of statistical numerical values) and the screen data illustrated in FIGS. It can be stored in the external memory 400.

  According to the present embodiment, since the average score and the lesion part score are calculated within one procedure, it can be said that the difference in imaging conditions at the time of calculation is suppressed as much as possible. Therefore, by considering the relative relationship between the lesion score and the average score, the surgeon eliminates the score error due to the difference in imaging conditions in each procedure, and the lesion site based on the lesion score from which the error has been eliminated. Can be inspected more accurately. For example, even in a test that is performed only once every six months, such as a test for inflammatory bowel disease, the operator can eliminate the difference in lesion score due to the difference in imaging conditions in each procedure. Therefore, it is possible to more accurately determine the progress of the lesion.

  The above is the description of the exemplary embodiments of the present invention. Embodiments of the present invention are not limited to those described above, and various modifications are possible within the scope of the technical idea of the present invention. For example, the embodiment of the present invention also includes contents appropriately combined with embodiments or the like clearly shown in the specification or obvious embodiments.

1 Electronic Endoscope System 100 Electronic Scope 102 LCB
104 Light distribution lens 106 Objective lens 108 Solid-state imaging device 110 Driver signal processing circuit 112 Memory 200 Processor 202 System controller 204 Timing controller 206 Lamp power source igniter 208 Lamp 210 Condensing lens 212 Memory 214 Operation panel 220 Pre-stage signal processing circuit 222 TE circuit 224 Scoring circuit 226 Memory 230 Post-stage signal processing circuit 232 Saved data control circuit 400 External memory

Claims (4)

A first scoring means for scoring an image being shot every time a predetermined time elapses and calculating an average value thereof;
A second scoring means for scoring an image taken at the time of receiving a predetermined operation input;
Display means for displaying the average value calculated by the first scoring means and the score calculated by the second scoring means on a predetermined display screen together with the image being shot;
Comprising
Image processing device. The first scoring means includes
When scoring the image being shot, set a higher score weight for the image periphery than for the image center.
The image processing apparatus according to claim 1. The second scoring means includes
When scoring an image taken at the time when a predetermined operation input is received, the score weight is set higher for the image center than for the image periphery.
The image processing apparatus according to claim 1. The display means includes
Placing a mask area around the image to be displayed on the display screen;
An average value calculated by the first scoring means and a score calculated by the second scoring means are displayed side by side on the mask area.
The image processing apparatus according to any one of claims 1 to 3.

Images