JP2014014022A - Ccd image correction device and ccd image correction method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect noise due to an electronic shutter pulse from a photographed image and to calculate the correction value.SOLUTION: An image is photographed in a CCD 14 of a scope 11 under illumination light from a lamp 21, and it is displayed on a monitor 13 via a processor 12. A lighting control diaphragm 24 is closed, radiation of the illumination light is stopped, and a dark frame for one frame is acquired with an operation of a lamp light switching button arranged in a front panel 29 as a trigger. When a dark frame image is outputted from the CCD 14, a shutter pulse is applied to the CCD 14, and correction values of respective pixels in a line to which noise is added are calculated from a pixel value of the line in a shutter damage detection circuit 31. A special light filter 25 is moved, an observation mode is switched, the illumination light is radiated again and the image is photographed. An image photographed in a pre-stage signal processing circuit 16 is corrected by using the calculated correction value, and the noise due to the shutter pulse is removed from the image.

Description

  The present invention relates to an apparatus and method for removing noise generated in one horizontal line of a CCD image by an electronic shutter pulse.

  In photographing an image using a CCD, an electronic shutter pulse, that is, a charge sweep signal is output during a normal horizontal blanking period. For example, noise is applied to a specific line due to the influence of the charge sweep-out signal, leading to deterioration in image quality. For such a problem, for example, a configuration is known in which a signal of a noise line is interpolated with a signal of a line in succession without using a signal of a line (noise line) in which noise is generated due to the influence of an electronic shutter pulse. (See Patent Document 1).

Japanese Patent Laid-Open No. 2003-092708

  However, interpolation of noise line signals using signals of other lines reduces the original resolution of the image.

  An object of the present invention is to detect noise caused by an electronic shutter pulse from a captured image and calculate a correction value thereof.

  An imaging apparatus according to the present invention includes a homogeneous image acquisition unit that acquires a homogeneous image for at least one frame, and a shutter pulse application unit that applies a shutter pulse to the imaging device when the homogeneous image is output from the imaging device. And a correction value calculating means for calculating a correction value of a pixel of the shutter pulse application line from a pixel value of the shutter pulse application line to which noise due to the shutter pulse is added.

  The correction value is preferably either an offset value or a gain of the pixel value. The homogeneous image is preferably a dark frame image. The dark frame image is acquired by, for example, stopping irradiation of illumination light for photographing. It is preferable that the imaging apparatus further includes an image correction unit that corrects an image captured by the imaging device using the correction value to remove noise.

  Further, the imaging apparatus includes an image output unit that outputs an image captured by the image sensor, an image storage unit that stores an image captured immediately before acquiring the homogeneous image, and an image output during the homogeneous image acquisition period. It is preferable to include an output image switching unit that switches to an image stored in the image storage unit and switches an image output after the homogeneous image acquisition period to an image captured by the image sensor and corrected by the image correction unit. .

  Furthermore, it is preferable that the imaging apparatus includes a homogeneous image determination unit that determines whether or not the image is homogeneous. In addition, the imaging apparatus includes, for example, a first imaging mode in which imaging is performed with illumination light having a first spectral distribution and a second imaging mode in which imaging is performed with illumination light having a second spectral distribution. This is executed during switching between the first and second imaging modes. Further, the imaging apparatus includes, for example, a motion evaluation unit that evaluates the motion of the image, and the homogeneous image acquisition unit is executed when it is evaluated that the motion of the image is small.

  In addition, an electronic endoscope apparatus according to the present invention includes the above-described imaging apparatus.

  According to the present invention, it is possible to detect noise caused by an electronic shutter pulse from a captured image and calculate a correction value thereof.

It is a block diagram which shows the structure of an electronic endoscope system provided with the imaging device of 1st Embodiment. It is a flowchart which shows the flow of the whole electronic endoscope imaging | photography operation | movement in 1st Embodiment. It is a timing chart which shows typically switching of the image output when it switches from normal observation mode to special light observation mode. It is a block diagram which shows the structure of an electronic endoscope system provided with the imaging device of 2nd Embodiment. It is a flowchart which shows the flow of the whole electronic endoscope imaging | photography operation | movement in 2nd Embodiment. 10 is a timing chart schematically showing the timing at which a light shielding (non-lighting) period is provided and the switching of an output image in the second embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of an electronic endoscope system including an imaging apparatus according to the first embodiment of the present invention.

  The electronic endoscope system 10 mainly includes an electronic endoscope apparatus including a scope 11, a processor apparatus 12 to which the scope 11 is detachably connected, and a monitor 13 connected to the processor apparatus 12. A CCD 14 is provided at the distal end of the insertion portion of the scope 11 and is controlled by, for example, a driving pulse (for example, a shutter control signal) from a timing controller 33 provided in the processor device 12. An image photographed by the CCD 14 via the imaging lens 14A is converted into a digital image signal (image data) by an analog / digital converter 15 provided in a connector portion of the scope 11, for example, as an analog image signal, and the processor device 12 The signal is input to the pre-stage signal processing circuit 16.

  The pre-stage signal processing circuit 16 performs predetermined signal processing such as conventionally known color conversion processing, white balance processing, and gamma correction processing, as well as image correction processing described later. The image signal output from the pre-stage signal processing circuit 16 is output to the frame memory 17, the selector 18, and the image discrimination processing circuit 19, respectively. In the frame memory 17, images of a predetermined frame (one frame or more) are always temporarily stored temporarily. In the present embodiment, an image for one frame is stored.

  The selector 18 receives the image signal output from the front-stage signal processing circuit 16 and the image signal output from the frame memory 17, and one of them is sent to the rear-stage video signal processing circuit 20 based on the signal from the timing controller 33. Output. The post-stage video signal processing circuit 20 converts, for example, an image signal into an image signal of a predetermined standard and outputs it to an output device such as a monitor 13. The post-stage video signal processing circuit 20 is controlled by the system controller 28 of the processor device 12.

  In addition, the image output from the pre-stage signal processing circuit 16 to the image discrimination processing circuit 19 is subjected to an image discrimination process described later, and it is determined whether or not the image can be used for correction value calculation. When it is determined that the input image can be used for correction value calculation, the image discrimination processing circuit 19 outputs the image to the shutter flaw detection circuit 31, and the shutter flaw detection circuit 31 extracts an electronic shutter pulse from the input image. Is detected, and a correction value of each pixel value of the line including the noise is calculated. The shutter flaw detection circuit 31 is notified of the timing at which the electronic shutter pulse is output from the timing controller 33, and a line including noise is detected based on the timing.

  The correction value calculated in the shutter flaw detection circuit 31 is output to the previous signal processing circuit 16, and the previous signal processing circuit 16 uses the correction value based on the correction value for the above-described image correction processing. In the present embodiment, the correction value is either an offset value or a gain for each pixel value.

  For imaging with the CCD 14, for example, light from a lamp 21 provided in the processor device 12 is transmitted through the light guide 22 to, for example, the tip of the scope 11 inserted into the body, and this light is irradiated to the subject through the illumination lens 22A. Is done by. The light from the lamp 21 is supplied to the light guide 22 via, for example, a condenser lens 23 and a dimming diaphragm 24. The lamp 21 is controlled to be lit by a lamp control circuit 27, and the diaphragm 24 is driven by a motor 24A to adjust the amount of light from the lamp 21 supplied to the light guide 22.

  The electronic endoscope apparatus according to the present embodiment uses the normal light observation mode in which the light from the lamp 21 is directly used as illumination light through the light guide 22 and the light from the lamp 21 to the light guide 22 through the special light filter 25. Provided with a special light observation mode for photographing under illumination light with a special spectral distribution. The special light filter 25 can be moved back and forth on the optical path by the motor 25A, and is retracted from the optical path in the normal light observation mode, and is disposed on the optical path in the special light observation mode.

  The motors 24A and 25A for driving the light control diaphragm 24 and the special light filter 25 are controlled by a driver 26, and the driver 26 and the lamp control circuit 27 are driven based on a command signal from the timing controller 33, respectively. The controller 33 is controlled by the system controller 28.

  The system controller 28 is connected to a front panel (F panel) 29 provided in the processor device 12, a scope button 30 provided in the scope 11, and the like, and the system controller 28 selects a mode based on these switch operations. Switch and change various settings.

  Next, referring to FIG. 1, FIG. 2, and FIG. 3, the noise detection method, correction value calculation method, image correction method based on the correction value, and noise detection processing in the first embodiment caused by the electronic shutter pulse are being performed. The image display method will be described.

  FIG. 2 is a flowchart showing a flow of the entire electronic endoscope photographing operation in the first embodiment. When the electronic endoscope photographing operation is started, the lamp 21 is turned on in step S100, and in step S102, the selector 18 outputs an image from the previous stage signal processing circuit, and normal image output is started. For example, the normal light observation mode is selected by default, the special light filter 25 is withdrawn from the optical path of the lamp 21, and the endoscopic image photographed by the CCD 14 is converted from the analog / digital converter 15, the pre-stage signal processing circuit 16, and the selector. 18. Displayed in real time on the monitor 13 via the post-stage video signal processing circuit 20.

  On the other hand, when the special light observation mode is already selected, illumination light with a special spectral distribution through the special light filter 25 is irradiated from the distal end of the insertion portion, and the special light is transmitted through the same path as the normal light observation mode. The endoscope image taken below is displayed on the monitor 13 in real time.

  In step S104, it is determined whether or not the user has operated a lamp switching button (mode switching button) provided on the front panel 29 or the scope button 30. If it is not operated, the process returns to step S102, and shooting in the current mode is continued.

  On the other hand, if it is determined that the lamp switching button has been operated, the output of the selector 18 is switched to the image output from the frame memory 17 in step S106, and the monitor 13 displays the image signal stored in the frame memory 17. Output begins. In step S108, the dimming diaphragm 24 is stopped to the maximum value, and the light from the lamp 21 is blocked. That is, irradiation of illumination light from the distal end of the insertion portion is stopped. Thereafter, in step S110, an electronic shutter pulse is output to the CCD 14, and a dark frame image including noise due to the shutter is acquired.

  In step S112, the image determination processing circuit 19 performs image determination for determining whether there is no problem even if the acquired dark frame image is used for calculating the correction value. Since the correction value is calculated by extracting a noise component due to the shutter, the same frame excluding the line including the noise needs to be sufficiently uniform. That is, the pixel values of the pixels other than the line including noise need to be uniform and the change in the value in each region should be small.

  In the image determination of the present embodiment, for example, the standard deviation of the pixel value is equal to or less than a certain value, or the number of pixels that deviate from the analog / digital converter (A / D) offset value by a certain value is equal to or less than a specified number. Any one of them is used as a determination condition, but the present invention is not limited to this as long as the above object can be achieved. The former determination method directly determines the uniformity of the image, and can be used even when the image has intermediate luminance. On the other hand, the latter determination method determines whether or not the image is a dark frame image suitable for correction value calculation (the image is sufficiently dark). An appropriate dark frame image is usually sufficiently uniform. When the former method is employed, the image used for calculating the correction value need not be a dark frame image.

  In the image determination of step S112, the acquired dark frame image is inappropriate for use in correction, that is, a pixel whose standard deviation is larger than a certain value or deviates from the A / D offset value by a certain value or more. Is determined to be larger than the prescribed number, the process proceeds to step S116. On the other hand, if it is determined that there is no problem even if the dark frame image acquired in the image determination is used for correction, in step S114, noise extraction processing and correction value calculation processing in the shutter flaw detection circuit 31 are performed. It is output to the signal processing circuit 16. Noise extraction and correction value calculation are performed by comparing the pixel value of each pixel of the line to which the electronic shutter pulse is applied with the pixel value of other pixels on the same vertical line, for example, the average pixel value. .

  In step S116, the dimming diaphragm 24 is returned to the normal state, that is, the automatic dimming mode, and in step S118, the output of the selector 18 is switched to the image from the preceding signal processing circuit 16 (returned to normal image output). ) At this time, simultaneously with the correction value calculated in step S114, the noise correction by the electronic shutter pulse is executed in the pre-stage signal processing circuit 16.

  In step S120, the lamp light is switched in accordance with the mode switching. That is, in response to the operation of the lamp light switching button in step S104, the normal light observation mode is switched to the special light observation mode, or the special light observation mode is switched to the normal light observation mode, and the motor 25A is switched. Corresponding to the mode, the special optical filter 25 is inserted into or retracted from the optical path. Thereafter, the processing returns to step S102 and the same processing is repeated.

  FIG. 3 is a timing chart schematically showing switching of images output when the normal observation mode is switched to the special light observation mode.

  FIG. 3A shows whether the illumination light emitted from the distal end of the insertion portion is normal light, shielded (no light), or special light. FIG. 3B shows the frame number of the image output from the CCD 14. FIG. 3C shows the frame number of the image output from the frame memory 17 to the selector 18, and FIG. 3D shows the frame number of the image output from the selector 18 to the subsequent video signal processing circuit 20. FIG. 3E shows the timing of lamp light (observation mode) changing operation, and FIG. 3F shows the correction value extraction period.

  As shown in FIG. 3C, the frame memory 17 always outputs an image one frame before the frame acquired by the CCD 14 to the selector 18. In normal image output, the selector 18 outputs the image from the CCD 14 as it is to the post-stage video signal processing circuit 20. However, when the lamp light switching button is operated (pulse P1), the selector 18 performs the post-stage video signal processing circuit. The output to 20 is switched to the image from the frame memory 17. That is, in FIG. 3D, the output from the selector 18 is output as frame numbers 1, 2, and 2, and the same image as the previous frame is output during the light-shielding (no-light) period, during which the correction value is calculated (extracted). ) Is performed (pulse P2). In the subsequent image output, the image from the CCD 14, that is, the images of the frame numbers 4, 5, and 6 are sequentially output from the selector 18, and the image of the frame number 3 (dark frame image) taken during the light shielding (no light) period. ) Is not output to the post-stage video signal processing circuit 20.

  As described above, according to the first embodiment, noise caused by the electronic shutter pulse can be detected during image shooting, and the correction value can be directly calculated. Further, based on this, the pixel value of the line affected by the noise due to the electronic shutter pulse is corrected, and the resolution is not lowered as in the case of using interpolation. As a result, it is possible to constantly correct noise changes due to deterioration of the image sensor over time, temperature changes, etc., and always display an appropriate image. In addition, noise detection is performed when the observation mode is switched, and an image stored in the frame memory during this time is displayed, so that a dark frame image, which is a homogeneous image, is displayed on the monitor, thereby preventing blackouts from occurring. Therefore, it is possible to prevent the monitor observer from feeling uncomfortable. Further, in this embodiment, since an image captured in a state where illumination light is shielded is used for obtaining a dark frame image that is a homogeneous image, no special equipment is required, and noise due to an electronic shutter pulse is simplified with a simple configuration. Can be detected and corrected.

  Next, a second embodiment of the present invention will be described with reference to FIGS. FIG. 4 is a block diagram of the electronic endoscope system according to the second embodiment, and FIG. 5 is a flowchart showing the flow of the entire electronic endoscope photographing operation according to the second embodiment. FIG. 6 is a timing chart schematically showing the timing at which a light shielding (non-lighting) period is provided and the switching of the output image in the second embodiment.

  In the first embodiment, using a lamp light switching button operation as a trigger, a dark frame image is acquired by creating a light-shielding (no-light) period for one frame (predetermined frame) when switching the observation mode. This is because when the observation mode is switched, the content of the screen changes greatly, so even if a past image is output for one frame (or several frames) while a dark frame image is being taken, This is because it does not give a sense of incongruity.

  In the second embodiment, instead of the above-described configuration of the first embodiment, a dark frame image (homogeneous image) is obtained by creating a light shielding (no light) period when there is no significant movement in the image. In other words, when there is little movement of the image, even if the image for the past one frame (or several frames) is output during the dark frame image acquisition, it does not differ greatly from the actual video and gives the viewer a sense of incongruity. There is nothing. Since other configurations of the second embodiment are the same as those of the first embodiment, the same reference numerals are used for the same configurations below, and descriptions thereof are also omitted.

  The electronic endoscope system 40 of the second embodiment shown in FIG. 4 differs from the first embodiment only in the configuration of the processor device. The processor device 41 of the second embodiment is provided with a motion detection circuit 32 in addition to the configuration of the processor device 12 of the first embodiment. Image data is sequentially input from the previous stage signal processing circuit 16 to the motion detection circuit 32, and changes in the images are monitored to detect the motion of the images. For motion detection, for example, a motion vector may be calculated, and an average of the absolute values may be obtained and compared with a threshold value. However, the motion detection is limited to this as long as the motion of the captured image can be evaluated. It is not something.

  The motion detection circuit 32 indicates to the timing controller 33 that the motion of the image is sufficiently small, for example, when the motion of the image is small, for example, when the average value of the absolute value of the motion vector is equal to or less than a threshold value over a predetermined number of frames. Inform. In response to this, the timing controller 33 instructs the driver 26 to squeeze the dimming diaphragm 24 to the maximum, for example, to generate a light shielding (no light) period for one frame.

  Hereinafter, as in the first embodiment, the dark frame image acquired by the CCD 14 is determined by the image determination processing circuit 19, and it is determined whether or not the acquired dark frame image can be sufficiently used for correction value calculation. If it is determined that the signal can be used, the shutter flaw detection circuit 31 detects noise due to the electronic shutter pulse and calculates a correction value thereof. Similarly to the first embodiment, during dark frame acquisition, an image stored in the frame memory 17 is output to the subsequent video signal processing circuit 20 via the selector 18, and thereafter, an image output from the CCD 14 is output. The correction is performed based on the correction value calculated in the upstream signal processing circuit 16 and is output to the downstream video signal processing circuit 20 in real time.

  That is, in the second embodiment, as shown in the flowchart of FIG. 5, the determination (trigger) as to whether or not to perform the dark frame image acquisition process and the noise correction value calculation process is performed based on the lamp light switching button in step S104 of FIG. This operation is replaced with step S204 for determining whether or not the motion of the image is small, and the step S120 for moving the special light filter 25 to switch the lamp light is deleted. The other steps S200, S202, S206 to S218 in FIG. 5 correspond to steps S100, S102, and S106 to S118 in FIG.

  Next, with reference to FIG. 6, a description will be given of the timing at which a light-shielding (no-light) period is provided and the switching of the output image in the second embodiment. 6A to FIG. 6D and FIG. 6F correspond to FIG. 3A to FIG. 3D and FIG. 3F of the first embodiment, and FIG. Indicates whether normal light is applied as illumination light from the distal end of the insertion portion or is shielded (no light). 6B shows the frame number of the image output from the CCD 14, FIG. 6C shows the frame number of the image output from the frame memory 17 to the selector 18, and FIG. The frame number of the image output to the signal processing circuit 20 is shown. FIG. 6F shows a correction value extraction period. On the other hand, FIG. 6E is different from FIG. 3E and shows the result of performing the motion detection process on each frame output from the CCD.

  As shown in FIG. 6E, in the example of FIG. 6, the movements of frame numbers 1 to 6 output from the CCD 14 are detected as large, small, small, small, large, and large, respectively. When the motion detection circuit 32 determines that the motion of the image of frame number 2 is small, the next frame is set as a correction value extraction period (pulse P3), and the light from the lamp 21 to the light guide 22 is controlled by the dimming diaphragm 24. Is stopped for one frame. Thereby, the CCD 14 acquires a dark frame image, and the shutter flaw detection circuit 31 calculates a noise correction value. The selector 18 outputs the image from the frame memory 17 to the post-stage video signal processing circuit 20 only during this one frame period, and outputs the image from the CCD 14 to the post-stage video signal processing circuit 20 as it is during other periods.

  As shown in FIG. 6C, also in the second embodiment, the image output from the frame memory 17 is an image one frame before the frame acquired by the CCD 14, and FIG. ), The output from the selector 18 is output as frame numbers 1, 2, and 2, and the same image as the previous frame is output during the light shielding (no light) period. Thereafter, the output from the selector 18 is switched again to the image from the CCD 14, and the images of frame numbers 4, 5 and 6 are sequentially output from the selector 18. In other words, the image of frame number 3 (dark frame image) captured during the light shielding (no light) period is not output to the subsequent video signal processing circuit 20. Note that this process is not executed again for a predetermined period even if it is determined that the motion of the image detected once is small. In the example of FIG. 6E, the next frame is set in a light-shielded (no light) state when the motion is determined to be small, but in actuality, the motion is determined to be small over a predetermined number of frames. The next frame is put in a light shielding (no light) state.

  As described above, also in the second embodiment, the same effect as in the first embodiment can be obtained. In the second embodiment, noise detection and correction value calculation are performed by electronic shutter pulses in accordance with the magnitude of image movement, so that the present invention can also be applied to an electronic endoscope apparatus that does not have a special light observation mode.

  In this embodiment, a process for acquiring a homogeneous image (such as a dark frame) is performed when the lamp light is switched or when the movement of the image is small. For example, a homogeneous image is acquired and an electronic shutter pulse is acquired. It is also possible to prepare a switch for detecting noise due to the above and calculating a correction value, and to start the process when this switch operation is performed. Moreover, although this embodiment demonstrated the electronic endoscope to the example, it can utilize also in imaging devices other than an electronic endoscope.

10, 40 Electronic endoscope system 11 Scope 12, 41 Processor unit 13 Monitor 14 CCD
DESCRIPTION OF SYMBOLS 16 Prestage signal processing circuit 17 Frame memory 18 Selector 19 Image discrimination processing circuit 21 Lamp 22 Light guide 24 Light control diaphragm 25 Special light filter 28 System controller 29 Front panel 31 Shutter flaw detection circuit 32 Motion detection circuit 33 Timing controller

Claims (10)

Homogeneous image acquisition means for acquiring a homogeneous image for at least one frame;
Shutter pulse applying means for applying a shutter pulse to the image sensor when outputting the homogeneous image from the image sensor;
An image pickup apparatus comprising: a correction value calculating unit that calculates a correction value of a pixel of the shutter pulse application line from a pixel value of the shutter pulse application line to which noise due to the shutter pulse is added.   The imaging apparatus according to claim 1, wherein the correction value is either an offset value or a gain of the pixel value.   The imaging device according to claim 1, wherein the homogeneous image is a dark frame image.   The imaging apparatus according to claim 3, wherein the dark frame image is acquired by stopping the irradiation of illumination light for photographing.   5. The image pickup apparatus according to claim 1, further comprising an image correction unit that corrects an image photographed by the image pickup device using the correction value and removes the noise.   Image output means for outputting an image photographed by the image sensor, image storage means for saving an image photographed immediately before obtaining the homogeneous image, and image preservation of an image output during a homogeneous image acquisition period And an output image switching means for switching the image to be output to the image captured by the image sensor and corrected by the image correction means after the homogeneous image acquisition period is finished. The imaging apparatus according to claim 4.   The imaging apparatus according to any one of claims 1 to 6, further comprising a homogeneous image determination unit that determines whether or not the image is homogeneous.   A first photographing mode for photographing with illumination light having a first spectral distribution; and a second photographing mode for photographing with illumination light having a second spectral distribution, wherein the homogeneous image acquisition means includes the first, The imaging apparatus according to claim 1, wherein the imaging apparatus is executed during switching between the second imaging modes.   7. The apparatus according to claim 1, further comprising a movement evaluation unit that evaluates a movement of the image, wherein the homogeneous image acquisition unit is executed when it is evaluated that the movement of the image is small. The imaging device described.   An electronic endoscope apparatus comprising the imaging apparatus according to claim 1.

Images