JP5863377B2 - Fixed pattern noise detection device, removal device, and method thereof - Google Patents

Description

  The present invention relates to an apparatus for detecting and removing fixed pattern noise from a video signal.

  In a general removal method of fixed pattern noise (FPN) contained in a video signal from an image sensor, a dark current component is detected in a light-shielded state, and this is subtracted from the video signal to remove the fixed pattern noise. Yes. In addition, as a method of removing fixed pattern noise specific to an image sensor that depends on temperature change, horizontal scanning is performed at double speed, normal shooting is performed in the first half of the horizontal scanning period, and immediately after resetting in the second half or after resetting. A configuration is also proposed in which a fixed pattern noise corresponding to a light-shielded state is acquired by scanning in (Patent Document 1).

Japanese Patent Laid-Open No. 04-152770

  However, in the conventional method for removing fixed pattern noise, it is necessary to shield the image sensor when detecting fixed pattern noise. In the method of Patent Document 1, reading is performed at high speed for each horizontal line, and thereafter, the video signal is expanded in time to separate fixed pattern noise, and the detected fixed pattern noise is further detected in the first half of the horizontal scanning period. It is necessary to remove from the video signal photographed using the unit, and the processing becomes complicated.

  It is an object of the present invention to detect at a high speed vertical line-shaped fixed pattern noise generated in an image sensor due to the influence of an external device.

  The fixed pattern noise detection apparatus of the present invention includes a first storage unit that stores image data of a horizontal optical black region of an image sensor, and all pattern data of vertical fixed pattern noise generated in the image sensor due to an external circuit. And a pattern specifying means for comparing the image data of the horizontal optical black area with a plurality of pattern data and specifying the pattern data closest to the image data of the horizontal optical black area. And

  The fixed pattern noise detection apparatus preferably includes an addition average calculation unit that calculates an average of a plurality of horizontal lines in the horizontal optical black region, and the image data stored in the first storage unit is calculated by the addition average unit. The

  The number of pixels in the horizontal direction of the horizontal optical black region is preferably equal to or greater than the number of pixels including one period of fixed pattern noise. Further, the horizontal optical black region is preferably a previous horizontal optical black region.

  The fixed pattern noise removing device of the present invention is characterized in that fixed pattern noise is removed from a video signal of an image sensor based on pattern data specified by the fixed pattern detecting device. Also, an electronic endoscope apparatus according to the present invention is characterized by including the fixed pattern noise removing apparatus.

  The fixed pattern noise detection method of the present invention stores image data of the horizontal optical black region of the image sensor, stores all pattern data of the vertical fixed pattern noise generated in the image sensor due to the external circuit, and horizontally The pattern data closest to the image data in the horizontal optical black area is specified by comparing the image data in the optical black area with a plurality of pattern data.

  ADVANTAGE OF THE INVENTION According to this invention, the fixed pattern noise of the vertical line shape which generate | occur | produces in an image pick-up element by the influence of an external device can be detected at high speed.

It is a block diagram which shows roughly the electrical structure of the imaging device of this embodiment. It is a top view which shows the structure of an image pick-up element. It is a conceptual diagram when carrying out the arithmetic mean of the image data of the several horizontal line containing fixed pattern noise. It is a flowchart of a fixed pattern noise detection / removal process in the present embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus including a fixed pattern noise (FPN) removing apparatus according to an embodiment of the present invention.

  The imaging device 10 is, for example, an electronic endoscope device, and a video signal generated by an imaging device (CCD) 11 provided at the distal end of the endoscope insertion portion is subjected to predetermined analog image processing in an analog signal processing circuit 12. After that, the AD conversion circuit 13 converts the digital image data. Image data is input to the FPN correction arithmetic circuit 14 and fixed in the form of vertical lines generated in the image sensor 11 using the extracted data storage memory 15, the pattern data recording memory 16, and the FPN comparison circuit 17 as will be described later. Pattern noise (FPN) is specified (detected).

  Fixed pattern noise detection processing (described later) by the FPN correction arithmetic circuit 14 is performed when the power of the imaging apparatus 10 is turned on. After the fixed pattern noise is identified, the FPN correction arithmetic circuit 14 removes the identified fixed pattern noise from the video signal (image data) from the image sensor 11 and outputs it to the digital signal processing (DSP) circuit 18.

  In the digital signal processing circuit 18, various types of conventionally well-known image processing are performed as necessary, and then the image data is, for example, an external device such as a monitor or a video printer (not shown) or a non-volatile recording medium (not shown). )) As necessary.

  FIG. 2 is a plan view schematically showing the configuration of the image sensor 11. In FIG. 2, a rectangular area 11 </ b> A disposed in the center covered with a vertical line is an effective pixel area that is actually used for shooting an image. An area 11B around the effective pixel area 11A is an optical black (OB) area that is shielded from light and used for determining the pedestal level and the like.

  The optical black regions 11B on both the left and right (horizontal line direction) sides of the effective pixel region 11A are called horizontal optical black regions, and the regions provided above and below (vertical line direction) are called vertical optical black regions. Further, in this specification, a region of the horizontal optical black region that is first read from the image sensor 11 is referred to as a front horizontal optical black region, and the opposite side is referred to as a rear horizontal optical black region. In the present embodiment, a part of the fixed pattern noise included in the video signal from the image sensor 11 is extracted using the pixel data included in the broken line area 19 of the preceding horizontal optical black area.

  Next, a fixed pattern noise detection / removal method according to this embodiment will be described with reference to FIGS.

  In general, the image sensor 11 generates vertical line-shaped fixed pattern noise using a device driven at high speed, such as a PLL in the DSP (18), as a noise source. The fixed pattern noise caused by such an external circuit exhibits the same pattern in each horizontal line. Therefore, if the fixed pattern noise is measured in advance and subtracted from the video signal, the fixed pattern noise caused by the external circuit can be removed from the video signal.

  However, for example, since the phase of the PLL changes every time the power is turned on, the fixed pattern noise appearing in the image sensor 11 also changes every time the power is turned on. Therefore, in order to remove such fixed pattern noise, it is necessary to extract and specify the fixed pattern noise at that time each time the power is turned on.

  On the other hand, fixed pattern noise caused by an external circuit is periodic, and the number of patterns is limited depending on the generated period. Therefore, for example, data of one cycle or more of all patterns including the phase is recorded in the memory 16 in advance, and image data of one cycle or more of fixed pattern noise generated in the image sensor 11 when the power is turned on is extracted. By specifying which pattern it matches, the fixed pattern noise over the entire horizontal line can be specified, and it is not necessary to extract or store the fixed pattern noise over the entire horizontal line.

  For this reason, in the present embodiment, image data including one cycle of fixed pattern noise is extracted from the shielded horizontal optical black region, particularly the preceding horizontal optical black region 19, and stored in the extracted data storage memory 15. By comparing this with the data of the fixed pattern noise measured in advance in the FPN comparison circuit 17 and stored in the memory 16, the fixed pattern noise caused by the external circuit generated in the image sensor 11 is detected and specified.

  However, since each horizontal line includes noise other than fixed pattern noise, it is actually difficult to accurately specify a pattern only from image data for one period of one horizontal line. Therefore, in the present embodiment, image data having the number of pixels including at least one period of the fixed pattern noise is acquired from a plurality of horizontal lines in the horizontal optical black area and stored in the extracted data storage memory 15. Then, in the FPN correction arithmetic circuit 14, for example, as shown in FIG. 3, image data in which noise other than fixed pattern noise is reduced is generated by taking an average of these, and stored in the extracted data storage memory 15.

  In order to sufficiently reduce noise other than fixed pattern noise, it is preferable to take an average of 500 lines or more. However, since the horizontal optical black area includes all horizontal lines, a large number of horizontal line image data can be obtained. Available. Therefore, in this embodiment, fixed pattern noise can be extracted with high accuracy, and a pattern can be specified accurately even by comparison of one cycle. In the present embodiment, after the power is turned on and the power is stabilized, the addition average of all the lines in the preceding horizontal optical black area 19 is calculated, and a comparison with a previously stored pattern is performed in the FPN comparison circuit 17.

  When the comparison in the FPN comparison circuit 17 specifies which pattern of the fixed pattern noise corresponds to the image data obtained as the addition average, the FPN correction calculation circuit 14 uses the image in the shooting of the subject to be started thereafter. The fixed pattern noise specified from the signal (image data of the effective pixel area 11A) is removed, and the video signal after the noise removal is output to a monitor (not shown) or the like via the digital signal processing circuit 18.

  Next, with reference to the flowchart of FIG. 4, a more detailed procedure of the method for identifying and removing fixed pattern noise will be described.

  When power is supplied to the image sensor 11, the digital signal processing circuit 18, etc., the FPN sequence in FIG. 4 is started centering on the FPN correction arithmetic circuit 14. In step S100, image data is read from the image sensor 11, and image data in the previous horizontal optical black (OB) area 19 is stored in the extracted data storage memory 15 for at least the number of pixels corresponding to one period of fixed pattern noise. Is done. The stored image data is added for every vertical line, for example, for all horizontal lines, and the average value is stored in the pattern data recording memory 16 as image data Av. Note that the image data Av corresponds to, for example, the number of images of one period of fixed pattern noise, and noise other than the fixed pattern noise is reduced from the image data Av by the arithmetic mean calculation.

  Next, a pattern detection sequence is started. In steps S102 and S104, pattern data of fixed pattern noises recorded in advance from the pattern data recording memory 16 are sequentially read out, and the pattern data recording memory 16 is read in the FPN comparison circuit 17. The comparison operation with the image data Av stored in is performed. In this embodiment, for example, the pattern data is counted down from the total number N of pattern data, and the pattern data Bn corresponding to the count value n is sequentially read from the pattern data recording memory 16, and the difference Cn (= Av−Bn) from the image data Av. Is calculated and stored in memory. In step S104, it is determined whether or not the count value n has reached 0, that is, whether or not the comparison operation for all pattern data has been completed.

  When the comparison operation of all the pattern data Bn and the image data Av is completed, the absolute value | Cn | of the difference Cn is calculated in step S106, and the pattern data Bn corresponding to the count value n having the smallest absolute value is fixed. It is specified (detected) as pattern noise data.

  In step S108, the FPN correction calculation circuit 14 subtracts the pattern data Bn specified as fixed pattern noise from the image signal (image data) from the image sensor 11. When shooting by the image sensor 11 is started, the fixed pattern noise is removed from the video signal and is output to the digital signal processing circuit 18, and this process ends.

  As described above, the image pickup apparatus according to the present embodiment detects fixed pattern noise caused by an external circuit with a simple configuration without using the image data of the horizontal optical black region to shield the entire image pickup element, and displays this as an image. It can be removed from the signal.

  Further, in the present embodiment, by using the image data of the horizontal optical black area, even when the fixed pattern noise is detected in a state where the effective pixel area is not shielded from light, the image data of the vertical optical black area is used. Compared to the above, it is possible to prevent the influence of smear and the like, and it is possible to more accurately remove fixed pattern noise. Furthermore, since the number of horizontal lines included in the vertical optical black area is small, image data of many fields is required to detect fixed pattern noise with high accuracy using image data in this area. In the embodiment, image data of one or two fields is sufficient, and high-speed processing is possible.

  Furthermore, in this embodiment, only a memory for one period of fixed pattern noise is sufficient, so that memory capacity can be saved.

  In the present embodiment, an electronic endoscope apparatus has been described as an example of an imaging apparatus. However, other than the electronic endoscope, for example, when a fixed pattern noise is detected without a light shielding unit such as a mechanical shutter. The present embodiment can be applied to any imaging device (for example, a mobile phone with a camera function) in which it is difficult to shield the imaging element.

  In the present embodiment, the image data of the front horizontal optical black area is used, but the image data of the rear horizontal optical black area can be used in the same manner. However, in this case, it is necessary to avoid a portion where clamping is performed.

  In the present embodiment, the most appropriate pattern is specified using the difference between the arithmetic average value of the image data of the horizontal optical black region and the pattern data measured and recorded in advance. Other methods for evaluating the data matching degree that are conventionally known may be adopted.

DESCRIPTION OF SYMBOLS 10 Image pick-up device 11 Image pick-up element 11A Effective pixel area | region 11B Optical black area | region 14 FPN correction | amendment arithmetic circuit 15 Memory for extraction data storage (1st memory | storage means)
16 Pattern data recording memory (second storage means)
17 FPN comparison circuit 19 Previous horizontal optical black area

Claims (7)

First storage means for storing image data of a horizontal optical black region of the image sensor;
Second storage means for storing all pattern data of the vertical fixed pattern noise generated in the image sensor due to an external circuit;
Fixed pattern noise detection comprising: pattern specifying means for comparing image data of the horizontal optical black region and the plurality of pattern data to specify pattern data closest to the image data of the horizontal optical black region apparatus. An addition average calculating means for calculating an arithmetic mean of a plurality of horizontal lines of the horizontal optical black region, the image data stored in said first storage means, characterized in that it is calculated by the arithmetic mean calculating means The fixed pattern noise detection apparatus according to claim 1.   The fixed pattern noise detection apparatus according to claim 1, wherein the number of pixels in the horizontal direction of the horizontal optical black region is equal to or greater than the number of pixels including one period of the fixed pattern noise.   The fixed pattern noise detection apparatus according to claim 1, wherein the horizontal optical black region is a previous horizontal optical black region.   The fixed pattern noise removing apparatus, wherein the fixed pattern noise is removed from a video signal of the image sensor based on pattern data specified by the fixed pattern detecting apparatus according to claim 1.   An electronic endoscope apparatus comprising the fixed pattern noise removing apparatus according to claim 5. Stores the image data of the horizontal optical black area of the image sensor,
Stores all pattern data of vertical fixed pattern noise generated in the image sensor due to an external circuit,
A fixed pattern noise detection method comprising: comparing image data of the horizontal optical black region and the plurality of pattern data to identify pattern data closest to the image data of the horizontal optical black region.

Images