JP2007150828A - Fixed pattern noise removing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To remove FPN precisely from a pixel signal outputted from an imaging element with a small number of black pixels. <P>SOLUTION: A noise eliminating device 10 includes a receive unit 11, a comparison and storage control unit 12, a black memory 13, an operation unit 14, and a subtraction unit 15. An image signal and a dark signal are received by the receive unit 11. In the comparison and storage control unit 12, a dark signal is stored in a black memory 13. The storage of the dark signal is performed while comparing with the dark signal, which was already stored in the black memory 13. Two dark signals stored in a prescribed storage region of the black memory 13 are transmitted to the operation unit 14. The operation unit generates an eliminating signal based on two dark signals. In the subtraction unit 15, the eliminating signal is subtracted from the pixel signal so that the FPN included in the pixel signal is removed. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a fixed pattern noise removing apparatus that removes fixed pattern noise from an image signal generated by an image sensor.

  The pixel signals output from the effective pixels that receive the subject luminous flux are required to be equal if they have the same received light amount in the same use environment, for example, the received light amount in the dark. However, noise called fixed pattern noise (FPN) is mixed due to manufacturing errors.

  Therefore, the FPN is removed by subtracting the dark signal generated in the black pixel shielded from the aperture from the pixel signal output from the effective pixel (see Patent Document 1).

  In addition, FPN due to a difference in transfer path may be mixed in the pixel signal. Therefore, a black pixel is provided for each transfer path, and FPN is removed by subtracting a dark signal generated in a black pixel common to the effective pixel and the transfer path from the pixel signal output from the effective pixel.

  Therefore, if a black defect or a white defect occurs in a black pixel, the FPN cannot be accurately removed from the pixel signal. Therefore, conventionally, a plurality of black pixels are connected to each transfer path, and an average value of dark signals output from each transfer path is calculated.

However, in order to reduce the influence of the defect of the black pixel, it is necessary to connect many black pixels for each transfer path, so that the area where the effective pixels are provided is narrow. On the other hand, it may be possible to generate a dark signal of a defective black pixel by interpolation processing, such as pixel interpolation in effective pixels, but if interpolation processing is performed using dark signals with different transfer lines, the FPN can be accurately determined. It was not possible to obtain a dark signal for removal.
JP 2004-15712 A

  Accordingly, an object of the present invention is to provide a fixed pattern noise removing device that accurately removes FPN from a pixel signal output from an image pickup device having few black pixels.

  The fixed pattern noise removing device of the present invention includes an effective pixel that generates a pixel signal according to the amount of received light, a plurality of black pixels and an effective pixel that generate a dark signal corresponding to a pixel signal according to the amount of light received during darkness. A receiving unit that receives a pixel signal and a dark signal from an imaging unit that is connected to a plurality of black pixels and has a transfer path that outputs a pixel signal and a plurality of dark signals; And an extraction unit for extracting other dark signals excluding at least one of the minimum dark signals, and a removal signal for removing fixed pattern noise included in the pixel signal based on the dark signals extracted by the extraction unit A removal signal output unit for outputting, and a removal unit for removing fixed pattern noise from the pixel signal based on the removal signal are provided.

  Furthermore, it is preferable that the extraction unit extracts dark signals excluding both dark signals having the maximum and minimum signal levels from among the plurality of dark signals.

  Note that the imaging unit includes a plurality of transfer paths each connecting an effective pixel and a plurality of black pixels, and the extraction unit selects from dark signals output from the plurality of black pixels connected to the common transfer path. The removal signal output unit performs extraction and outputs a removal signal for removing fixed pattern noise included in pixel signals generated in effective pixels connected to a common transfer path based on the dark signal extracted by the extraction unit. Preferably, the removal unit removes the fixed pattern noise from the pixel signal generated by the effective pixels connected to the common transfer path based on the removal signal.

  Further, there are a plurality of dark signals extracted by the extraction unit, and the removal signal output unit generates and outputs a removal signal by calculating an average value of some or all of the dark signals extracted by the extraction unit. It is preferable. Alternatively, the extraction unit extracts a dark signal having a median signal level from the plurality of dark signals received by the reception unit, and the removal signal output unit outputs the dark signal extracted to the extraction unit as a removal signal. It is preferable.

  Further, the extraction unit includes a black memory having a plurality of storage areas for storing a dark signal, a signal level of the dark signal already stored in the storage area, and a signal level of the dark signal newly received by the receiving unit A comparison unit for comparing the magnitudes of the signal level, the dark signal already stored in the storage area according to the comparison result in the comparison unit, and the dark signal newly received by the reception unit in the order of the signal level. After storage of dark signals generated in a plurality of black pixels connected to a common transfer path during the period of one frame or one field by the storage processing unit stored in each storage area determined accordingly And a selection unit that extracts a dark signal stored in a predetermined storage area.

  Further, the number of storage areas is smaller than the number of black pixels connected to a common transfer path, and the storage processing unit stores a dark signal in all of the plurality of storage areas during one frame or one field. When a new dark signal is received by the receiver, the dark signal having the maximum signal level among the dark signal newly received by the receiver and the dark signal stored in the storage area may be deleted. preferable.

  In addition, the selection unit receives all dark signals output from the black pixels connected to the common transfer path during one frame or one field period, and then receives the dark signals stored in all the storage areas. Among the signals, it is preferable to extract a dark signal having at least a maximum signal level.

  Further, the number of storage areas is smaller than the number of black pixels connected to a common transfer path, and the storage processing unit stores a dark signal in all of the plurality of storage areas during one frame or one field. When a new dark signal is received by the receiving unit, the dark signal having the minimum signal level among the dark signal newly received by the receiving unit and the dark signal stored in the storage area may be deleted. preferable.

  In addition, the selection unit receives all dark signals output from the black pixels connected to the common transfer path during one frame or one field period, and then receives the dark signals stored in all the storage areas. It is preferable to extract a dark signal having a minimum signal level from the signal.

  Further, the number of storage areas is smaller than the number of black pixels connected to a common transfer path, and the storage processing unit stores a dark signal in all of the plurality of storage areas during one frame or one field. When the dark signal is newly received by the receiver at the first timing, the dark signal having the maximum signal level among the dark signal newly received by the receiver and the dark signal stored in the storage area Between the dark signal newly received by the receiver and the dark signal stored in the storage area when a new dark signal is received by the receiver at the second timing after the first timing. It is preferable to delete the dark signal having the minimum signal level.

  Further, the selection unit receives the dark signals stored in all the storage areas after receiving all the dark signals output from the black pixels connected to the common transfer path during the period of one frame or one field to the receiving unit. Among them, it is preferable to extract a dark signal in a storage area in which a dark signal having a median signal level is stored, and the removal signal output unit outputs the dark signal extracted to the selection unit as a removal signal.

  Further, the selection unit receives the dark signals stored in all the storage areas after receiving all the dark signals output from the black pixels connected to the common transfer path during the period of one frame or one field to the receiving unit. Preferably, the removal signal output unit generates and outputs a removal signal by calculating an average value of the dark signals extracted by the selection unit.

  According to the present invention, the removal signal is generated based on the dark signal from which at least one of the dark signals having the maximum or minimum signal level among the plurality of dark signals is removed, and the pixel signal is generated using the generated removal signal. FPN can be removed from Therefore, it is possible to remove the FPN with high accuracy by eliminating the influence of the white defect or black defect of the black pixel with respect to the pixel signal output from the imaging device having few black pixels.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram schematically showing a configuration of an imaging unit having a fixed pattern noise removing apparatus to which the first embodiment of the present invention is applied. The imaging unit 20 includes an imaging element 30, an A / D converter 21, a noise removal device 10, and a signal processing circuit 22.

  The image sensor 30 is, for example, a CMOS image sensor. An effective pixel area EA and a black pixel area BA are provided on the light receiving surface of the image sensor 30. In the effective pixel area EA, a plurality of effective pixels (not shown in FIG. 1) having photodiodes (PD) are two-dimensionally arranged. In the black pixel area BA, black pixels (not shown in FIG. 1) having light-shielded PDs are arranged two-dimensionally.

  When light forming the subject image is incident on the light receiving surface, a pixel signal corresponding to the amount of received light is generated in each effective pixel. A dark signal corresponding to a dark pixel signal is generated in each black pixel.

  A pixel signal of one field or one frame generated in a plurality of effective pixels arranged in the effective pixel area EA is sent from the image sensor 30 to the A / D converter 21 as an image signal. A dark signal is also sent to the A / D converter 21. The image signal and the dark signal are converted from an analog signal to a digital signal by the A / D converter 21.

  The image signal and the dark signal converted into the digital signal are sent to the noise removing device 10. In the noise removal apparatus 10, the FPN is removed from the image signal using the dark signal. The image signal from which the FPN has been removed is sent to the signal processing circuit 22 where predetermined signal processing is performed.

  Next, the configuration of the light receiving surface of the image sensor 30 will be described with reference to FIG. FIG. 2 is a block diagram illustrating the configuration of the light receiving surface of the image sensor 30. As described above, the effective pixel area EA and the black pixel area BA are provided on the light receiving surface.

  As described above, the effective pixels 31 are arranged in a matrix so that the effective pixels 31 are formed in the effective pixel area EA and the black pixels 32 are formed in the black pixel area BA. A vertical signal line 33 is provided for each column in which the effective pixels 31 and the black pixels 32 are arranged vertically.

  Effective pixels 31 and black pixels 32 adjacent to the left side of the vertical signal line 33 are connected to the vertical signal line 33. Seven black pixels 32 are connected to the common vertical signal line 33. The vertical signal line 33 in each column is connected to the horizontal signal line 34. The horizontal signal line 34 is connected to the output amplifier 35. The pixel signal generated in the effective pixel 31 and the dark signal generated in the black pixel 32 are output from the image sensor 30 via the vertical signal line 33, the horizontal signal line 34, and the output amplifier 35.

  In the present embodiment, a single horizontal signal line 34 and a single output amplifier 35 are provided. However, the same number of output amplifiers 35 as the plurality of horizontal signal lines 34 and vertical signal lines 33 are provided. May be. With the configuration in which a plurality of horizontal signal lines 34 are provided, the reading speed of the image signal from the image sensor 30 can be improved. For example, when two horizontal signal lines 34 and an output amplifier 35 are provided, the reading speed is improved twice.

  Note that the pixel signals and dark signals generated in the effective pixels 31 and the black pixels 32 are generated and output by a shift register (not shown), and the timing is controlled one by one from the image sensor 30. Is output.

  The output of the pixel signal and the dark signal is performed for each vertical signal line 33. For example, when outputting a pixel signal and dark signal of one frame or one field, signals are read from the effective pixel 31 and the black pixel 32 connected to the vertical signal line 33 at the left end of the light receiving surface. Thereafter, signals are read out for each vertical signal line 33 in order from left to right.

  Further, reading of signals on each vertical signal line 33 is performed in the order from the black pixel 32 at the lower end to the effective pixel 31 at the upper end. Therefore, the dark signal from the black pixel 32 is read out before the pixel signal from the effective pixel 31 is read out.

  Next, the internal configuration of the noise removal apparatus 10 will be described in detail with reference to FIG. FIG. 3 is a block diagram schematically showing the internal configuration of the noise removing apparatus 10. The noise removal apparatus 10 includes a reception unit 11, a comparison / storage control unit 12 (comparison unit, storage processing unit), a black memory 13, a calculation unit 14 (selection unit, removal signal output unit), and a subtraction unit 15. .

  The pixel signal and the dark signal converted into digital signals by the A / D converter 21 are received by the receiving unit 11. The dark signal received by the receiving unit 11 is sent to the comparison / storage control unit 12. Further, the pixel signal received by the receiving unit 11 is sent to the subtracting unit 15. The output destination of the signal sent from the receiving unit 11 is controlled by a system controller (not shown).

  As will be described later, a removal signal is generated by the black memory 13 and the calculation unit 14 based on the dark signal sent to the comparison / storage control unit 12. The generated removal signal is sent to the subtraction unit 15. The generated removal signal is used to remove FPN from the pixel signal of the effective pixel 31. A removal signal is generated for each dark signal from the black pixels 32 connected to the common vertical signal line 33 and is used for FPN removal of the pixel signals of the effective pixels 31 connected to the same vertical signal line 33.

  When the removal signal is sent to the subtraction unit 15, the pixel signal of the effective pixel connected to the common vertical signal line 33 is sent from the reception unit 11 to the subtraction unit 15. By subtracting the signal level of the removal signal from the signal level of the pixel signal, the FPN is removed from the pixel signal. The pixel signal from which the FPN has been removed is output as a correction signal from the noise removing device 10 to the signal processing circuit 22 as described above.

  Next, after the configuration of the black memory 13 is described, processing performed in the comparison / storage control unit 12, the black memory 13, and the calculation unit 14 to generate a removal signal from the dark signal will be described.

  As shown in FIG. 4, the black memory 13 is provided with first, second, third, and fourth storage units 13a, 13b, 13c, and 13d. A plurality of storage areas are provided in each of the first, second, third, and fourth storage units 13a, 13b, 13c, and 13d. Each storage area in each of the first, second, third, and fourth storage units 13a, 13b, 13c, and 13d is assigned to one of the vertical signal lines 33.

  The leftmost vertical signal line 33 is the first column, and in the following description, the storage area allocated to the nth column vertical signal line 33 in the first storage unit 13a is displayed as RA (n, 1). (N is a natural number less than or equal to x). Similarly, the storage area allocated to the n-th column vertical signal line 33 in the second storage unit 13b is displayed as RA (n, 2).

  Similarly, the storage area assigned to the nth column vertical signal line 33 in the third storage unit 13c is displayed as RA (n, 3), and the nth column vertical signal is displayed in the fourth storage unit 13d. The storage area allocated to the signal line 33 is displayed as RA (n, 4).

  The storage area RA (n, 1) in the first storage unit 13a has the lowest signal level among the plurality of dark signals generated in the plurality of black pixels 32 connected to the common vertical signal line 33. A dark signal is stored. The storage area RA (n, 2) in the second storage unit 13b has the second lowest signal level among the plurality of dark signals generated in the black pixels 32 connected to the common vertical signal line 33. A dark signal is stored.

  The storage area RA (n, 3) in the third storage unit 13c has the third smallest signal level among the plurality of dark signals generated in the black pixels 32 connected to the common vertical signal line 33. A dark signal is stored. The storage area RA (n, 4) in the fourth storage unit 13d has the fourth lowest signal level among the plurality of dark signals generated in the black pixels 32 connected to the common vertical signal line 33. A dark signal is stored.

  When a dark signal is input from the receiving unit 11 to the comparison / storage control unit 12, the comparison / storage control unit 12 newly adds the signal level of the dark signal stored in the black memory 13 before reception by the receiving unit 11. The level of the input dark signal is compared with the signal level.

  Based on the comparison result, the comparison / storage control unit 12 stores the dark signal having the lowest signal level in the storage area RA (n, 1) in the first storage unit 13a. The second smallest dark signal is stored in the storage area RA (n, 2) in the second storage unit 13b. The third smallest dark signal is stored in the storage area RA (n, 3) in the third storage unit 13c. The third smallest dark signal is stored in the storage area RA (n, 4) in the fourth storage unit 13d.

  When dark signals are stored in the storage areas in all the storage units 13a, 13b, 13c, and 13d before a dark signal is newly received, the newly input dark signal and the already stored dark signal are stored. The dark signal having the largest signal level is deleted.

  When the comparison / storage control unit 12 compares the signal levels and stores the dark signals of all the black pixels 32 connected to the common vertical signal line 33 based on the comparison result, a removal signal is generated next. Is done.

  As described above, after the signal level comparison and storage, the dark signals stored in the storage areas RA (n, 3) and RA (n, 4) in the third and fourth storage units 13c and 13d are calculated. 14. In the calculation part 14, the average value of the signal level of the transmitted dark signal is calculated. A removal signal that is the signal level of the calculated average value is sent to the subtraction unit 15. As described above, the removal signal is used in the subtraction unit 15 to remove the FPN from the pixel signal.

  The FPN removal process performed in the noise removal apparatus as described above will be described in more detail with reference to the flowcharts of FIGS. FIG. 5 is a flowchart showing an overall outline of the FPN removal process. FIG. 6 is a flowchart illustrating a process of determining the row of black pixels that has generated the received dark signal.

  FIG. 7 is a first flowchart showing a process of storing a dark signal in the black memory. FIG. 8 is a second flowchart showing the process of storing the dark signal in the black memory. FIG. 9 is a third flowchart showing the process of storing the dark signal in the black memory. FIG. 10 is a fourth flowchart showing the process of storing the dark signal in the black memory. FIG. 11 is a fifth flowchart showing the process of storing the dark signal in the black memory.

  In step S100, the receiving unit 11 receives a pixel signal or a dark signal. In the next step S101, it is confirmed whether the signal newly received by the receiving unit 11 is a pixel signal or a dark signal. If it is a dark signal, the process proceeds to step S104.

  If it is a pixel signal in step S101, the process proceeds to step S102. As described above, the image sensor 30 outputs the dark signal of the black pixel 32 first on each vertical signal line 33 and then outputs the pixel signal of the effective pixel 31 common to the vertical signal line 33 thereafter. Driven. In addition, among the plurality of black pixels 32 connected to the common vertical signal line 33, the image sensor 30 is driven so as to sequentially output the dark signal of the black pixel at the bottom end.

  Therefore, the process proceeds to step S102 when the process up to step S132 is completed as described later, and in the first, second, third, and fourth storage units 13a, 13b, 13c, and 13d. Dark signals are stored in all the storage areas RA (n, 1), RA (n, 2), RA (n, 3), RA (n, 4).

  In step S102, the average value of the signal levels of the dark signals stored in the storage areas RA (n, 3) and RA (n, 4) in the third and fourth storage units 13c and 13d is calculated. A signal having an average signal level is generated as a removal signal. After the generation of the removal signal, the process proceeds to step S103.

  In step S103, a correction signal from which the FPN has been removed from the pixel signal is generated by subtracting the signal level of the removal signal from the pixel signal. After the subtraction process in step S103 ends, the FPN removal process ends.

  Next, processing in the case where a dark signal is received in step S101 will be described. If a dark signal is received, the process proceeds to step S104. In steps S104 to S107, a determination is made as to which column the black pixel 32 that generated the received dark signal is counted from the lowest end (see FIG. 6).

  That is, in step S104, it is determined whether or not it is the first column. If it is the first column, the process proceeds to step S108 (see FIG. 7). In step S105, it is determined whether or not it is the second column. If it is the second column, the process proceeds to step S109 (see FIG. 8). In step S106, it is determined whether or not it is the third column. If it is the third column, the process proceeds to step S112 (see FIG. 9). Further, in step S107, it is determined whether or not it is the fourth column. If it is the fourth column, the process proceeds to step S117 (see FIG. 10). If it is a dark signal generated from the black pixels 32 in the fourth and subsequent columns in step S107, the process proceeds to step S124 (see FIG. 11).

  As shown in FIG. 7, when the process proceeds to step S108, the dark signal generated and received at the black pixels 32 in the first column, that is, first read out from the plurality of black pixels 32 connected to the common vertical signal line 33. The dark signal to be stored is stored in the storage area RA (n, 1) in the first storage unit 13a. After storing the dark signal, the process returns to step S100.

  As shown in FIG. 8, when the process proceeds to step S109, the signal level of the dark signal generated and received at the black pixel 32 in the second column and already stored in the storage area RA (n, 1) in the first storage unit 13a. The signal level of the dark signal is compared.

  When the signal level of the received dark signal is smaller, the process proceeds to step S110. In step S110, the dark signal stored in the storage area RA (n, 1) in the first storage unit 13a is stored in the storage area RA (n, 2) in the second storage unit 13b. The newly received dark signal is stored in the storage area RA (n, 1) in the first storage unit 13a. After storing, the process returns to step S100.

  In step S109, when the signal level of the received dark signal is higher, the process proceeds to step S111. In step S111, the newly received dark signal is stored in the storage area RA (n, 2) in the second storage unit 13b. After storing, the process returns to step S100.

  As shown in FIG. 9, when the process proceeds to step S112, the signal level of the dark signal generated and received at the black pixel 32 in the third column and already stored in the storage area RA (n, 1) in the first storage unit 13a. The signal level of the dark signal is compared.

  When the signal level of the received dark signal is lower, the process proceeds to step S113. In step S113, the dark signal stored in the storage area RA (n, 1) in the first storage unit 13a and the dark signal stored in the storage area RA (n, 2) in the second storage unit 13b are respectively The data is stored in the storage area RA (n, 2) in the second storage unit 13b and the storage area RA (n, 3) in the third storage unit 13c. The newly received dark signal is stored in the storage area RA (n, 1) in the first storage unit 13a. After storing, the process returns to step S100.

  In step S112, if the received dark signal has a higher signal level, the process proceeds to step S114. In step S114, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA (n, 2) in the second storage unit 13b.

  If the signal level of the received dark signal is lower, the process proceeds to step S115. In step S115, the dark signal stored in the storage area RA (n, 2) in the second storage unit 13b is stored in the storage area RA (n, 3) in the third storage unit 13c. The newly received dark signal is stored in the storage area RA (n, 2) in the second storage unit 13b. After storing, the process returns to step S100.

  If the signal level of the received dark signal is higher in step S114, the process proceeds to step S116. In step S116, the newly received dark signal is stored in the storage area RA (n, 3) in the third storage unit 13c. After storing, the process returns to step S100.

  As shown in FIG. 10, when the process proceeds to step S117, the signal level of the dark signal generated and received at the black pixel 32 in the fourth column and already stored in the storage area RA (n, 1) in the first storage unit 13a. The signal level of the dark signal is compared.

  When the signal level of the received dark signal is lower, the process proceeds to step S118. In step S118, the dark signal stored in the storage area RA (n, 1) in the first storage unit 13a, the dark signal stored in the storage area RA (n, 2) in the second storage unit 13b, and the first The dark signals stored in the storage area RA (n, 3) in the third storage section 13c are respectively stored in the storage area RA (n, 2) in the second storage section 13b and the storage area RA (in the third storage section 13c). n, 3) and the storage area RA (n, 4) in the fourth storage unit 13d. The newly received dark signal is stored in the storage area RA (n, 1) in the first storage unit 13a. After storing, the process returns to step S100.

  If the received dark signal has a higher signal level in step S117, the process proceeds to step S119. In step S119, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA (n, 2) in the second storage unit 13b.

  When the signal level of the received dark signal is lower, the process proceeds to step S120. In step S120, the dark signals stored in the storage area RA (n, 2) in the second storage unit 13b and the storage area RA (n, 3) in the third storage unit 13c are respectively converted into the third storage unit 13c. In the storage area RA (n, 3) and the storage area RA (n, 4) in the fourth storage unit 13d. The newly received dark signal is stored in the storage area RA (n, 2) in the second storage unit 13b. After storing, the process returns to step S100.

  If the received dark signal has a higher signal level in step S119, the process proceeds to step S121. In step S121, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA (n, 3) in the third storage unit 13c.

  When the signal level of the received dark signal is lower, the process proceeds to step S122. In step S122, the dark signal stored in the storage area RA (n, 3) in the third storage unit 13c is stored in the storage area RA (n, 4) in the fourth storage unit 13d. Further, the newly received dark signal is stored in the storage area RA (n, 3) in the third storage unit 13c. After storing, the process returns to step S100.

  In step S121, when the signal level of the received dark signal is higher, the process proceeds to step S123. In step S123, the newly received dark signal is stored in the storage area RA (n, 4) in the fourth storage unit 13d. After storing, the process returns to step S100.

  As shown in FIG. 11, when the process proceeds to step S124, the signal level of the dark signal generated and received in the black pixels 32 in the fifth column and thereafter and the storage area RA (n, 1) in the first storage unit 13a are already stored. The signal level of the stored dark signal is compared.

  When the signal level of the received dark signal is lower, the process proceeds to step S125. In step S125, first, the dark signal stored in the storage area RA (n, 4) in the fourth storage unit 13d is deleted. Next, the dark signal stored in the storage area RA (n, 1) in the first storage unit 13a, the dark signal stored in the storage area RA (n, 2) in the second storage unit 13b, and the first The dark signals stored in the storage area RA (n, 3) in the third storage unit 13c are respectively stored in the storage area RA (n, 2) in the second storage unit 13b and the storage area RA ( n, 3) and the storage area RA (n, 4) in the fourth storage unit 13d. The newly received dark signal is stored in the storage area RA (n, 1) in the first storage unit 13a. After storing, the process returns to step S100.

  In step S124, when the signal level of the received dark signal is higher, the process proceeds to step S126. In step S126, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA (n, 2) in the second storage unit 13b.

  When the signal level of the received dark signal is lower, the process proceeds to step S127. In step S127, first, the dark signal stored in the storage area RA (n, 4) in the fourth storage unit 13d is deleted. Next, the dark signal stored in the storage area RA (n, 2) in the second storage unit 13b and the dark signal stored in the storage area RA (n, 3) in the third storage unit 13c are respectively The data is stored in the storage area RA (n, 3) in the third storage unit 13c and the storage area RA (n, 4) in the fourth storage unit 13d. The newly received dark signal is stored in the storage area RA (n, 2) in the second storage unit 13b. After storing, the process returns to step S100.

  In step S126, when the signal level of the received dark signal is higher, the process proceeds to step S128. In step S128, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA (n, 3) in the third storage unit 13c.

  If the signal level of the received dark signal is lower, the process proceeds to step S129. In step S129, first, the dark signal stored in the storage area RA (n, 4) in the fourth storage unit 13d is deleted. Next, the dark signal stored in the storage area RA (n, 3) in the third storage unit 13c is stored in the storage area RA (n, 4) in the fourth storage unit 13d. Further, the newly received dark signal is stored in the storage area RA (n, 3) in the third storage unit 13c. After storing, the process returns to step S100.

  If the signal level of the received dark signal is higher in step S128, the process proceeds to step S130. In step S130, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA (n, 4) in the fourth storage unit 13d.

  If the signal level of the received dark signal is lower, the process proceeds to step S131. In step S132, first, the dark signal stored in the storage area RA (n, 4) in the fourth storage unit 13d is deleted. The newly received dark signal is stored in the storage area RA (n, 4) in the fourth storage unit 13d. After storing, the process returns to step S100.

  In step S130, when the signal level of the received dark signal is higher, the process proceeds to step S132. In step S132, the newly received dark signal is deleted. After deletion, the process returns to step S100.

  As described above, the storage areas RA (n, 1) and RA (n, 2 in the first, second, third, and fourth storage units 13a, 13b, 13c, and 13d are obtained by the processes in steps S104 to S132. ), RA (n, 3), and RA (n, 4) are all stored with dark signals.

  According to the noise removal apparatus of the present embodiment having the above-described configuration, when the black pixel 32 connected to the common vertical signal line 33 has a black defect or a white defect, the influence of the defect can be removed. It becomes possible. Therefore, FPN can be accurately removed from the pixel signal of the effective pixel 31 without widening the black pixel area BA.

  Next, the noise removal apparatus of 2nd Embodiment is demonstrated. In the present embodiment, the dark signal stored in the first to fourth storage units and the dark signal storage processing method by the comparison / storage control unit are different from those in the first embodiment. Hereinafter, a description will be given focusing on differences from the first embodiment. In addition, the same code | symbol is attached | subjected to the site | part which has the same function as 1st Embodiment.

  The noise removing device 10 is configured by a receiving unit 11, a comparison / storage control unit 12 (comparison unit, storage processing unit), a black memory 13, a calculation unit 14 (selection unit, removal signal output unit), and a subtraction unit 15. Is the same as in the first embodiment.

  The dark signal is received by the receiving unit 11 and sent from the receiving unit 11 to the comparison / storage control unit 12 as in the first embodiment. Further, the pixel signal is received by the receiving unit 11 and sent to the subtracting unit 15 as in the first embodiment.

  As will be described later, the dark signal is stored in the black memory 13 from the comparison / storage control unit 12 by a method different from that of the first embodiment. The removal signal is generated by the black memory 13 and the calculation unit 14 based on the stored dark signal, and the FPN is removed from the pixel signal based on the generated removal signal, as in the first embodiment. .

  As in the first embodiment, the black memory 13 includes first, second, third, and fourth storage units 13a, 13b, 13c, and 13d. Each storage unit 13a, 13b, 13c, 13d is provided with a plurality of storage areas, and each storage area is assigned to one of the vertical signal lines 33. In this embodiment, the storage area assigned to the vertical signal line 33 in the n-th column in the first storage unit 13a is displayed as RA '(n, 1).

  In the present embodiment, the storage areas RA ′ (n, 1), RA ′ (n, 2), RA ′ (n, n, n) in the first, second, third, and fourth storage units 13a, 13b, 13c, and 13d. 3) The dark signal stored in RA ′ (n, 4) is different from that in the first embodiment. In the present embodiment, among the plurality of dark signals generated in the plurality of black pixels 32 connected to the common vertical signal line 33, the dark signal having the maximum signal level is stored in the storage area RA in the fourth storage unit 13d. '(N, 4).

  Similarly, the dark signal having the second highest signal level is stored in the storage area RA '(n, 3) in the third storage unit 13c. The dark signal having the third highest signal level is stored in the storage area RA ′ (n, 2) in the second storage unit 13b. In addition, the dark signal having the fourth highest signal level is stored in the storage area RA ′ (n, 1) in the first storage unit 13a.

  When a dark signal is input from the receiving unit 11 to the comparison / storage control unit 12, the signal level of the dark signal stored in the black memory 13 before reception by the receiving unit 11 and the signal level of the newly received dark signal Based on the comparison result, the newly received dark signal and the previously stored dark signal are stored in the storage area defined by the comparison / storage control unit 12 as described above. Is done.

  Unlike the first embodiment, when the dark signal is stored in the storage areas in all the storage units 13a, 13b, 13c, and 13d before the dark signal is newly received, it is newly received. The dark signal having the smallest signal level among the dark signal and the already stored dark signal is deleted.

  As in the first embodiment, the comparison / storage control unit 12 compares the signal levels and stores the dark signals of all the black pixels 32 connected to the common vertical signal line 33 based on the comparison result. Then, a removal signal is generated.

  Unlike the first embodiment, after the comparison and storage of the signal level, the signals are stored in the storage areas RA ′ (n, 1) and RA ′ (n, 2) in the first and second storage units 13a and 13b. A dark signal is sent to the calculation unit 14.

  As in the first embodiment, the arithmetic unit 14 calculates the average value of the signal level of the transmitted dark signal. The removal signal that is the calculated signal level of the average value is used in the subtraction unit 15 to remove the FPN from the pixel signal. Similar to the first embodiment, the generation of the removal signal and the removal of the FPN from the pixel signal using the removal signal are performed for each vertical signal line 33.

  The FPN removal process performed in the noise removal apparatus of the present embodiment configured as described above will be described in more detail with reference to the flowcharts of FIGS. 6 and 12 to 17. FIG. 12 is a flowchart showing an overview of the FPN removal process.

  FIG. 13 is a first flowchart showing a process for storing a dark signal in a black memory. FIG. 14 is a second flowchart showing the process of storing the dark signal in the black memory. FIG. 15 is a third flowchart showing the process of storing the dark signal in the black memory. FIG. 16 is a fourth flowchart showing the process of storing the dark signal in the black memory. FIG. 17 is a fifth flowchart showing the process of storing the dark signal in the black memory.

  In step S200 and step S201, processing similar to that in step S100 and step S101 in the first embodiment is performed. Therefore, when the signal newly received by the receiving unit 11 is a dark signal, the process proceeds to step S104 (see FIG. 6).

  If it is a pixel signal in step S201, the process proceeds to step S202. As in the first embodiment, when proceeding to step S202, the storage areas RA ′ (n, 1), RA ′ (in the first, second, third, and fourth storage units 13a, 13b, 13c, and 13d n, 2), RA ′ (n, 3), and RA ′ (n, 4) all store dark signals.

  In step S202, unlike the first embodiment, the signal of the dark signal stored in the storage areas RA ′ (n, 1) and RA ′ (n, 2) in the first and second storage units 13a and 13b. Calculate the average level. A signal having an average signal level is generated as a removal signal.

  In step S203 after the generation of the removal signal, the same processing as that in step S103 of the first embodiment is executed to remove the FPN from the pixel signal.

  Next, processing in the case where a dark signal is received in step S201 will be described. If a dark signal is received, the process proceeds to step S104. In steps S104 to S107, a determination is made as to which column the black pixel 32 that has generated the received dark signal is counted from the lowest end.

  That is, in step S104, it is determined whether or not it is the first column. If it is the first column, the process proceeds to step S204 (see FIG. 13). In step S105, it is determined whether or not it is the second column. If it is the second column, the process proceeds to step S205 (see FIG. 14). In step S106, it is determined whether or not it is the third column. If it is the third column, the process proceeds to step S208 (see FIG. 15). Further, in step S107, it is determined whether or not it is the fourth column. If it is the fourth column, the process proceeds to step S213 (see FIG. 16). If it is a dark signal generated from the black pixels 32 in the fourth and subsequent columns in step S107, the process proceeds to step S220 (see FIG. 17).

  As shown in FIG. 13, when the process proceeds to step S204, the dark signal generated and received in the black pixels 32 in the first column, that is, first read out from the plurality of black pixels 32 connected to the common vertical signal line 33. The dark signal to be stored is stored in the storage area RA ′ (n, 4) in the fourth storage unit 13d. After storing the dark signal, the process returns to step S200.

  As shown in FIG. 14, when the process proceeds to step S205, the signal level of the dark signal generated and received at the black pixel 32 in the second column and the storage area RA ′ (n, 4) already in the fourth storage unit 13d. The signal level of the stored dark signal is compared.

  When the signal level of the received dark signal is higher, the process proceeds to step S206. In step S206, the dark signal stored in the storage area RA '(n, 4) in the fourth storage unit 13d is stored in the storage area RA' (n, 3) in the third storage unit 13c. Further, the newly received dark signal is stored in the storage area RA '(n, 4) in the fourth storage unit 13d. After storing, the process returns to step S200.

  In step S205, if the signal level of the received dark signal is lower, the process proceeds to step S207. In step S207, the newly received dark signal is stored in the storage area RA '(n, 3) in the third storage unit 13c. After storing, the process returns to step S200.

  As shown in FIG. 15, when the process proceeds to step S208, the signal level of the dark signal generated and received at the black pixel 32 in the third column and the storage area RA ′ (n, 4) in the fourth storage unit 13d are already stored. The signal level of the stored dark signal is compared.

  When the signal level of the received dark signal is higher, the process proceeds to step S209. In step S209, the dark signal stored in the storage area RA ′ (n, 3) in the third storage unit 13c and the dark signal stored in the storage area RA ′ (n, 4) in the fourth storage unit 13d are obtained. And stored in the storage area RA ′ (n, 2) in the second storage unit b and the storage area RA ′ (n, 3) in the third storage unit 13c, respectively. Further, the newly received dark signal is stored in the storage area RA '(n, 4) in the fourth storage unit 13d. After storing, the process returns to step S200.

  In step S208, when the signal level of the received dark signal is lower, the process proceeds to step S210. In step S210, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA '(n, 3) in the third storage unit 13c.

  When the signal level of the received dark signal is higher, the process proceeds to step S211. In step S211, the dark signal stored in the storage area RA '(n, 3) in the third storage unit 13c is stored in the storage area RA' (n, 2) in the second storage unit 13b. Further, the newly received dark signal is stored in the storage area RA ′ (n, 3) in the third storage unit 13c. After storing, the process returns to step S200.

  In step S210, when the signal level of the received dark signal is lower, the process proceeds to step S212. In step S212, the newly received dark signal is stored in the storage area RA '(n, 2) in the second storage unit 13b. After storing, the process returns to step S200.

  As shown in FIG. 16, when the process proceeds to step S213, the signal level of the dark signal generated and received at the black pixel 32 in the fourth column and the storage area RA ′ (n, 4) in the fourth storage unit 13d are already stored. The signal level of the stored dark signal is compared.

  When the signal level of the received dark signal is higher, the process proceeds to step S214. In step S214, the dark signal stored in the storage area RA ′ (n, 2) in the second storage unit 13b and the dark signal stored in the storage area RA ′ (n, 3) in the third storage unit 13c. , And the dark signals stored in the storage area RA ′ (n, 4) in the fourth storage unit 13d are respectively stored in the storage area RA ′ (n, 1) and the second storage unit 13b in the first storage unit 13a. In the storage area RA ′ (n, 2) and the storage area RA ′ (n, 3) in the third storage unit 13c. Further, the newly received dark signal is stored in the storage area RA '(n, 4) in the fourth storage unit 13d. After storing, the process returns to step S200.

  If the signal level of the received dark signal is lower in step S213, the process proceeds to step S215. In step S215, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA '(n, 3) in the third storage unit 13c.

  When the signal level of the received dark signal is higher, the process proceeds to step S216. In step S216, the dark signals stored in the storage area RA ′ (n, 2) in the second storage unit 13b and the storage area RA ′ (n, 3) in the third storage unit 13c are respectively stored in the first storage. The data is stored in the storage area RA ′ (n, 1) in the unit 13a and the storage area RA ′ (n, 2) in the second storage unit 13b. Further, the newly received dark signal is stored in the storage area RA ′ (n, 3) in the third storage unit 13c. After storing, the process returns to step S200.

  If the signal level of the received dark signal is lower in step S215, the process proceeds to step S217. In step S217, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA '(n, 2) in the second storage unit 13b.

  If the signal level of the received dark signal is higher, the process proceeds to step S218. In step S218, the dark signal stored in the storage area RA '(n, 2) in the second storage unit 13b is stored in the storage area RA' (n, 1) in the first storage unit 13a. Also, the newly received dark signal is stored in the storage area RA ′ (n, 2) in the second storage unit 13b. After storing, the process returns to step S200.

  If the signal level of the received dark signal is lower in step S217, the process proceeds to step S219. In step S219, the newly received dark signal is stored in the storage area RA '(n, 1) in the first storage unit 13a. After storing, the process returns to step S200.

  As shown in FIG. 17, when the process proceeds to step S220, the signal level of the dark signal generated and received in the black pixels 32 in the fifth and subsequent columns and the storage area RA ′ (n, 4) in the fourth storage unit 13d already. Is compared with the signal level of the dark signal stored in.

  When the signal level of the received dark signal is higher, the process proceeds to step S221. In step S221, first, the dark signal stored in the storage area RA '(n, 1) in the first storage unit 13a is deleted. Next, the dark signal stored in the storage area RA ′ (n, 2) in the second storage unit 13b, the dark signal stored in the storage area RA ′ (n, 3) in the third storage unit 13c, And the dark signal stored in the storage area RA ′ (n, 4) in the fourth storage unit 13d are stored in the storage area RA ′ (n, 1) in the first storage unit 13a and in the second storage unit 13b, respectively. The data is stored in the storage area RA ′ (n, 2) and the storage area RA ′ (n, 3) in the third storage unit 13c. Further, the newly received dark signal is stored in the storage area RA '(n, 4) in the fourth storage unit 13d. After storing, the process returns to step S200.

  In step S220, if the signal level of the received dark signal is lower, the process proceeds to step S222. In step S222, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA '(n, 3) in the third storage unit 13c.

  When the signal level of the received dark signal is higher, the process proceeds to step S223. In step S223, first, the dark signal stored in the storage area RA '(n, 1) in the first storage unit 13a is deleted. Next, the dark signal stored in the storage area RA ′ (n, 2) in the second storage unit 13b and the dark signal stored in the storage area RA ′ (n, 3) in the third storage unit 13b are obtained. , Respectively, in the storage area RA ′ (n, 1) in the first storage unit 13a and in the storage area RA ′ (n, 2) in the second storage unit 13b. The newly received dark signal is stored in the storage area RA ′ (n, 3) in the third storage unit 13c. After storing, the process returns to step S200.

  If the signal level of the received dark signal is lower in step S222, the process proceeds to step S224. In step S224, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA '(n, 2) in the second storage unit 13b.

  When the signal level of the received dark signal is higher, the process proceeds to step S225. In step S225, first, the dark signal stored in the storage area RA '(n, 1) in the first storage unit 13a is deleted. Next, the dark signal stored in the storage area RA '(n, 2) in the second storage unit 13b is stored in the storage area RA' (n, 1) in the first storage unit 13a. Also, the newly received dark signal is stored in the storage area RA ′ (n, 2) in the second storage unit 13b. After storing, the process returns to step S200.

  If the signal level of the received dark signal is lower in step S224, the process proceeds to step S226. In step S226, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA '(n, 1) in the first storage unit 13a.

  When the signal level of the received dark signal is higher, the process proceeds to step S227. In step S227, first, the dark signal stored in the storage area RA '(n, 1) in the first storage unit 13a is deleted. The newly received dark signal is stored in the storage area RA '(n, 1) in the first storage unit 13a. After storing, the process returns to step S200.

  If the signal level of the received dark signal is lower in step S226, the process proceeds to step S228. In step S228, the newly received dark signal is deleted. After deletion, the process returns to step S200.

  As described above, the storage areas RA ′ (n, 1 and 1) in the first, second, third, and fourth storage units 13a, 13b, 13c, and 13d are obtained by the processes in steps S104 to S107 and steps S200 to S228. ), RA ′ (n, 2), RA ′ (n, 3), and RA ′ (n, 4) are all stored with dark signals.

  Even with the noise removal apparatus of the present embodiment having the above-described configuration, it is possible to remove the influence of defects when the black pixels 32 connected to the common vertical signal line 33 have black defects or white defects. It is. Therefore, it is possible to accurately remove the FPN from the pixel signal of the effective pixel without widening the black pixel region.

  Next, the noise removal apparatus of 3rd Embodiment is demonstrated. In the present embodiment, the dark signal stored in the first to fourth storage units and the dark signal storage processing method by the comparison / storage processing unit are different from those in the first embodiment. Hereinafter, a description will be given focusing on differences from the first embodiment. In addition, the same code | symbol is attached | subjected to the site | part which has the same function as 1st Embodiment.

  The noise removing device 10 is configured by a receiving unit 11, a comparison / storage control unit 12 (comparison unit, storage processing unit), a black memory 13, a calculation unit 14 (selection unit, removal signal output unit), and a subtraction unit 15. Is the same as in the first embodiment.

  The dark signal is received by the receiving unit 11 and sent from the receiving unit 11 to the comparison / storage control unit 12 as in the first embodiment. Further, the pixel signal is received by the receiving unit 11 and sent to the subtracting unit 15 as in the first embodiment.

  As will be described later, the dark signal is stored in the black memory 13 from the comparison / storage control unit 12 by a method different from that of the first embodiment. The removal signal is generated by the black memory 13 and the calculation unit 14 based on the stored dark signal, and the FPN is removed from the pixel signal based on the generated removal signal, as in the first embodiment. .

  As in the first embodiment, the black memory 13 includes first, second, third, and fourth storage units 13a, 13b, 13c, and 13d. Each storage unit 13a, 13b, 13c, 13d is provided with a plurality of storage areas, and each storage area is assigned to one of the vertical signal lines 33. In this embodiment, the storage area allocated to the nth column vertical signal line 33 in the first storage unit 13a is displayed as RA ″ (n, 1).

  In the present embodiment, the storage areas RA ″ (n, 1), RA ″ (n, 2), RA ″ (n, n) in the first, second, third, and fourth storage units 13a, 13b, 13c, and 13d. 3) The dark signal stored in RA ″ (n, 4) is different from that of the first embodiment. In the present embodiment, the dark signal having the minimum signal level among the dark signals of the plurality of black pixels 32 connected to the common vertical signal line 33 and stored simultaneously is the first storage. Stored in the storage area RA ″ (n, 1) in the section 13a.

  Similarly, the dark signal having the second lowest signal level among the plurality of dark signals stored at the same time is stored in the storage area RA ″ (n, 2) in the second storage unit 13b. Among the plurality of dark signals, the dark signal having the third lowest signal level is stored in the storage area RA ″ (n, 3) in the third storage unit 13c. The dark signal having the fourth lowest signal level among the plurality of dark signals stored simultaneously is stored in the storage area RA ″ (n, 4) in the fourth storage unit 13d.

  When a dark signal is input from the receiving unit 11 to the comparison / storage control unit 12, the signal level of the dark signal stored in the black memory 13 before reception by the receiving unit 11 and the signal level of the newly received dark signal Are compared with each other, and based on the comparison result, the newly input dark signal and the previously stored dark signal are stored in the storage area determined by the comparison / storage control unit 12 as described above. Is done.

  Unlike the first embodiment, when dark signals are stored in the storage areas of all the storage units 13a, 13b, 13c, and 13d before a dark signal is newly input, a new signal is generated at a certain timing. The dark signal with the maximum signal level is deleted from the dark signal received and the dark signal already stored, and the dark signal with the minimum signal level is deleted at the next dark signal reception timing. Is done.

  As in the first embodiment, the comparison / storage control unit 12 compares the signal levels and stores the dark signals of all the black pixels 32 connected to the common vertical signal line 33 based on the comparison result. Then, a removal signal is generated.

  Unlike the first embodiment, after the comparison and storage of the signal level, the signals are stored in the storage areas RA ″ (n, 2) and RA ″ (n, 3) in the second and third storage units 13b and 13c. A dark signal is sent to the calculation unit 14.

  As in the first embodiment, the arithmetic unit 14 calculates the average value of the signal level of the transmitted dark signal. The removal signal that is the calculated signal level of the average value is used in the subtraction unit 15 to remove the FPN from the pixel signal. Similar to the first embodiment, the generation of the removal signal and the removal of the FPN from the pixel signal using the removal signal are performed for each vertical signal line 33.

  The FPN removal process performed in the noise removal apparatus of the present embodiment as described above will be described in more detail using the flowcharts of FIGS. 7 to 11 and FIGS. 18 to 20. FIG. 18 is a flowchart showing an overview of the FPN removal process. FIG. 19 is a flowchart illustrating a process of determining the row of black pixels that has generated the received dark signal. FIG. 20 is a sixth flowchart showing the process of storing the dark signal in the black memory.

  In step S300 and step S301, processing similar to that in step S100 and step S101 in the first embodiment is performed. Therefore, when the signal newly received by the receiving unit 11 is a dark signal, the process proceeds to step S304 (see FIG. 19).

  If it is a pixel signal in step S301, the process proceeds to step S302. As in the first embodiment, when proceeding to step S302, the storage areas RA ″ (n, 1), RA ″ (in the first, second, third, and fourth storage units 13a, 13b, 13c, and 13d n, 2), RA ″ (n, 3), and RA ″ (n, 4) all store dark signals.

  In step S302, unlike the first embodiment, the signal of the dark signal stored in the storage area RA ″ (n, 2), RA ″ (n, 3) in the second and third storage units 13b and 13c. Calculate the average level. A signal having an average signal level is generated as a removal signal.

  In step S303 after the generation of the removal signal, the same process as in step S103 of the first embodiment is executed to remove the FPN from the pixel signal.

  Next, processing in the case where a dark signal is received in step S301 will be described. If a dark signal is received, the process proceeds to step S304. In steps S304 to S308, a determination is made as to which column the black pixel 32 that generated the received dark signal is counted from the lowest end (see FIG. 19).

  That is, in step S304, it is determined whether or not it is the first column. If it is the first column, the process proceeds to step S108 (see FIG. 7). In step S305, it is determined whether or not it is the second column. If it is the second column, the process proceeds to step S109 (see FIG. 8). In step S306, it is determined whether or not it is the third column. If it is the third column, the process proceeds to step S112 (see FIG. 9).

  In step S307, it is determined whether or not it is the fourth column. If it is the fourth column, the process proceeds to step S117 (see FIG. 10). Further, in step S308, it is determined whether or not it is an odd column from the fifth column. If it is a dark signal generated from the black pixels 32 in the odd column, the process proceeds to step S309 (see FIG. 20). On the other hand, if it is the dark signal generated from the black pixels 32 in the even-numbered columns after the fifth column in step S308, the process proceeds to step S124 (see FIG. 11).

  When the dark signal generated in the black pixels 32 in the first to fourth columns is received, the same processing as in the first embodiment is performed, and the process returns to step S300 (see FIGS. 7 to 10). When a dark signal generated from the black pixels 32 in the even-numbered columns after the fifth column is received, a dark signal generated from the black pixels 32 in the fifth and subsequent columns in the first embodiment is received. The same processing is performed, and the process returns to step S300 (see FIG. 11).

  When a dark signal generated from the odd-numbered black pixels 32 in the fifth and subsequent columns is received, the same as when a dark signal generated from the black pixels 32 in the fifth and subsequent columns in the second embodiment is received. Is processed.

  That is, as shown in FIG. 20, when the process proceeds to step S309, the signal level of the dark signal generated and received in the black pixels 32 in the odd-numbered columns after the fifth column and the storage area RA "already in the fourth storage unit 13d. The signal level of the dark signal stored in (n, 4) is compared.

  When the signal level of the received dark signal is higher, the process proceeds to step S310. In step S310, first, the dark signal stored in the storage area RA ″ (n, 1) in the first storage unit 13a is deleted. Next, the storage area RA ″ (n, 2 in the second storage unit 13b is deleted. ), The dark signal stored in the storage area RA ″ (n, 3) in the third storage unit 13c, and the storage area RA ″ (n, 4) in the fourth storage unit 13d. The stored dark signals are stored in the storage area RA ″ (n, 1) in the first storage unit 13a, the storage area RA ″ (n, 2) in the second storage unit 13b, and the third storage unit 13c, respectively. The newly received dark signal is stored in the storage area RA ″ (n, 4) in the fourth storage unit 13d. After storing, the process returns to step S300.

  In step S309, if the signal level of the received dark signal is lower, the process proceeds to step S311. In step S311, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA ″ (n, 3) in the third storage unit 13c.

  When the signal level of the received dark signal is higher, the process proceeds to step S312. In step S312, first, the dark signal stored in the storage area RA ″ (n, 1) in the first storage unit 13a is deleted. Next, the storage area RA ″ (n, 2 in the second storage unit 13b is deleted. ) And the dark signal stored in the storage area RA ″ (n, 3) in the third storage unit 13c are stored in the storage area RA ″ (n, 1) and in the first storage unit 13a, respectively. The newly stored dark signal is stored in the storage area RA ″ (n, 3) in the third storage section 13c. After storing, the process returns to step S300.

  In step S311, when the signal level of the received dark signal is lower, the process proceeds to step S313. In step S313, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA ″ (n, 2) in the second storage unit 13b.

  When the signal level of the received dark signal is higher, the process proceeds to step S314. In step S314, the dark signal stored in the storage area RA ″ (n, 1) in the first storage unit 13a is first deleted. Next, the storage area RA ″ (n, 2 in the second storage unit 13b is deleted. ) Is stored in the storage area RA ″ (n, 1) of the first storage unit 13a. The newly received dark signal is stored in the storage area RA ″ (n of the second storage unit 13b. 2). After storing, the process returns to step S300.

  If the signal level of the received dark signal is lower in step S313, the process proceeds to step S315. In step S315, the signal level of the newly received dark signal is compared with the signal level of the dark signal already stored in the storage area RA ″ (n, 1) in the first storage unit 13a.

  When the signal level of the received dark signal is higher, the process proceeds to step S316. In step S316, first, the dark signal stored in the storage area RA ″ (n, 1) in the first storage unit 13a is deleted. The newly received dark signal is stored in the storage area RA ″ (in the first storage unit 13a. n, 1). After storing, the process returns to step S300.

  If the signal level of the received dark signal is lower in step S315, the process proceeds to step S317. In step S317, the newly received dark signal is deleted. After the deletion, the process returns to step S300.

  As described above, the storage areas RA "(n, 1 and 2) in the first, second, third, and fourth storage units 13a, 13b, 13c, and 13d are obtained by the processes in steps S108 to S132 and steps S300 to S317. ), RA ″ (n, 2), RA ″ (n, 3), and RA ″ (n, 4).

  Even with the noise removal apparatus of the present embodiment having the above-described configuration, it is possible to remove the influence of defects when the black pixels 32 connected to the common vertical signal line 33 have black defects or white defects. It is. Therefore, it is possible to accurately remove the FPN from the pixel signal of the effective pixel 31 without widening the black pixel area BA.

  In the first to third embodiments, each time a dark signal is newly received, the magnitude of the signal level is compared with the stored dark signal and the magnitude of the signal level. Each of the dark signals is stored in the storage area. After all the dark signals are stored in the black memory 13, the removal signal is generated by excluding at least the dark signals having the maximum and minimum signal levels. May be.

  This is because the influence of the black defect or white defect that may occur in each vertical signal line 33 can be eliminated by generating the removal signal by excluding the dark signal having the maximum and minimum signal levels.

  However, if the dark signals are stored while being compared as in the first to third embodiments, it is not necessary to store all the dark signals. Therefore, the number of storage units provided in the black memory 13 can be made smaller than the number of black pixels connected to the vertical signal line 33. Therefore, according to the first to third embodiments, a memory having a low capacity can be used as the black memory.

  In the first to third embodiments, with respect to the image pickup device 30 provided with a plurality of black pixels 32 for each vertical signal line 33, a removal signal is generated for each vertical signal line 33 to remove the FPN. However, if the imaging device 30 has the same signal level for the FPN for each vertical signal line 33, the generation of the removal signal and the removal of the FPN do not have to be performed for each vertical signal line 33.

  In the first to third embodiments, the FPN is removed from the pixel signal with respect to the image sensor 30 in which the black pixels 32 connected to the common vertical signal line 33 are seven. The present invention can be applied to the image sensor 30 to which three or more black pixels 32 are connected every 33.

  If it is connected to three or more black pixels 32, one or more dark signals remain even if the dark signals having the maximum and minimum signal levels are excluded, so that a removal signal can be generated and an accurate FPN can be generated. This is because it is possible to remove this.

  In the first to third embodiments, the removal signal is generated by calculating the average value of the two dark signals, but the removal signal is generated by calculating the average value of three or more dark signals. It may be configured to do so.

  In the first embodiment, if the dark signal stored in the first storage unit 13a is excluded, it is possible to generate the removal signal by excluding the maximum and minimum dark signals of the signal level. However, after reading out the dark signals from all the black pixels 32 connected to the common vertical signal line 33, the dark signals stored in the fourth storage unit 13d having the maximum signal level are used to generate the removal signals. It is preferable to use it.

  As in the first embodiment, when the difference between the number of black pixels 32 connected to the common vertical signal line 33 and the number of storage units is larger than half the number of storage units, the signal level becomes maximum. This is because the dark signal is close to the median value.

  In the second embodiment, if the dark signal stored in the fourth storage unit 13d is excluded, the removal signal can be generated by excluding the maximum and minimum dark signals of the signal level. However, after reading the dark signals from all the black pixels 32 connected to the common vertical signal line 33, the dark signals stored in the first storage unit 13a having the minimum signal level are used to generate the removal signals. It is preferable to use it.

  As in the second embodiment, when the difference between the number of black pixels 32 connected to the common vertical signal line 33 and the number of storage units is larger than half the number of storage units, the signal level is minimized. This is because the dark signal is close to the median value.

  In the third embodiment, since dark signals having the maximum and minimum signal levels are excluded and stored, a removal signal can be generated using all the dark signals stored in the storage unit.

  In the first to third embodiments, the dark signal stored in the plurality of storage units is averaged to calculate the removal signal, but the dark signal stored in the single storage unit is removed. It may be configured to output as a signal.

  A major factor that deteriorates the accuracy of FPN removal is the white defect and black defect of the black pixel 32. Except for these effects, the signal levels of the respective dark signals are substantially equal, and therefore, using any of the dark signals enables FPN removal with higher accuracy than before.

  However, it is preferable to use the median value among a plurality of dark signals for each common vertical signal line 33. Therefore, in the configuration in which the dark signal is extracted after all the dark signals are stored in the black memory 13, it is preferable to extract the median value and output it as a removal signal.

  In the first to third embodiments, the black memory 13 is provided with four storage units. However, two or more storage units may be provided. This is because, in any embodiment, if there are two or more storage units, it is possible to eliminate both dark signals having the maximum and minimum signal levels.

  In the first embodiment, the dark signal having the maximum and minimum signal levels can be excluded by using the dark signal stored in the storage unit in which the dark signal having the maximum signal level is stored. In the second embodiment, the dark signal having the maximum and minimum signal levels can be excluded by using the dark signal stored in the storage unit in which the dark signal having the minimum signal level is stored. In the third embodiment, the dark signal having the maximum and minimum signal levels can be excluded by using the dark signal in the storage unit from which the dark signal originally stored when the dark signal was last received is deleted. is there.

  In the first embodiment, the dark signal stored in the first storage unit 13a is not used to generate the removal signal. However, all dark signals may be used. Although the influence of the dark signal having the minimum signal level cannot be removed, it is possible to remove the influence of the dark signal having at least the maximum signal level. Therefore, it is possible to perform sufficiently accurate FPN removal even with such an FPN removal, for an image sensor that produces only white defects.

  As described above, in the case where all the dark signals read from the black pixels 32 connected to the common vertical signal line 33 are stored in the black memory 13, dark signals other than the dark signal having the maximum signal level are stored. The removal signal may be generated using

  In the second embodiment, the dark signal stored in the fourth storage unit 13d is not used to generate the removal signal. However, all dark signals may be used. Although the influence of the dark signal having the maximum signal level cannot be removed, it is possible to remove the influence of the dark signal having at least the minimum signal level. Therefore, it is possible to perform sufficiently accurate FPN removal even with such an FPN removal, for an image sensor that causes only black defects.

  As described above, in the case where all the dark signals read from the black pixels 32 connected to the common vertical signal line 33 are stored in the black memory 13, dark signals other than the dark signal having the minimum signal level are stored. The removal signal may be generated using

  In the first to third embodiments, the image pickup device is a CMOS image pickup device. However, the image pickup device is applied to an image pickup device in which an effective pixel and a plurality of black pixels are connected to a common signal line. Is possible. Further, even in the case of a charge transfer type image pickup device such as a CCD image pickup device, for example, an image pickup device in which an effective pixel 31 and a plurality of black pixels 32 are connected to a common vertical CCD (charge transfer path). It is also possible to apply.

1 is a block diagram schematically showing a configuration of an imaging unit having a noise removal device to which a first embodiment of the present invention is applied. It is a block diagram which shows the structure of the light-receiving surface of an image pick-up element. It is a block diagram which shows roughly the internal structure of a noise removal apparatus. It is a structural diagram which shows the structure about the storage area | region of a black memory. It is a flowchart which shows the whole outline | summary of the process of FPN removal in 1st Embodiment. It is a flowchart which shows the process which judges the row | line | column of the black pixel which produced | generated the received dark signal in 1st, 2nd embodiment. It is a 1st flowchart which shows the process which stores the dark signal in 1st, 3rd embodiment in a black memory. It is a 2nd flowchart which shows the process which stores the dark signal in 1st, 3rd embodiment in black memory. It is a 3rd flowchart which shows the process which stores the dark signal in 1st, 3rd embodiment in black memory. It is a 4th flowchart which shows the process which stores the dark signal in 1st, 3rd embodiment in black memory. It is a 5th flowchart which shows the process which stores the dark signal in 1st, 3rd embodiment in black memory. It is a flowchart which shows the whole outline | summary of the process of FPN removal in 2nd Embodiment. It is a 1st flowchart which shows the process which stores the dark signal in 2nd Embodiment in black memory. It is a 2nd flowchart which shows the process which stores the dark signal in 2nd Embodiment in black memory. It is a 3rd flowchart which shows the process which stores the dark signal in 2nd Embodiment in black memory. It is a 4th flowchart which shows the process which stores the dark signal in 2nd Embodiment in black memory. It is a 5th flowchart which shows the process which stores the dark signal in 2nd Embodiment in black memory. It is a flowchart which shows the whole outline | summary of the process of FPN removal in 3rd Embodiment. It is a flowchart which shows the process which judges the row | line | column of the black pixel which produced | generated the received dark signal in 3rd Embodiment. It is a 6th flowchart which shows the process which stores the dark signal in 3rd Embodiment in black memory.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Noise removal apparatus 11 Receiving part 12 Comparison / storage control part 13 Black memory 13a, 13b, 13c, 13d 1st, 2nd, 3rd, 4th storage part 14 Computation part 15 Subtraction part 20 Imaging unit 30 Imaging element 31 Effective pixel 32 Black pixel 33 Vertical signal line BA Black pixel area EA Effective pixel area

Claims (14)

An effective pixel that generates a pixel signal corresponding to the amount of light received; a plurality of black pixels that generate a dark signal corresponding to a pixel signal corresponding to the amount of light received in the dark; the effective pixel and the plurality of black pixels; A receiving unit that receives the pixel signal and the dark signal from an imaging unit that is connected to the image pickup unit and has a transfer path that outputs the pixel signal and a plurality of the dark signals;
An extraction unit for extracting other dark signals excluding at least one of dark signals having a maximum and minimum signal level among the plurality of dark signals;
A removal signal output unit that outputs a removal signal for removing fixed pattern noise included in the pixel signal based on the dark signal extracted by the extraction unit;
A fixed pattern noise removal apparatus comprising: a removal unit that removes the fixed pattern noise from the pixel signal based on the removal signal.   The fixed pattern noise removing apparatus according to claim 1, wherein the extraction unit extracts a dark signal excluding both dark signals having a maximum and minimum signal level among the plurality of dark signals. The imaging means includes a plurality of transfer paths each connecting the effective pixel and the plurality of black pixels,
The extraction unit performs extraction from dark signals output from a plurality of black pixels connected to the common transfer path,
The removal signal output unit removes a fixed pattern noise included in a pixel signal generated in an effective pixel connected to the common transfer path based on the dark signal extracted by the extraction unit. Output
The said removal part removes the said fixed pattern noise from the pixel signal which the effective pixel connected to the said common transfer path produces | generates based on the said removal signal. Fixed pattern noise removal device. There are a plurality of dark signals extracted by the extraction unit,
The said removal signal output part produces | generates and outputs the said removal signal by calculating the average value of the one part or all part dark signal extracted by the said extraction part, The Claim 1 characterized by the above-mentioned. 4. The fixed pattern noise removing device according to any one of 3 above. The extraction unit extracts a dark signal having a median signal level among a plurality of dark signals received by the reception unit,
The fixed pattern noise removing apparatus according to any one of claims 1 to 3, wherein the removal signal output unit outputs a dark signal extracted by the extraction unit as the removal signal. The extraction unit includes:
A black memory having a plurality of storage areas for storing the dark signal;
A comparison unit for comparing the magnitude of the signal level of the dark signal already stored in the storage area and the signal level of the dark signal newly received by the reception unit;
According to the comparison result in the comparison unit, the dark signal already stored in the storage area and the dark signal newly received by the reception unit are determined according to the order of the magnitude of the signal level. A storage processing unit for storing in the storage area;
And a selection unit that extracts dark signals stored in a predetermined storage area after storing dark signals generated in the plurality of black pixels during one frame or one field by the storage processing unit. The fixed pattern noise removing device according to claim 1 or 2. The imaging means includes a plurality of transfer paths each connecting the effective pixel and the plurality of black pixels,
The extraction unit performs extraction from dark signals output from a plurality of black pixels connected to the common transfer path,
The removal signal output unit removes a fixed pattern noise included in a pixel signal generated in an effective pixel connected to the common transfer path based on the dark signal extracted by the extraction unit. Output
The fixed pattern noise according to claim 6, wherein the removing unit removes the fixed pattern noise from a pixel signal generated by an effective pixel connected to the common transfer path based on the removal signal. Removal device. The number of storage areas is less than the number of black pixels connected to the common transfer path,
The storage processing unit receives the dark signal when the dark signal is newly received by the receiving unit when the dark signal is stored in all of the plurality of storage areas during a period of one frame or one field. The fixed pattern noise removal according to claim 7, wherein the dark signal having the maximum signal level is deleted from the dark signal newly received by the unit and the dark signal stored in the storage area. apparatus.   The selection unit stores all dark signals output from black pixels connected to the common transfer path during the period of one frame or one field in all the storage areas after reception by the reception unit. 9. The fixed pattern noise removing apparatus according to claim 8, wherein at least a dark signal having a maximum signal level is extracted from the dark signals. The number of storage areas is less than the number of black pixels connected to the common transfer path,
The storage processing unit receives the dark signal when the dark signal is newly received by the receiving unit when the dark signal is stored in all of the plurality of storage areas during a period of one frame or one field. The fixed pattern noise removal according to claim 7, wherein a dark signal having a minimum signal level is deleted from the dark signal newly received by the unit and the dark signal stored in the storage area. apparatus.   The selection unit stores all dark signals output from black pixels connected to the common transfer path during the period of one frame or one field in all the storage areas after reception by the reception unit. The fixed pattern noise removing apparatus according to claim 10, wherein a dark signal having at least a minimum signal level is extracted from the dark signals being processed. The number of storage areas is less than the number of black pixels connected to the common transfer path,
When the dark signal is stored in all of the plurality of storage areas during the period of one frame or one field, the storage processing unit receives the dark signal newly at the receiving unit at the first timing. The dark signal having the maximum signal level among the dark signal newly received by the receiving unit and the dark signal stored in the storage area is deleted, and the second signal after the first timing is deleted. When the dark signal is newly received by the receiving unit at timing 2, the signal level is the lowest among the dark signal newly received by the receiving unit and the dark signal stored in the storage area. The fixed pattern noise removing apparatus according to claim 7, wherein a dark signal is deleted. The selection unit stores all the dark signals output from the black pixels connected to the common transfer path during the period of one frame or one field, and stores them in all the storage areas after reception to the reception unit. The dark signal in the storage area in which the dark signal whose signal level is the median value is stored,
The fixed pattern noise removal apparatus according to claim 12, wherein the removal signal output unit outputs a dark signal extracted to the selection unit as the removal signal. The selection unit stores all the dark signals output from the black pixels connected to the common transfer path during the period of one frame or one field, and stores them in all the storage areas after reception to the reception unit. Extract some or all of the dark signal
The fixed pattern noise removal device according to claim 12, wherein the removal signal output unit generates and outputs the removal signal by calculating an average value of dark signals extracted by the selection unit. .

Images