JP4557795B2 - Data correction processing apparatus and data correction processing method - Google Patents

Description

  The present invention relates to a data correction processing apparatus that performs correction processing on a large number of pixel data obtained from a solid-state imaging device including a large number of photoelectric conversion elements arranged in a row direction on a semiconductor substrate and in a column direction perpendicular thereto.

  2. Description of the Related Art A solid-state imaging device including a large number of photoelectric conversion elements arranged in a row direction on a semiconductor substrate and a column direction orthogonal thereto is known. The photoelectric conversion element included in such a solid-state image sensor includes a photoelectric conversion element that does not react to incident light due to its manufacturing process, or a photoelectric conversion that generates an abnormally large dark current without incident light. Often, defective elements such as elements are included. These defective elements are referred to as, for example, “black scratches” and “white scratches”, respectively, and it is difficult to completely remove these defective elements themselves in a solid-state imaging device.

  Therefore, in a large number of pixel data output from the solid-state image sensor, defective pixel data for correcting defective pixel data generated by these defective elements using pixel data obtained from the surrounding photoelectric conversion elements is used. Correction methods have been proposed. When correcting defective pixel data, for example, there is a method of simply averaging pixel data from a plurality of photoelectric conversion elements adjacent to the periphery of the defective element and replacing the average value with the defective pixel data.

  However, in the correction method as described above, for example, when a plurality of defective elements exist adjacent to each other, defect correction is performed using defective pixel data from other defective elements adjacent to the defective element to be processed. Therefore, it could not be said that appropriate defect correction was performed. Therefore, in order to solve such a problem, a method described in Patent Document 1 has been proposed.

JP 2000-244823 A

  In the method described in Patent Document 1, a large number of pixel data obtained from a solid-state image sensor is temporarily stored in a frame memory and then defect correction processing is performed. Absent. Further, since a frame memory is required, the circuit scale is increased.

  The present invention has been made in view of the above circumstances, and provides a data correction processing apparatus capable of performing defect correction of defective pixel data obtained from a defective photoelectric conversion element at high speed and reducing the circuit scale. The purpose is to do.

  The data correction processing apparatus of the present invention performs data correction processing for correcting a large number of pixel data obtained from a solid-state imaging device including a large number of photoelectric conversion elements arranged in a row direction on a semiconductor substrate and in a column direction perpendicular thereto. A processing device that temporarily holds the pixel data, and is arranged in a two-dimensional manner around a first photoelectric conversion element included in the solid-state imaging element and the first photoelectric conversion element. Two-dimensional pixel data holding / outputting means for simultaneously outputting two-dimensional pixel data which is pixel data obtained from a photoelectric conversion element group including two photoelectric conversion elements, and the first photoelectric conversion element is a defective element Among the two-dimensional pixel data, pixel data obtained from other than the photoelectric conversion element that detects a color different from that of the first photoelectric conversion element, and a defect data obtained from the defective element included in the pixel data. Defect pattern determination means for determining a defect pattern determined by the position and number of pixel data, calculation formula determination means for determining a calculation formula for defect correction processing based on the defect pattern, the two-dimensional pixel data, and the calculation And defect correction processing means for performing defect correction processing of the first pixel data which is pixel data obtained from the first photoelectric conversion element using the equation.

  With this configuration, defect correction of defective pixel data obtained from a defective photoelectric conversion element can be performed at high speed and the circuit scale can be reduced.

  In the data correction processing apparatus of the present invention, the calculation formula determining means may calculate the calculation formula based on the defect pattern and a luminance value of pixel data excluding the defective pixel data among pixel data constituting the defect pattern. decide.

  With this configuration, it is possible to perform defect correction in consideration of a sudden change in luminance.

  The data correction processing apparatus of the present invention is a pixel data replacing unit that replaces the defective pixel data of the pixel data input to the pixel data holding / outputting unit with pixel data obtained from photoelectric conversion elements around the defective element. Is provided.

  With this configuration, the reliability of defect correction can be further improved.

  The data correction processing apparatus of the present invention includes noise reduction processing means for performing noise reduction processing of the first pixel data using the two-dimensional pixel data when the first photoelectric conversion element is not the defective element. .

  In the data correction processing apparatus of the present invention, the defect correction processing means includes a plurality of arithmetic units, and selects and uses an arithmetic unit necessary for the arithmetic expression from the plurality of arithmetic units.

  With this configuration, the circuit scale can be reduced.

  In the data correction processing apparatus of the present invention, the noise reduction processing means performs the noise reduction processing using at least a part of the plurality of arithmetic units included in the defect correction processing means.

  With this configuration, the circuit scale can be reduced.

  The data correction processing method of the present invention is a data correction that performs correction processing of a large number of pixel data obtained from a solid-state imaging device including a large number of photoelectric conversion elements arranged in a row direction on a semiconductor substrate and in a column direction orthogonal thereto. In the processing method, the pixel data is temporarily held, and the first photoelectric conversion element included in the solid-state imaging element and the first photoelectric conversion element arranged in a two-dimensional manner around the first photoelectric conversion element A two-dimensional pixel data holding and outputting step for simultaneously outputting two-dimensional pixel data that is pixel data obtained from a photoelectric conversion element group including two photoelectric conversion elements, and the first photoelectric conversion element is a defective element Among the two-dimensional pixel data, pixel data obtained from other than the photoelectric conversion element that detects a color different from that of the first photoelectric conversion element, and a defect data obtained from the defective element included in the pixel data. A defect pattern discriminating step for discriminating a defect pattern determined by the position and number of pixel data; an arithmetic equation determining step for determining an arithmetic equation for defect correction processing based on the defect pattern; the two-dimensional pixel data and the arithmetic operation And a defect correction processing step of performing defect correction processing of the first pixel data that is pixel data obtained from the first photoelectric conversion element using the equation.

  ADVANTAGE OF THE INVENTION According to this invention, the data correction processing apparatus which can perform defect correction of the defective pixel data obtained from a defective photoelectric conversion element at high speed, and can reduce a circuit scale can be provided.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is a diagram showing a schematic configuration of a main part of a digital camera for explaining a first embodiment of the present invention.
A digital camera 100 shown in FIG. 1 includes a CCD-type or MOS-type solid-state image pickup device 1 having a large number of photoelectric conversion elements arranged in a row direction on a semiconductor substrate and a column direction orthogonal thereto, and a solid-state image pickup device 1. An A / D conversion unit 2 that digitally converts a large number of pixel data output from a large number of photoelectric conversion elements, a buffer 3, a timing generator (TG) 4, and a large number of photoelectric conversion elements included in the solid-state imaging element 1. Among them, a replacement unit 5 for replacing defective pixel data obtained from a defective photoelectric conversion element (hereinafter referred to as a defective element) with pixel data obtained from photoelectric conversion elements around the defective element, an H / V counter 6, A lookup table (LUT) 7 for storing defect position information indicating the position of the defective element of the solid-state imaging device 1 obtained by inspecting in advance by a tester device or the like First, the pixel data output from the first photoelectric conversion element that is a data conversion target photoelectric conversion element and a plurality of photoelectric conversion elements that are two-dimensionally arranged around the first photoelectric conversion element are stored. Two-dimensional pixel data holding / outputting unit 8 that simultaneously outputs two-dimensional pixel data that is pixel data obtained from a photoelectric conversion element group including two photoelectric conversion elements, and two-dimensional pixel data holding / outputting unit 8 that outputs two-dimensional pixel data. Among the two-dimensional pixel data, the data correction unit 9 that performs data correction such as defect correction processing or noise reduction (NR) processing on the first pixel data obtained from the first photoelectric conversion element, and the data correction unit 9 A signal processing unit 10 that performs signal processing such as interpolation processing, synchronization processing, YC conversion processing, gamma correction processing, and OB correction processing on a large number of output pixel data. It is assumed that pipeline processing is performed in any of the replacement unit 5, the two-dimensional pixel data holding output unit 8, the data correction unit 9, and the signal processing unit 10, thereby enabling high-speed sequential processing of pixel data. Become.

  The solid-state imaging device 1 includes a large number of photoelectric conversion elements arranged in a square lattice, a large number of photoelectric conversion elements arranged in a so-called honeycomb shape as described in JP-A-10-136391, etc. Can be used. In the following description, it is assumed that a large number of photoelectric conversion elements included in the solid-state imaging element 1 are arranged in a honeycomb shape. In the present embodiment, data output from one photoelectric conversion element is defined as pixel data.

  The solid-state imaging device 1 has a primary color filter, and a number of photoelectric conversion elements include, for example, a photoelectric conversion element that detects red (R), and a photoelectric conversion element that detects green (G). , And a photoelectric conversion element for detecting blue (B). The color filter may be a complementary color type. The arrangement of the color filters is not particularly limited as long as it is a regular arrangement (an arrangement in which RGB relative positions are uniquely determined).

  The buffer 3 absorbs a difference in driving speed between the solid-state imaging device 1, the replacement unit 5, the two-dimensional pixel data holding output unit 8, the data correction unit 9, and the signal processing unit 10, and is a general FIFO or the like. Consists of. The buffer 3 takes in the pixel data at the set position in accordance with the synchronization signal from the TG 4 and outputs it to the replacement unit 5 together with an enable signal indicating valid data.

  The H / V counter 6 specifies the color component of the pixel data input to the replacement unit 5 and the position of the photoelectric conversion element from which the pixel data is output, according to the enable signal and the synchronization signal from the TG 4.

  The replacement unit 5 compares the information indicating the position of the photoelectric conversion element specified by the H / V counter 6 with the information indicating the position of the defective element stored in the LUT 7. The data is determined as defective pixel data, and the defective pixel data is replaced with pixel data obtained from a photoelectric conversion element that detects the same color component around the photoelectric conversion element, and the pixel data is replaced with the replaced pixel data. Is added with defect information indicating that the pixel data is defective pixel data. The replacement unit 5 corresponds to pixel data replacement means in the claims.

FIG. 2 is a block diagram showing a schematic configuration inside the replacement unit shown in FIG.
As shown in FIG. 2, the replacement unit 5 includes a pixel data holding element 52 such as a flip-flop that holds R component pixel data, and a pixel data holding element 53 such as a flip-flop that holds G component pixel data. The pixel data holding element 54 such as flip-flop that holds the pixel data of the B component, the pixel data held in any of the pixel data holding elements 52, 53, 54, and the replacement unit 5 A multiplexer (MUX) 55 that outputs any of the pixel data, and a data holding determination unit 51 that determines whether or not the pixel data holding elements 52, 53, and 54 hold the pixel data are provided. The data holding determination unit 51 includes a data selection determination unit 51a that specifies pixel data to be output from the MUX 55, and a defect information addition unit that adds defect information indicating whether the pixel data output from the MUX 55 is defective pixel data. 51b.

  The defect information is, for example, information indicating that there is no defect in 1 bit, and this 1-bit data is added to either the MSB or LSB of the pixel data. The defect information is not only the presence or absence of defects, but also more detailed information, for example, information indicating whether or not the pixel data has been replaced, information indicating what replacement method has been taken, The preceding and following pixel data may be multi-bit information indicating information indicating whether or not defective pixel data is included. In addition, it is assumed that color information indicating the color component is also added to the pixel data output from the replacement unit 5.

  In the pixel data input to the data holding determination unit 51, the color component and the position of the photoelectric conversion element of the output source are specified by the H / V counter 6, and the position information of the photoelectric conversion element and the LUT 7 are stored. The position information of the defective element is compared. If the two do not match, it is determined that the input pixel data is not defective pixel data, and the pixel data is held in the pixel data holding elements 52, 53, and 54 corresponding to the specified color component. The defect information indicating that the pixel data is not defective pixel data is added to the output pixel data.

  On the other hand, if the two match, the input pixel data is not held in the pixel data holding elements 52, 53, and 54, but is held in any one of the pixel data holding elements 52, 53, and 54 from the MUX 55. Pixel data having the same color component as the color component of the input pixel data is output, and defect information indicating defective pixel data is added to the output pixel data.

  For example, in the case where R component pixel data and B component pixel data are alternately input to the replacement unit 5 and the third to fifth input pixel data is defective pixel data, for example, FIG. As shown in FIG. 5, the first and second input pixel data is output with defect information indicating that there is no defective pixel data, and the third input defective pixel data is input first. After replacement with the R component pixel data, defect information indicating that the pixel data is defective is added and output. Similarly, the fourth input defective pixel data is replaced with the second input B component pixel data, and the fifth input defective pixel data is the first input R component pixel data. After the replacement, defect information indicating defective pixel data is added and output.

  Returning to FIG. 1, the two-dimensional pixel data holding / outputting unit 8 temporarily holds the pixel data obtained from the solid-state imaging device 1, and among the many photoelectric conversion elements included in the solid-state imaging device 1, FIG. As shown, 13 obtained from a photoelectric conversion element group of 5 rows × 5 columns, for example, including a first photoelectric conversion element 30 and a second photoelectric conversion element 31 arranged two-dimensionally around the first photoelectric conversion element 30. The pieces of pixel data (hereinafter also referred to as two-dimensional pixel data) are simultaneously output to the data correction unit 9. The two-dimensional pixel data holding / outputting unit 8 has a configuration in which four line memories 81 and 13 delay elements 82 composed of flip-flops are connected as shown in the figure.

FIG. 4 is a block diagram showing a schematic configuration of the data correction unit shown in FIG.
The data correction unit 9 acquires color information and defect information added to each of the 13 pieces of pixel data output from the two-dimensional pixel data holding output unit 8, and based on these, the first photoelectric conversion element 30 is acquired. Data processing content determination unit 91 for determining the data processing content to be performed on the first pixel data obtained from the data processing, and data processing for instructing data processing with the content determined by the data processing content determination unit 91 A content instruction unit 92 and a data correction processing unit 93 that performs data correction processing such as defect correction processing or NR processing on the first pixel data in accordance with an instruction from the data processing content instruction unit 92 are provided.

  The data correction processing unit 93 includes a plurality of arithmetic units including an adder, a subtractor, a multiplier, a comparator, and the like used for defect correction processing and NR processing, and selects an optimal arithmetic unit according to the content of the data correction processing. Select to execute arithmetic processing. Note that the data correction processing unit 93 may separately have an arithmetic unit necessary for performing defect correction processing and an arithmetic unit necessary for performing NR processing. The data correction processing unit 93 corresponds to the defect correction processing means and the noise reduction processing means in the claims.

FIG. 5 is a diagram showing a schematic configuration of the data processing content determination unit 91 shown in FIG.
The data processing content determination unit 91 includes a defective pixel data number determination unit 91a that determines the number of defective pixel data included in the two-dimensional pixel data based on the defect information, and a two-dimensional data of the defective pixel data included in the two-dimensional pixel data. Defect correction based on the defective pixel data position determining unit 91b for determining the position on the pixel data based on the defect information, the determination result of the defective pixel data number determining unit 91a and the defective pixel data position determining unit 91b, and the color information A defect correction processing selection unit 91c that generates and outputs defect correction processing instruction information for instructing processing, and NR processing instruction information for instructing NR processing based on the determination result and the color information An output NR process selection unit 91d.

  When the first pixel data included in the two-dimensional pixel data is defective pixel data, the NR processing selection unit 91d does not generate NR processing instruction information, and the first pixel data included in the two-dimensional pixel data is the defective pixel. If it is not data, NR processing instruction information is generated and output. The NR processing instruction information is information that specifies what input should be input to what arithmetic expression to perform the NR processing.

  When the first pixel data included in the two-dimensional pixel data is not defective pixel data, the defect correction processing selection unit 91c does not generate defect correction processing instruction information, and the first pixel data included in the two-dimensional pixel data If it is defective pixel data, defect correction processing instruction information is generated and output. The defect correction processing instruction information is information for designating what input should be input to what calculation formula to perform the defect correction processing.

  When the first pixel data included in the two-dimensional pixel data is defective pixel data, the defect correction selection processing unit 91c includes, among the two-dimensional pixel data, the first pixel data, the pixel data of the same color component, A defect pattern determined by the position and number of defective pixel data included in these pixel data is determined, an optimum arithmetic expression is determined based on the determined defect pattern, and defect correction processing instruction information is generated. The defect correction process selection unit 91c corresponds to a defect pattern determination unit and an arithmetic expression determination unit in claims. The defect pattern is determined according to which of the defect pattern determination patterns stored in the internal look-up table (LUT) 911 matches.

  Here, the pattern of the two-dimensional pixel data will be described. In the drawings (FIGS. 6 to 10) used in the following description, an element with “R” indicates R component pixel data, an element with “G” indicates G component pixel data, and “B”. The elements marked with indicate the B component pixel data, and the elements filled in with black indicate defective pixel data.

  As shown in FIG. 6, a case where three defective elements are continuously included in a part of a large number of photoelectric conversion elements will be described as an example. In this case, four patterns as shown in FIGS. 6A to 6D are conceivable as a combination pattern of a large number of photoelectric conversion element arrays and defective element arrays.

  Further, in the case of the pattern shown in FIG. 6A, four patterns as shown in FIGS. 7A to 7D are conceivable in combination with the color filter pattern. When attention is paid to only 13 pixel data output from the two-dimensional pixel data holding / output unit 8, the three patterns shown in FIGS. 8A to 8C are changed from the pattern shown in FIG. The three patterns shown in FIGS. 9A to 9C can be extracted from the pattern shown in FIG. 7B.

  Since three patterns can be extracted in the same manner from the patterns shown in FIGS. 7C and 7D, 12 patterns can be considered using only the patterns shown in FIG. Since each of FIGS. 6B to 6D can have 12 patterns, the solid-state imaging device 1 having three consecutive defective elements as shown in FIG. When only the 13 pixel data output from is focused on, there are 48 patterns in total as the 13 pixel data patterns.

  The defect correction processing selection unit 91c stores these 48 patterns in the internal memory 911 as defect pattern discrimination patterns, and the defect pattern discrimination patterns and 13 actually output from the pixel data holding output unit 8 The defect pattern of the 13 pixel data may be discriminated by comparing with the pattern of the pixel data, but this takes time to discriminate the defect pattern and increases the capacity of the internal memory 911. This will increase the circuit scale.

  Therefore, in this embodiment, 48 patterns are collected into five patterns as shown in FIGS. 10A to 10E and stored in the internal memory 911. The defect correction processing when the first pixel data is defective pixel data is performed using the first pixel data and the pixel data of the same color component as the first pixel data out of 13 pieces of pixel data. . For this reason, the defect correction process is performed using the same calculation formula for the pattern shown in FIG. 8A and the pattern shown in FIG. 9A. The pattern shown in FIG. 8B and the pattern shown in FIG. 9B are the same. The pattern shown in FIG. 8C and the pattern shown in FIG. 9C are different in the color component of the defective pixel data to be processed, but the arithmetic expressions used for the defect correction processing are the same.

  Based on this idea, the defect pattern discrimination pattern is determined by the first pixel data, the pixel data of the same color component, and the position and number of defective pixel data included in these pixel data as shown in FIG. It is enough to have only street patterns. Thereby, the circuit scale can be reduced.

  Instead of discriminating the defective pattern by pattern matching with the five patterns shown in FIG. 10, the number and position of the defective pixel data in the pattern of the pixel data of the same color component of the 13 pixel data are stored in the LUT 911. A plurality of patterns are stored as indices, and defect information attached to 13 pieces of pixel data is input to the LUT 911 to determine which index matches, and the defect pattern can be determined. good. By performing such processing, pipeline processing is possible and high-speed processing is possible.

  FIG. 11 is a diagram illustrating a case where five defective elements are continuously present (a) and a case where seven defective elements are continuously present (b). In such a case, in addition to the pattern shown in FIG. 10, it is possible to discriminate all defect patterns only by adding a defect pattern discriminating pattern as shown in FIG.

  The data correction processing unit 93 performs defect correction processing using the arithmetic expression specified in the defect correction processing instruction information and the input two-dimensional pixel data. Specifically, an arithmetic unit necessary for the arithmetic expression specified by the defect correction processing instruction information is selected, and two-dimensional pixel data is input to the arithmetic unit to perform defect correction processing. As a result, optimum defect processing can be performed according to the defect pattern, and even with a solid-state imaging device in which defective elements are continuously present, highly accurate defect correction is possible.

Hereinafter, a specific example of the data correction processing performed by the data correction processing unit 93 will be described.
First, the operation during NR processing will be described.
As an example, 13 pieces of pixel data as shown in FIG. 13A are output as two-dimensional pixel data. FIG. 13A shows an example in which the first pixel data is a G component, and pixel data other than the G component is not labeled. In this case, since the first pixel data g5 is not defective pixel data, the NR processing selection unit 91d determines an arithmetic expression for performing NR processing on the first pixel data g5, and generates NR processing instruction information To do.

  The NR process selection unit 91d determines one of the arithmetic expression Gnr_nm11 and the arithmetic expression Gnr_nm12 shown in FIG. Then, the data correction processing unit 93 selects an arithmetic unit necessary for the determined arithmetic expression, and performs NR processing on the first pixel data g5 using the selected arithmetic unit.

  The arithmetic expression Gnr_nm11 uses pixel data located in the middle when the pixel data g2, g4, g5, g6, and g8 are arranged in ascending order as an output value, and is a process called a median filter.

  The arithmetic expression Gnr_nm12 performs low-pass filter processing by weighted average processing, and here is configured by two arithmetic units, a 4-input adder and a 2-input adder.

  For example, the NR processing selection unit 91d obtains the maximum value and the minimum value of the pixel data g2, g4, g5, g6, and g8, calculates the difference between them, and if the value is larger than the threshold, Gnr_nm11 is obtained. Determined as an arithmetic expression. Note that although one of the two arithmetic expressions is selected and determined here, only one of Gnr_nm11 and Gnr_nm12 may always be determined as the arithmetic expression in order to simplify the circuit. .

  Although noise can be sufficiently reduced only by performing NR processing using the arithmetic expression Gnr_nm11 or the arithmetic expression Gnr_nm12, even in this NR processing, by performing processing in consideration of the position and number of defective pixel data, Noise can be effectively reduced.

  In this case, the NR process selection unit determines the NR defect pattern determined by the first pixel data included in the two-dimensional pixel data, the pixel data of the same color component, and the position and number of defective pixel data included in these pixel data. What is necessary is just to determine the optimal arithmetic expression according to the determined NR defect pattern by 91d. Note that the number of discrimination patterns for discriminating the NR defect pattern can be reduced by using the same concept as the above-described defect pattern discrimination pattern.

  An example in which the defect pattern for NR is a pattern as shown in FIG. In FIG. 13B, pixel data other than the G component is not labeled. In this example, if the pixel data is 8 bits, the input of the arithmetic expression Gnr_dg1 in which the input of the arithmetic expression Gnr_nm11 is changed to the pixel data g4, g5, g6, 0, and 255 and the input of the 4-input adder of the arithmetic expression Gnr_nm12 is the pixel data. The NR process selection unit 91d determines one of the arithmetic expressions Gnr_dg2 changed to g1, g3, g7, and g9 as the arithmetic expression.

  The arithmetic units used for the arithmetic expression Gnr_dg1 can use all the arithmetic units used for the arithmetic expression Gnr_nm11, and the arithmetic units used for the arithmetic expression Gnr_dg2 can use all the arithmetic units used for the arithmetic expression Gnr_nm12. As described above, the arithmetic unit to be used is not changed, and the content of the NR process can be changed only by changing the input. Therefore, even when performing the NR process considering the defective pixel data, the data correction processing unit 93 There is no need to increase the circuit scale.

Next, the operation at the time of defect correction processing will be described.
A case where the defect pattern determined by the defect correction processing selection unit 91c is as shown in FIG. In FIG. 13C, pixel data other than the G component is not labeled. In this example, when the pixel data is 8 bits, out of the inputs of the arithmetic expression Gnr_nm11, either the arithmetic expression Gdc_spg1 in which the pixel data g5 is changed to 0 and the pixel data g6 is changed to 255 or the arithmetic expression Gdc_spg2 is changed. Then, the defect correction process selection unit 91c determines as an arithmetic expression. Then, the data correction processing unit 93 selects an arithmetic unit necessary for the determined arithmetic expression, and performs defect correction processing on the first pixel data g5 using the selected arithmetic unit.

  The arithmetic unit used for the arithmetic expression Gdc_spg1 can use all the arithmetic units used for the arithmetic expression Gnr_nm11, and the arithmetic unit used for the arithmetic expression Gdc_spg2 can use a part of the arithmetic unit used for the arithmetic expression Gnr_nm12. As described above, since at least a part of the arithmetic unit used for the NR process can be used as at least a part of the arithmetic unit used for the defect correction process, the circuit scale of the data correction processing unit 93 can be reduced. is there.

  As described above, according to the digital camera 100 of the present embodiment, even when a plurality of defective elements exist continuously in the solid-state imaging device 1, the position and number of defective elements and the color filter pattern are determined. Since the defect pattern is discriminated and the defect correction processing is performed using the optimum arithmetic expression corresponding to the discriminated defect pattern, it is possible to perform highly accurate defect correction.

  In addition, since the optimum defect correction and NR correction described above can be performed by pipeline processing in this embodiment, it is particularly effective for a digital camera having a moving image shooting mode that requires high-speed processing.

  Further, according to the digital camera 100 of the present embodiment, the number of defect pattern discrimination patterns stored in the internal memory 911 can be reduced as much as possible, so that the circuit scale can be reduced.

  In addition, according to the digital camera 100 of the present embodiment, the defect correction process is not performed during the NR process, and the NR process is not performed during the defect correction process. Since only the necessary ones are selected and used from the plurality of arithmetic units during the defect correction process, the data correction processing unit 93 has the arithmetic units necessary for the NR process and the defect correction process. It is not necessary to have an arithmetic unit necessary for the circuit independently, and the circuit scale can be greatly reduced.

  Further, according to the digital camera 100 of the present embodiment, even if the defect correction processing of the defective pixel data is not normally performed, the defective pixel data becomes data replaced by normal pixel data by the replacement unit 5. Therefore, it is possible to avoid a situation in which the image quality is greatly lowered.

(Second embodiment)
The digital camera for describing the second embodiment of the present invention is obtained by slightly changing the configuration of the data processing content determination unit 91 in the digital camera 100 described in the first embodiment.

  FIG. 14 is a diagram showing a schematic configuration of a data processing content determination unit of the digital camera for explaining the second embodiment of the present invention. In FIG. 14, the same components as those in FIG.

  The data processing content determination unit 91 shown in FIG. 14 generates luminance information of the two-dimensional pixel data in the configuration of the data processing content determination unit 91 shown in FIG. 5 and uses this for an equation for defect correction processing or NR processing. A luminance information generation unit 91e that is output as decision condition information is added, the defect correction process selection unit 91c is changed to 91c ′, and the NR process selection unit 91d is changed to an NR process selection unit 91d ′.

  The defect correction processing selection unit 91c ′ determines an optimal arithmetic expression based on the defect pattern and the luminance information output from the luminance information generation unit 91e.

  The NR process selection unit 91d 'determines an optimal arithmetic expression based on the NR defect pattern and the luminance information output from the luminance information generation unit 91e. The NR process selection unit 91d 'may be left as the NR process selection unit 91d, and the luminance information may not be output from the luminance information generation unit 91e to the NR process selection unit 91d.

Hereinafter, a specific example of the data correction processing performed by the data correction processing unit 93 will be described.
First, the operation during NR processing will be described.
As an example, 13 pieces of pixel data as shown in FIG. 15A are output as two-dimensional pixel data. In FIG. 15A, the pixel data other than the G component is not labeled. The luminance information generation unit 91e has an absolute value A of a difference between a value obtained by adding the luminance value of the pixel data g1 and the luminance value of the pixel data g3, and a value obtained by adding the luminance value of the pixel data g7 and the luminance value of the pixel data g9. The absolute value B of the difference between the value obtained by adding the luminance value of the pixel data g1 and the luminance value of the pixel data g7 and the value obtained by adding the luminance value of the pixel data g3 and the luminance value of the pixel data g9 is calculated. The value A and the absolute value B are input to the NR process selection unit 91d ′.

  The NR process selection unit 91d ′ discriminates the NR defect pattern, and based on the discriminated NR defect pattern, three arithmetic expressions (arithmetic expression Gnr_dg3, arithmetic expression Gdc_spg2, and arithmetic expression Gnr_dg4) shown in FIG. The optimum arithmetic expression is determined based on the absolute value A and the absolute value B.

  Specifically, when the absolute value A is larger than the absolute value B and the absolute value A is larger than the threshold value Th1, the NR processing selection unit 91d ′ determines the arithmetic expression Gnr_dg3 as the arithmetic expression, and the absolute value A is the absolute value. If it is larger than B and the absolute value A is smaller than the threshold Th1, the arithmetic expression Gdc_spg2 is determined as the arithmetic expression. Further, when the absolute value B is larger than the absolute value A and the absolute value B is larger than the threshold Th1, the arithmetic expression Gnr_dg4 is determined as the arithmetic expression, the absolute value B is larger than the absolute value A, and the absolute value When B is smaller than the threshold Th1, the arithmetic expression Gdc_spg2 is determined as the arithmetic expression. Then, the data correction processing unit 93 selects an arithmetic unit necessary for the determined arithmetic expression, and performs NR processing on the first pixel data g5 using the selected arithmetic unit.

  By doing in this way, in the two-dimensional pixel data shown in FIG. 15A, when the brightness of the pixel data is greatly changed in the first and fifth columns, or in the first and fifth rows Even when the brightness of the data changes greatly, it is possible to perform an optimal NR process considering the change.

Next, the operation at the time of defect correction processing will be described.
As an example, 13 pieces of pixel data as shown in FIG. 15B are output as two-dimensional pixel data. In FIG. 15B, pixel data other than the G component is not labeled. The luminance information generation unit 91e calculates the absolute value A and the absolute value B described above, and inputs them to the defect correction process selection unit 91c ′.

  The defect correction processing selection unit 91c ′ discriminates the defect pattern, determines three arithmetic expressions (arithmetic expression Gdc_spg1, arithmetic expression Gdc_spg2, and arithmetic expression Gdc_spg3) shown in FIG. 15B based on the determined defect pattern, An optimum arithmetic expression is determined based on the absolute value A and the absolute value B.

  Specifically, when the absolute value A is larger than the absolute value B and the absolute value A is larger than the threshold Th2, the defect correction processing selection unit 91c ′ determines the arithmetic expression Gdc_spg1 as the arithmetic expression, and the absolute value A is absolute. When the value is larger than the value B and the absolute value A is smaller than the threshold value Th2, the arithmetic expression Gdc_spg2 is determined as the arithmetic expression. Further, when the absolute value B is larger than the absolute value A and the absolute value B is larger than the threshold Th2, the arithmetic expression Gdc_spg3 is determined as the arithmetic expression, the absolute value B is larger than the absolute value A, and the absolute value When B is smaller than the threshold Th2, the arithmetic expression Gdc_spg2 is determined as the arithmetic expression. Then, the data correction processing unit 93 selects an arithmetic unit necessary for the determined arithmetic expression, and performs defect correction processing on the first pixel data g5 using the selected arithmetic unit.

  By doing in this way, in the two-dimensional pixel data shown in FIG. 15B, when the brightness of the pixel data is greatly changed in the first and fifth columns, or in the first and fifth rows Even when the luminance of the data changes greatly, it is possible to perform an optimal defect correction process in consideration of the change.

  In the above description, the optimal expression is determined with reference to the luminance value of the pixel data of the same color as the first pixel data to be corrected, but the present invention is not limited to this. For example, when the first pixel data to be corrected is an R component or a B component, it has a larger influence on the luminance than when referring to the luminance values of the R component and B component pixel data of the same color component. More optimal data correction is possible by referring to the luminance value of the G component pixel data. Hereinafter, the defect correction processing operation in such a case will be described.

  As an example, 13 pieces of pixel data as shown in FIG. 15C are output as two-dimensional pixel data. The luminance information generation unit 91e includes the luminance value MIN of the pixel data having the minimum luminance value among the G component pixel data gr1 to gr3 and the luminance value of the pixel data having the maximum luminance value among the G component pixel data gr1 to gr3. MAX is extracted and input to the defect correction processing selection unit 91c ′.

  The defect correction process selection unit 91c ′ discriminates the defect pattern and, based on the discriminated defect pattern, five arithmetic expressions (arithmetic expression Rdc_sprb1, arithmetic expression Rdc_sprb2, arithmetic expression Rdc_sprb3, arithmetic expression Rdc_sprb4, An arithmetic expression Rdc_sprb5) is determined, and an optimal arithmetic expression is determined based on the luminance value MAX and the luminance value MIN.

  Specifically, when the difference between the luminance value MAX and the luminance value MIN is smaller than the threshold value Th3, the defect correction process selection unit 91c 'determines the arithmetic expression Rdc_sprb5 as an optimal arithmetic expression. Further, the difference between the luminance value MAX and the luminance value MIN is larger than the threshold value Th3, the luminance value MAX is one of the pixel data gr2 and gr3, and the absolute value of the difference between the luminance values of the pixel data gr2 and gr3. Is smaller than the threshold Th4, the arithmetic expression Rdc_sprb1 is determined as an optimal arithmetic expression. When the difference between the luminance value MAX and the luminance value MIN is larger than the threshold value Th3 and the luminance value MAX is the luminance value of the pixel data gr1, the arithmetic expression Rdc_sprb2 is determined as an optimal arithmetic expression. When the difference between the luminance value MAX and the luminance value MIN is larger than the threshold value Th3 and the luminance value MAX is the luminance value of the pixel data gr2, the arithmetic expression Rdc_sprb3 is determined as the optimum arithmetic expression. When the difference between the luminance value MAX and the luminance value MIN is larger than the threshold value Th3 and the luminance value MAX is the luminance value of the pixel data gr3, the arithmetic expression Rdc_sprb4 is determined as the optimum arithmetic expression. Then, the data correction processing unit 93 selects an arithmetic unit necessary for the determined arithmetic expression, and performs defect correction processing on the first pixel data r5 using the selected arithmetic unit.

  By doing in this way, a more accurate defect correction process becomes possible.

  Note that the arithmetic expressions used in the defect correction process and the NR process described in the first and second embodiments are examples, and various other arithmetic expressions can be used.

  In the first and second embodiments, the solid-state imaging device having a large number of photoelectric conversion elements that detect three different colors is taken as an example, but a large number of photoelectric conversions that detect two or four or more different colors. The same defect correction processing and NR processing can be performed also in a digital camera equipped with a solid-state imaging device having an element.

The figure which shows the principal part schematic structure of the digital camera for describing 1st embodiment of this invention. The block diagram which shows schematic structure inside the substitution part shown in FIG. The figure which shows the arrangement | sequence of the photoelectric conversion element of the production | generation origin of the pixel data output from the two-dimensional pixel data holding output part shown in FIG. The block diagram which shows schematic structure of the data correction part shown in FIG. The figure which shows schematic structure of the data processing content determination part 91 shown in FIG. Diagram for explaining the defect pattern Diagram for explaining the defect pattern Diagram for explaining the defect pattern Diagram for explaining the defect pattern The figure which shows the pattern for defect pattern discrimination The figure which shows the modification of the defect pattern The figure which shows the pattern for defect pattern discrimination added in the case of the defect pattern of FIG. The figure for demonstrating the operation | movement at the time of the NR process in the digital camera of 1st embodiment, and a defect correction process The figure which shows schematic structure of the data processing content determination part of the digital camera for describing 2nd embodiment of this invention The figure for demonstrating the operation | movement at the time of the NR process in the digital camera of 2nd embodiment, and a defect correction process

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Solid-state image sensor 2 A / D conversion part 3 Buffer 4 Timing generator 5 Replacement part 6 H / V counter 7 LUT
8 Two-dimensional pixel data holding / output unit 9 Data correction unit 10 Signal processing unit 93 Data correction processing unit 91 Data processing content determination unit 91a Defective pixel data number determination unit 91b Defective pixel data position determination unit 91c Defect correction processing selection unit 91d Noise reduction Processing selection part

Claims (8)

A data correction processing device that performs correction processing of a large number of pixel data obtained from a solid-state imaging device including a large number of photoelectric conversion elements arranged in a row direction on a semiconductor substrate and in a column direction orthogonal thereto,
A first photoelectric conversion element that temporarily holds the pixel data and is included in the solid-state imaging device; and a second photoelectric conversion element that is two-dimensionally arranged around the first photoelectric conversion element; Two-dimensional pixel data holding output means for simultaneously outputting two-dimensional pixel data which is pixel data obtained from a photoelectric conversion element group including:
When the first photoelectric conversion element is a defective element, pixel data obtained from other than the photoelectric conversion element that detects a color different from the first photoelectric conversion element among the two-dimensional pixel data; and Defect pattern determining means for determining a defect pattern determined by the position and number of defective pixel data obtained from the defective element included in the pixel data;
An arithmetic expression determining means for determining an arithmetic expression for defect correction processing based on the defect pattern;
A data correction processing apparatus comprising: a defect correction processing unit that performs a defect correction process on the first pixel data that is pixel data obtained from the first photoelectric conversion element using the two-dimensional pixel data and the arithmetic expression. The data correction processing device according to claim 1,
The calculation formula determining means determines the calculation formula based on the defect pattern and a luminance value of pixel data excluding the defective pixel data among pixel data constituting the defect pattern. The data correction processing device according to claim 1 or 2,
A data correction processing apparatus comprising pixel data replacing means for replacing the defective pixel data of pixel data input to the pixel data holding / outputting means with pixel data obtained from photoelectric conversion elements around the defective element. The data correction processing device according to any one of claims 1 to 3,
A data correction processing apparatus comprising noise reduction processing means for performing noise reduction processing of the first pixel data using the two-dimensional pixel data when the first photoelectric conversion element is not the defective element. The data correction processing device according to any one of claims 1 to 3,
The defect correction processing means includes a plurality of arithmetic units, and selects and uses an arithmetic unit necessary for the arithmetic expression from the plurality of arithmetic units. The data correction processing apparatus according to claim 4, wherein
The defect correction processing means includes a plurality of arithmetic units, and selects and uses an arithmetic unit necessary for the arithmetic expression from the plurality of arithmetic units. The data correction processing apparatus according to claim 6, wherein
The data correction processing device, wherein the noise reduction processing means performs the noise reduction processing using at least a part of the plurality of arithmetic units included in the defect correction processing means. A data correction processing method for correcting a large number of pixel data obtained from a solid-state imaging device including a large number of photoelectric conversion elements arranged in a row direction on a semiconductor substrate and in a column direction perpendicular thereto,
A first photoelectric conversion element that temporarily holds the pixel data and is included in the solid-state imaging device; and a second photoelectric conversion element that is two-dimensionally arranged around the first photoelectric conversion element; A two-dimensional pixel data holding output step for simultaneously outputting two-dimensional pixel data that is pixel data obtained from a photoelectric conversion element group including:
When the first photoelectric conversion element is a defective element, pixel data obtained from other than the photoelectric conversion element that detects a color different from the first photoelectric conversion element among the two-dimensional pixel data; and A defect pattern determining step for determining a defect pattern determined by the position and number of defective pixel data obtained from the defective element included in the pixel data;
An arithmetic expression determining step for determining an arithmetic expression for defect correction processing based on the defect pattern;
A data correction processing method including a defect correction processing step of performing defect correction processing of first pixel data that is pixel data obtained from the first photoelectric conversion element using the two-dimensional pixel data and the arithmetic expression.

Images