JP4903246B2 - Brightness correction circuit for image sensor - Google Patents

Description

  The present invention relates to a luminance correction circuit for an image sensor that is suitable for use in a machine vision system that requires a high-quality captured image without luminance unevenness.

  In a general-purpose simplified camera using a solid-state image sensor, the pixel output of the solid-state image sensor is directly handled as a frame image, and predetermined image processing such as luminance correction and color correction is performed on a frame basis. In contrast, machine vision systems that require high-definition and high-quality captured images require a brightness correction circuit that takes into account variations in the characteristics of solid-state image sensors that are not installed in general-purpose simple cameras. To do.

  The output image level of an image pickup device typified by a CCD, CMOS, or the like differs in offset level and sensitivity for each pixel depending on the characteristics of the semiconductor. This is output as noise. In particular, the dark signal and highlight signal fluctuate due to changes in exposure time and temperature. Therefore, if the signal processing in the subsequent stage is performed with the dark and highlight conditions fixed to one, the exposure time to be photographed and so on for each pixel. Or, the output image varies in brightness over the entire captured image. It is an object to reduce noise by correcting these phenomena in the subsequent stage of the image sensor. For example, in the case of a CMOS image sensor, a fixed transistor is assigned to each pixel. Therefore, a noise of a fixed pattern is caused by a difference (variation) in characteristics of each transistor.

  Conventionally, as a luminance correction means due to variations in characteristics of this type of image sensor, a black correction memory having a frame size is prepared for each of a plurality of settable exposure times as a first correction means. There has been means for storing correction values associated with each exposure time in the correction memory, obtaining correction values corresponding to the exposure time during actual image capturing from the black correction memory, and correcting the dark level.

  Further, as the second correction means, in the case of long exposure, since a high-speed response process is not required, a typical dark frame image and an exposure time range of long exposure (1 second to 64 seconds at 1 second intervals). The correction value corresponding to each exposure time was stored, the correction value corresponding to the exposure time at the time of actual image capturing was calculated from these data, and dark level correction means for long-time exposure existed .

  However, in the first correction means described above, a black correction memory having a frame size must be prepared for each of a plurality of settable exposure times. Therefore, the memory capacity for correction becomes enormous and the hardware configuration is increased. There was a problem of becoming complicated. For example, when 1024 gradation image correction is performed on an image sensor with 12 million pixels, a correction memory for 12 million pixels that holds 2 bytes of luminance information per pixel can be set as the number of exposure time units that can be set. It is necessary to provide as many areas corresponding to the above, and the hardware configuration including the memory mechanism becomes extremely complicated. Further, the second correction means described above has a problem that it is not applicable to multi-stage exposure including high-speed exposure (for example, 1/20000 seconds) even though it can be applied to long-time exposure.

  Further, as dark level correction means, conventionally, there has been means for performing black level correction uniquely on the entire screen using a part or all of the light shielding level of the light shielding area arranged around the effective screen of the image sensor. However, this correction means has a configuration in which the shading level of a partially sampled area is reflected on the entire screen, and correction is not performed for variations in black level (brightness unevenness) in pixel units of the image sensor. Therefore, it cannot be applied to a machine vision system that requires a high-quality photographed image without uneven brightness.

JP 2001-094882 A JP 2007-259135 A

  As described above, in a machine vision system that requires high-quality captured images without uneven brightness, when implementing a brightness correction circuit that takes into account variations in characteristics of solid-state image sensors and high-speed processing, There is a problem in that the scale is increased, the software is complicated, and the cost is increased.

  The present invention has been made in view of the above circumstances, and in a brightness correction circuit of an image sensor applied to a machine vision system that requires a high-quality captured image without unevenness of brightness, high-speed processing is performed with a simple hardware configuration. An object of the present invention is to provide a luminance correction circuit for an image sensor that enables multi-level gradation luminance correction.

The present invention relates to luminance correction for the purpose of removing noise associated with variations in characteristics of image units in an image sensor, rather than luminance correction for an image in frame units, and an image sensor that constitutes an area image sensor A clamp circuit that holds a DC voltage level of a dark signal that is a reference of an image signal output from the image signal, and a gradation circuit that is provided at a subsequent stage of the clamp circuit and has a gradation bit number n exceeding 1 byte output from the image sensor for one image signal for each pixel that is, the biased levels allocated to the gradation range of low tone set in advance, the number of smaller gradation bits than the gradation number of bits n of the image signal m (m < consists of n), and using a non-linear gradation correction coefficient correction coefficients are allocated more than the correction coefficient of the high tone of the low gradation side, correcting the tone of the image signal for each pixel Dakurebe A correction circuit provided in a subsequent stage of the dark level correction circuit, the image signal for each pixel output from the imaging device, and biased levels allocated to the tone range of the high tone set in advance, using the gradation compensation coefficient of the gradation bit number m, and the highlight level correction circuit for correcting the gradation of the image signal for each pixel, disposed downstream of the highlight level correction circuit, said image sensor for the pixel defect designated coordinates by a defect image coordinate holding means for holding a defective pixel at the coordinate value, and includes a pixel defect correction processing unit for interpolating the luminance value based on the luminance values of the surrounding pixels, the It is characterized by a luminance correction circuit of the image sensor characterized by the above.

  According to the present invention, in a brightness correction circuit of an image sensor applied to a machine vision system that requires a high-quality captured image without brightness unevenness, the brightness of multi-value gradation by high-speed processing with a simple hardware configuration It is possible to provide a luminance correction circuit for an image sensor that enables correction.

1 is a block diagram showing an overall configuration of a luminance correction circuit according to an embodiment of the present invention. FIG. 3 is a block diagram showing a specific circuit configuration of the luminance correction circuit according to the embodiment. The figure which shows the image output histogram in the imaging surface light-shielding state for demonstrating the dark level correction circuit which concerns on the said embodiment. The figure which shows the image output histogram in the imaging surface uniform light irradiation state for demonstrating the highlight level correction circuit which concerns on the said embodiment. The figure which shows the image output histogram for demonstrating the effect | action of the dark level correction circuit which concerns on the said embodiment. The figure which shows the image output histogram for demonstrating the effect | action of the highlight level correction circuit which concerns on the said embodiment. Explanatory drawing for demonstrating operation | movement of the dark level correction circuit which concerns on the said embodiment. Explanatory drawing for demonstrating operation | movement of the dark level correction circuit which concerns on the said embodiment. Explanatory drawing for demonstrating operation | movement of the dark level correction circuit which concerns on the said embodiment. Explanatory drawing for demonstrating operation | movement of the dark level correction circuit which concerns on the said embodiment. Explanatory drawing for demonstrating operation | movement of the highlight level correction circuit which concerns on the said embodiment. Explanatory drawing for demonstrating operation | movement of the highlight level correction circuit which concerns on the said embodiment. Explanatory drawing for demonstrating operation | movement of the highlight level correction circuit which concerns on the said embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 shows an overall configuration of a luminance correction circuit according to an embodiment of the present invention, and FIG. 2 shows a specific circuit configuration of the luminance correction circuit. Note that the brightness correction circuit of the image sensor according to the embodiment of the present invention is applied to both an industrial camera system that handles still images or an industrial camera system that handles moving images, for example, for surveillance and for production lines. It is possible, but here, after the brightness correction processing of the image sensor, the subsequent image processing centered on the frame image, the correction processing for the color image, the specific configuration of the imaging camera mechanism, etc. will be described. Omitted. Further, in this embodiment, an image pickup device in which a photoelectric conversion unit (image sensor unit) having an image pickup surface and a sensor peripheral circuit including an A / D conversion unit that digitally outputs a photoelectrically converted signal are system-on-chip. However, an image pickup device configuration in which a peripheral circuit including an A / D conversion unit is externally attached to the photoelectric conversion unit may be used.

  As shown in FIG. 1, the present invention provides a clamp circuit (V that holds a DC voltage level of a dark signal that serves as a reference for a black level (dark level) of an image signal output from an image sensor 11 that constitutes an area image sensor. -CLAMP) 12 and the image signal for each pixel output from the image pickup device 11 provided at the subsequent stage of the clamp circuit 12, the level distribution is biased to a preset gradation range on the low gradation side Further, the gradation is corrected pixel by pixel using a non-linear gradation correction coefficient having a plurality of refraction points configured with a gradation bit number (m) smaller than the gradation bit number (n) of the image signal. The dark level correction circuit 13 and the dark level correction circuit 13 are provided at a subsequent stage, and the level distribution is performed on the gradation range on the high gradation side for the image signal for each pixel output from the image sensor 11. Biased A highlight level correction circuit 14 that corrects gradation for each pixel using a gradation correction coefficient having a gradation bit number (m) smaller than the gradation bit number (n) of the image signal, and this highlight level correction The pixel defect of the image signal corresponding to the defective pixel is provided in the subsequent stage of the circuit 14, holds the defective pixel of the image sensor 11 as a coordinate value, and the image signal for each pixel output from the image sensor 11 It features a luminance correction circuit for an image pickup device including a pixel defect correction circuit 15 that corrects the above.

  In this embodiment, the image signal output from the image sensor 11 is composed of, for example, n = 10 bits (1024 gradations) exceeding 1 byte (8 bits), and the dark level correction circuit 13 and the highlight level correction circuit 14. The gradation correction coefficient handled by 1 is composed of, for example, m = 8 bits (256 gradations) not exceeding 1 byte. The dark level correction circuit 13 and the highlight level correction circuit 14 each of the 8-bit gradation correction coefficient data in the pre-processing before product shipment (or may be performed separately according to the environment by the user). Generation processing and storage processing of gradation correction coefficient data are performed, and gradation correction processing using the gradation correction coefficient is performed in normal photographing processing after product shipment.

  The dark level correction circuit 13 performs leveling in a gradation range on the low gradation side set in advance based on the content of 10-bit gradation data output from the image sensor 11 in pre-processing described later before product shipment. A conversion table (ENC) for obtaining non-linear 8-bit tone correction coefficient data having a plurality of refraction points whose distribution is biased and the 8-bit tone correction coefficient data for one frame are stored, A dark correction coefficient storage means for sequentially reading out and outputting the stored 8-bit gradation correction coefficient data for one frame in synchronization with the pixel output of the captured image during normal shooting operation, and the dark correction coefficient storage A conversion table for converting the 8-bit nonlinear gradation correction coefficient data output from the means into 10-bit linear gradation correction coefficient data in accordance with the number of gradation bits of the image signal ( EC) and a dark image signal correction calculation unit that performs gradation correction by subtracting (−) the image signal output from the image sensor 11 using the gradation correction coefficient output from the conversion table (DEC). Configured.

  The conversion table (ENC) is realized by the dark image correction memory normalization table 13a shown in FIG. 2, and the dark correction coefficient storage means includes the correction data storage nonvolatile memory 13f and the dark image correction data frame shown in FIG. The conversion table (DEC) is realized by the dark image correction inverse conversion table 13c shown in FIG. 2, and the dark image signal correction calculation unit is realized by the dark image signal correction calculation unit 13e shown in FIG. Is done.

  The highlight level correction circuit 14 is the content of 10-bit gradation data output from the image sensor 11 in pre-processing described later before product shipment (or may be performed separately by the user according to the environment). Based on the above, a conversion table (ENC) for obtaining 8-bit gradation correction coefficient data in which the level distribution is biased to a gradation range on the high gradation side set in advance, and this 8-bit gradation correction coefficient data as 1 Highlight correction for storing frames, and reading and outputting the stored 8-bit gradation correction coefficient data for each frame sequentially in synchronization with the pixel output of the captured image during normal shooting operation after product shipment Coefficient storage means and 10-bit gradation correction coefficient data obtained by matching the 8-bit gradation correction coefficient data output from the highlight correction coefficient storage means with the number of gradation bits of the image signal. A highlight table that performs gradation correction by multiplying (×) the conversion table (DEC) to be converted into the image and the image signal output from the image sensor 11 using the gradation correction coefficient output from the conversion table (DEC). And a signal correction calculation unit.

  The conversion table (ENC) is realized by the highlight image correction memory normalization table 14a shown in FIG. 2, and the highlight correction coefficient storage means includes the correction data storage nonvolatile memory 14f and the highlight image correction shown in FIG. The conversion table (DEC) is realized by the highlight image correction inverse conversion table 14c shown in FIG. 2, and the highlight image signal correction calculation unit is the highlight image signal shown in FIG. This is realized by the correction calculation unit 14e.

  The pixel defect correction circuit 15 includes a defective pixel coordinate holding unit that holds a defective pixel as a coordinate value, and a luminance value of a surrounding pixel with respect to a pixel defect that is coordinate-designated by the defective pixel coordinate holding unit. And a pixel defect correcting means for performing interpolation processing. The defective pixel coordinate holding means is realized by the defective pixel coordinate storing nonvolatile memory 15a and the defective pixel coordinate reproduction memory 15b shown in FIG. 2, and the pixel defect correcting means is realized by the pixel defect correction processing unit 15c shown in FIG. .

  FIG. 2 shows circuit components of the main part of the luminance correction circuit of the image sensor shown in FIG. Each component shown in FIG. 1 is controlled under the control of the CPU 20 shown in FIG. 2 via the camera synchronization signal generation circuit 21, the exposure time control circuit 22, the memory control circuits 23 and 24, and the like shown in FIG. The operation is controlled synchronously by predetermined timing control.

  The image sensor 11 is configured by an area image sensor such as a CCD image sensor (Charge Coupled Device Image Sensor) or a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor). Here, an image sensor is shown by way of example of a CMOS image sensor in which a photoelectric conversion unit (image sensor unit) and a sensor peripheral circuit including an A / D conversion unit that digitally outputs a photoelectrically converted signal are system-on-chip. However, an image sensor configuration in which peripheral circuits including an A / D conversion unit are externally attached to the photoelectric conversion unit (image sensor unit) may be used.

In this embodiment, as shown in FIG. 2, a photoelectric conversion unit (image sensor unit) 11a having an imaging surface that forms an image via a lens, and an analog signal preprocessing unit that amplifies an output signal of the photoelectric conversion unit 11a. 11b, an A / D converter 11c that performs analog / digital conversion on the output signal of the analog signal preprocessing unit 11b, a horizontal / vertical drive unit 11d that controls driving of the photoelectric conversion unit 11a, and an operation timing within the sensor. The image pickup device 11 is configured to include a sensor synchronization signal generation unit 11e. The sensor synchronization signal generation unit 11e controls the operation of the analog signal preprocessing unit 11b, the A / D conversion unit 11c, and the horizontal / vertical drive unit 11d under the control of the camera synchronization signal generation circuit 21. The A / D conversion unit 11c receives the analog amount of the image signal output from the photoelectric conversion unit 11a via the analog signal preprocessing unit 11b, and outputs an image signal converted into a digital value in units of pixels. Here, an image signal having a gradation bit number of 10 bits (1024 gradations) is output.

  The clamp circuit (V-CLAMP) 12 holds the DC voltage level of the dark signal that is a reference signal for stabilizing and processing the dark level (black level) as a digital value. Based on the output of the light-shielding pixels provided around a part of the effective pixel area to be formed (pixel output of the photoelectric conversion element that blocks the incident light), the level fluctuation due to temperature rise etc. is corrected and the black level reference Holds the dark signal and outputs it to the subsequent circuit.

  The dark level correction circuit 13 is subsequent to the clamp circuit 12, and includes a dark image correction memory normalization table 13a, a dark image correction data frame memory 13b, a dark image correction reverse conversion table 13c, and an exposure time correction. A processing unit 13d, a dark image signal correction calculation unit 13e, and a correction data storage nonvolatile memory 13f are provided. Each component of the dark level correction circuit 13 is controlled under the control of the CPU 20 through the camera synchronization signal generation circuit 21, the exposure time control circuit 22, the memory control circuit 23, and the like. The CPU 20 controls the entire camera, and inputs various setting information and operation information via an input interface (not shown) to control each circuit inside the camera. Here, a trigger signal for starting photographing accompanying an external operation, data (exposure time data) indicating a shutter speed (for example, 1/20000 second to 2 seconds) set by the external operation, and the like are input via the input interface. The CPU 20 sends the input exposure time data to the exposure time control circuit 22. The exposure time control circuit 22 has an exposure time correction coefficient corresponding to a settable shutter speed as a fixed value, and exposure time correction coefficient data corresponding to dark level and highlight level exposure time correction data according to the input exposure time data. The data is sent to the processing units 13d and 14d. The exposure time control circuit 22 has a dark level (dark luminance level), a high level under two or more time conditions between a high-speed exposure time and a low-speed exposure time of an image pickup device (corresponding to a settable shutter speed) according to camera operation. Measureable amount of light level (white if the image sensor is monochrome type, bright brightness level of each color if the primary or complementary color type), and settable shutter speed obtained based on the amount of variation An exposure time correction coefficient correlated to each exposure time is stored in advance as a fixed value.

  The dark image correction memory normalization table 13a is a dark image captured in a state where the image pickup surface of the image pickup element 11 is shielded in a pre-process before product shipment (or may be performed separately according to the environment by the user). Based on the luminance value indicated by the gradation data (10 bits; 1024 gradations) of each pixel (effective pixel) of the image, for each effective pixel of the image sensor 11, a preset low gradation side is set for each pixel. Non-linear gradation correction coefficient data having a plurality of refraction points of 8 bits (256 gradations) whose level distribution is biased to the gradation range is generated and output. The 8-bit gradation correction coefficient data output from the dark image correction memory normalization table 13a is stored in the correction data storage nonvolatile memory 13f under the write control of the memory control circuit 23. . A specific example of processing for generating gradation correction coefficient data by the dark image correction memory normalization table 13a will be described later with reference to FIGS. 3 and 5 and FIGS.

  The dark image correction data frame memory 13b is 8 frames for one frame read from the correction data storing nonvolatile memory 13f under the control of the memory control circuit 23 in the initial setting process in the normal photographing process after product shipment. Write and hold gradation correction coefficient data in bit units. The process of writing the gradation correction coefficient data from the correction data storing nonvolatile memory 13f to the dark image correction data frame memory 13b is for enabling high-speed exposure (for example, 1/20000 seconds). In order to enable high-speed read access of correction coefficient data, the DRAM or the DRAM capable of high-speed read access in one frame of gradation correction coefficient data stored in the correction data storage nonvolatile memory 13f in the initial setting at the time of shooting The image data is held in a dark image correction data frame memory 13b using an SRAM or the like, for example, having a DRAM configuration. The dark image correction data frame memory 13b reads out 8-bit gradation correction coefficient data corresponding to the correction target pixel in synchronization with the shooting operation under the control of the memory control circuit 23 in a normal shooting process. It is sent to the dark image correction reverse conversion table 13c.

  The dark image correction inverse conversion table 13c converts the 8-bit (256 gradations) gradation correction coefficient data input from the dark image correction data frame memory 13b into the gradation bit number of the image data output from the image sensor 11. The combined 10-bit (1024 gradation) gradation correction coefficient data is converted and sent to the exposure time correction processing unit 13d.

  The exposure time correction processing unit 13d receives the gradation correction coefficient data from the dark image correction inverse conversion table 13c and the exposure time correction data from the exposure time control circuit 22 to obtain the value indicated by the gradation correction coefficient data. The value indicated by the exposure time correction data is calculated, and the gradation correction coefficient data after the exposure time correction is sent to the dark image signal correction calculation unit 13e.

  The dark image signal correction calculation unit 13e inputs the image signal output from the image sensor 11 via the clamp circuit 12, and applies the gradation (luminance value) of the correction target pixel via the exposure time correction processing unit 13d. Then, the value indicated by the gradation correction coefficient data input is added / subtracted to correct the gradation of the correction target pixel. The image signal corrected by the dark image signal correction calculation unit 13e is sent to a highlight level correction circuit 14 provided at the subsequent stage of the dark level correction circuit 13.

The highlight level correction circuit 14 is subsequent to the dark level correction circuit 13, and includes a highlight image correction memory normalization table 14a, a highlight image correction data frame memory 14b, and a highlight image correction reverse conversion table 14c. An exposure time correction processing unit 14d, a highlight image signal correction calculation unit 14e, and a correction data storing nonvolatile memory 14f. Each component of the highlight level correction circuit 14 is controlled under the control of the CPU 20 through the camera synchronization signal generation circuit 21, the exposure time control circuit 22, the memory control circuit 24, and the like.

  The normalization table 14a for the highlight image correction memory has a luminance value that is saturated on the imaging surface of the image sensor 11 in pre-processing before product shipment (or may be performed separately according to the environment by the user). On the basis of the luminance value indicated by the gradation data (10 bits; 1024 gradations) of each pixel (effective pixel) of the highlight image taken in the state of irradiating uniform light of a preset highlight level before reaching Then, for all effective pixels of the image sensor 11, for each pixel, 8-bit (256 gradations) gradation correction coefficient data in which the level distribution is biased to a preset gradation range on the high gradation side is generated and output. . The 8-bit gradation correction coefficient data output from the highlight image correction memory normalization table 14a is stored in the correction data storage nonvolatile memory 14f under the write control of the memory control circuit 24. The Note that the tone correction coefficient data generation processing by the highlight image correction memory normalization table 14a will be described later with reference to FIGS. 4 and 6 and FIGS.

  The highlight image correction data frame memory 14b corresponds to one frame read from the correction data storing nonvolatile memory 14f under the control of the memory control circuit 24 in the initial setting process in the normal photographing process after the product is shipped. Write and hold gradation correction coefficient data in units of 8 bits. The gradation correction coefficient data writing process from the correction data storing nonvolatile memory 14f to the highlight image correction data frame memory 14b is for enabling high-speed exposure (for example, 1/20000 seconds). In order to enable high-speed read access of tone correction coefficient data, high-speed read access is possible for one frame of gradation correction coefficient data stored in the correction data storage nonvolatile memory 14f in the initial setting at the time of shooting. For example, it is held in the frame memory 14b for highlight image correction data having a DRAM configuration. The highlight image correction data frame memory 14b reads out 8-bit gradation correction coefficient data corresponding to the correction target pixel in synchronization with a shooting operation under the control of the memory control circuit 24 in a normal shooting process. The highlight image correction inverse conversion table 14c is sent out.

  The inverse conversion table 14c for highlight image correction uses the 8-bit (256 gradations) gradation correction coefficient data input from the highlight image correction data frame memory 14b and the gradation bits of the image data output from the image sensor 11. The data is converted into 10-bit (1024 gradation) gradation correction coefficient data corresponding to the number and sent to the exposure time correction processing unit 14d.

  The exposure time correction processing unit 14d receives the gradation correction coefficient data from the highlight image correction inverse conversion table 14c, the exposure time correction data from the exposure time control circuit 22, and the value indicated by the gradation correction coefficient data. Then, the value indicated by the exposure time correction data is calculated, and the gradation correction coefficient data after the exposure time correction is sent to the highlight image signal correction calculation unit 14e.

  The highlight image signal correction calculation unit 14e receives the gradation input via the exposure time correction processing unit 14d with respect to the gradation (luminance value) of the correction target pixel of the image signal input via the dark level correction circuit 13. The value indicated by the correction coefficient data is added and subtracted to correct the gradation of the correction target pixel. The image signal corrected by the highlight image signal correction calculation unit 14e is sent to the pixel defect correction circuit 15 provided at the subsequent stage of the highlight level correction circuit 14.

  The pixel defect correction circuit 15 includes a defective pixel coordinate storage nonvolatile memory 15a, a defective pixel coordinate reproduction memory 15b, and a pixel defect correction processing unit 15c.

  The non-volatile memory 15a for storing defective pixel coordinates is a coordinate value of a pixel determined as a defective pixel in pixel defect processing of the CPU 20 in pre-processing before product shipment (or may be performed separately according to the environment by the user). (Pixel position of the photoelectric conversion unit 11a) is stored. For example, when detecting a defective pixel of dark (black) level, the defective pixel is determined by photographing a black screen a plurality of times, performing cyclic integration to obtain an average of black, removing a random noise component, and then detecting the black pixel. As a reference, each pixel is binarized at a preset pixel defect level (for example, 512 gradations; 10 bits), a pixel that exceeds the pixel defect level is defined as a defective pixel, and the coordinate value of the defective pixel is stored for defective pixel coordinates. Means for storing in the nonvolatile memory 15a, or in addition to this pixel defect processing, when obtaining the above-mentioned dark image gradation correction coefficient data, a pixel whose luminance value is out of a certain allowable range is regarded as a defective pixel. The allowable range of the set dark level is deviated from the means for storing the coordinate value of the pixel in the non-volatile memory 15a for storing the defective pixel coordinate or the image sensor output of the dark image. Pixels with the specified gradation (luminance value) are detected, and pixels with gradations that deviate from the set highlight level tolerance are detected from the pixel output of the highlight image described above, and each detected pixel is defective. As a pixel, any means such as a means for storing the coordinate value of the pixel in the nonvolatile memory 15a for storing defective pixel coordinates may be used.

  The coordinate values of the pixels stored in the defective pixel coordinate storage nonvolatile memory 15a are configured by, for example, DRAM or SRAM, which can be accessed at high speed under the control of the CPU 20 at the start of imaging processing after product shipment. And stored in the defective pixel coordinate reproduction memory 15b. The writing process of the coordinate data from the defective pixel coordinate storing nonvolatile memory 15a to the defective pixel coordinate reproducing memory 15b is for enabling the above-described high-speed access.

  Based on the pixel defect data (pixel coordinate data) stored in the defective pixel coordinate reproduction memory 15b, the pixel defect correction processing unit 15c performs gradation correction on the defective pixel based on the gradation of surrounding pixels. I do. This gradation correction based on surrounding pixels is an existing processing technique. Note that the post-processing circuit 16 shown in the subsequent stage of the pixel defect correction circuit 15 is a functional circuit that performs various types of image processing on a frame-by-frame basis and is not directly related to the present invention.

  In pre-processing prior to product shipment (or separately performed by the user according to the environment), the CPU 20 follows a command input from the outside, and the camera synchronization signal generation circuit 21 and memory control circuits 23 and 24, 25, the image sensor 11, the dark image correction memory normalization table 13a, the highlight image correction memory normalization table 14a, the correction data storage nonvolatile memories 13f and 14f, and the defective pixel coordinate storage nonvolatile memory. The memory 15a is controlled in operation, 8-bit gradation correction coefficient data for one frame is stored in each of the nonvolatile memories 13f and 14f, and the coordinate value of the defective pixel is stored in the nonvolatile memory 15a for storing defective pixel coordinates. Store.

  In the pre-processing before the product shipment (or the user may separately perform according to the environment), first, an image is captured in a state where the imaging surface of the photoelectric conversion unit 11a is shielded, and this captured image is displayed. The dark level tone correction coefficient data is generated and stored, and then, on the imaging surface of the photoelectric conversion unit 11a, a preset brightness value of about 50 to 70% before the brightness value reaches the saturation level. An image is captured in a state where the uniform light is irradiated, and gradation level correction coefficient data at a highlight level based on the captured image is generated and stored. Thereafter, extraction of defective pixels and storage processing of the coordinate values of the pixels are performed.

  The dark image correction memory normalization table 13a is a luminance value indicated by gradation data (10 bits; 1024 gradations) of each pixel (effective pixel) of a dark image captured in a state where the imaging surface of the photoelectric conversion unit 11a is shielded from light. Based on the above, for all effective pixels of the photoelectric conversion unit 11a, a plurality of 8-bit (256 gradations) refraction points in which the level distribution is biased to a preset gradation range on the low gradation side for each pixel. Is generated, and the gradation correction coefficient data is stored in the correction data storing nonvolatile memory 13f.

  The normalization table 14a for the highlight image correction memory has gradation data (10 bits; 10 bits) of each pixel (effective pixel) of the highlight image captured in a state where the imaging surface of the photoelectric conversion unit 11a is irradiated with preset uniform light. Based on the luminance value indicated by (1024 gradations), for all effective pixels of the photoelectric conversion unit 11a, 8 bits (256th floor) in which the level distribution is biased to a predetermined gradation range on the high gradation side for each pixel. Tone correction coefficient data is generated and output, and the gradation correction coefficient data is stored in the correction data storing nonvolatile memory 14f.

  The CPU 20 performs pixel defect processing, and stores the coordinate value (pixel position of the photoelectric conversion unit 11a) of the pixel determined to be a defective pixel in the defective pixel coordinate storage nonvolatile memory 15a.

  Here, an example of specific processing operations of the dark level correction circuit 13 and the highlight level correction circuit 14 according to the embodiment of the present invention will be described with reference to FIGS.

  The brightness correction circuit of the image sensor according to the embodiment of the present invention includes a clamp circuit 12 that holds (clamps) a DC voltage level of a dark signal that serves as a reference of an image signal output from the image sensor 11, and dark for each pixel. The gray level is corrected for each pixel by the dark level correction circuit 13 and the highlight level correction circuit 14 for correcting the level and the highlight level, and the exposure time correction coefficient data corresponding to the shutter speed output from the exposure time control circuit 22. Exposure time correction processing units 13d and 14d, and a pixel defect correction circuit 15 that corrects pixels (pixel defects) that cannot be processed by dark level correction and highlight level correction.

  Looking at the dark level histogram when the image sensor 11 (photoelectric conversion unit 11a) is photographed in a light-shielded state, the output signal level is widened due to variations in the DC voltage component of each pixel. FIG. 3 shows an example in which a digital value in the case of 10-bit A / D conversion of an analog signal output from the image sensor 11 by the above photographing is used as a histogram. In addition, it clamps by giving arbitrary offset values so that a black component may not become zero or less. In this example, about 40 gradations are used as the clamp level (in FIG. 3, the vertical axis indicates the number of pixels and the horizontal axis indicates the gradation).

Similarly, FIG. 4 shows a histogram when the imaging element 11 is irradiated with uniform light that is not saturated (highlight level, here, equivalent to 744 gradations) and photographed. (In FIG. 4, the vertical axis is the number of pixels, and the horizontal axis is the gradation)
Noise related to these appears as static noise in a grid-like luminance unevenness in an area sensor that captures images in a plane, and in a line sensor and an area sensor that captures images in a linear shape. . If the open width of these histograms is narrow, it can be said that there is little noise due to these uneven brightness, and if the width is wide, there is much noise.

  FIG. 5 shows a dark level histogram when the gradation correction processing according to the embodiment of the present invention is applied to the dark level histogram shown in FIG. As shown in FIG. 5, the number of openings is reduced with about 40 gradations, and it can be seen that noise due to dark level luminance unevenness is improved (in FIG. 5, the vertical axis is the number of pixels, and the horizontal axis is the gradation. ).

  FIG. 6 shows a highlight level histogram when the gradation correction processing according to the embodiment of the present invention is applied to the highlight level histogram shown in FIG. As shown in FIG. 6, the number of openings is reduced around 744 gradations, and it can be seen that noise due to uneven brightness at the highlight level is improved (in FIG. 6, the vertical axis is the number of pixels, and the horizontal axis is the floor. Key).

  The luminance correction processing according to the embodiment of the present invention includes 1) image signal clamping processing, 2) dark level correction processing and exposure time correction processing, 3) highlight level correction processing and exposure time correction processing, and 4) pixel defect correction processing. It is carried out in the order.

[Image signal clamp processing]
For the image signal output from the image sensor 11, the clamp circuit 12 holds the DC voltage level as a dark level reference as a digital value, thereby facilitating stabilization of signal processing in the subsequent stage. This dark level must not be less than 0 gradations, and must always be 0 gradations or more under the use conditions of the camera. Otherwise, correct correction processing will not be performed. If there is a pixel that is optically completely shielded from the image sensor 11 at all times, the level of that pixel is taken, otherwise the reference level is taken with the normal pixel shielded, and a clamping process is performed.

[Dark level correction processing and exposure time correction processing]
The dark level correction process is divided into a prior correction value generation process and a normal imaging process.
1. Correction value generation processing
The correction value generation process is performed according to the following procedure.
1) Dark image capture
The dark image is captured in a state where the image sensor 11 is completely shielded from light. In the example according to this embodiment, a signal subjected to A / D conversion with 10 bits (1024 gradations) is handled. In this imaging, the reference voltage of the A / D converter 11c is adjusted so that the dark level becomes a level with an average value of about 40 gradations.

2) Normalization table conversion
In a camera using the image sensor 11, digital signal processing is mainly performed in order to improve efficiency of signal processing in general. An analog signal from the photoelectric conversion unit 11a serving as the light receiving unit of the image sensor 11 is converted into a digital signal by the A / D converter 11c, and then output through the subsequent image signal processing unit. In order to improve the image quality of the output image signal, it is necessary to increase the number of gradations of this A / D conversion. In the description of the embodiment of the present invention, a digital signal with 1024 gradations is taken up by 10-bit A / D conversion, but 12 bits (4096 gradations), 14 bits (16384 gradations), and 16 bits (65536 gradations). In many cases, image processing is performed using a digital signal. However, for example, when a 9 to 15 bit signal is handled in the circuit configuration, these must be handled as 16 bit data (2 byte data) in units of bytes (8 bits). The circuit configuration and processing including the memory configuration become complicated.

  In the normalization table conversion by the dark image correction memory normalization table 13a, in order to improve the efficiency of the circuit, the table conversion is efficiently performed so that the correction value becomes 8 bits (1 byte), and the nonvolatile data for storing correction data is stored. Data generation for storing in the memory 13f (for setting in the image correction data frame memory 13b) is performed.

  Normally, a 10-bit (1024 gradation) signal can be realized by simple processing if the correction data is handled as 10 bits as shown in FIG. 7 (“10-bit shown by a one-dot chain line in the figure”). Refer to “Linear correction value”). However, if an attempt is made to store the 10-bit signal as it is in the correction data storage nonvolatile memory 13f, the correction data storage nonvolatile memory 13f and the dark image correction data frame memory 13b each have one pixel of 2 bytes (16 bits). It must be handled as a signal, which is inefficient. Therefore, it is necessary to devise somehow to have 8 bits. Here, as shown in FIG. 7, in order to easily convert from 10 bits to 8 bits, the signal on the 10-bit side is realized by 1/4 (“8-bit linear correction value indicated by a solid line in the figure). "reference). However, as can be seen from the dark level histogram before signal correction (see FIG. 3), most of the pixels are gathered to 128 gradations or less, and there are extremely few pixels having gradations higher than 256 gradations. not exist. In addition, when the correction value shown in FIG. 7 is set to ¼, correction unevenness also appears in the corrected value (in FIG. 7, the vertical axis indicates the number of pixels and the horizontal axis indicates the gradation).

  Further, when it is possible to simply correct up to 256 gradations and ignore further gradation correction, pixels having a level of 256 gradations or more are not corrected.

  Therefore, in the embodiment of the present invention, instead of taking a linear correction coefficient as shown in FIG. 7, as shown by a solid line 8-bit correction value in FIG. Then, an optimal 8-bit correction coefficient is obtained by conversion using a table that reduces the coefficient distribution on the high gradation side. This table is realized by the dark image correction memory normalization table 13a (in the histogram shown by a broken line in FIG. 8, the vertical axis indicates the number of pixels, the horizontal axis indicates the gradation, and in the same figure, the one-dot chain line, the two-dot chain line, and the solid line) (10-bit straight line correction value, 8-bit straight line correction value, 8-bit correction value), the vertical axis is normalized table output data, and the horizontal axis is normalized table input gradation data).

  The normalization table shown in this example is one half up to 128 input gradations (up to 128 levels in table values), half up to 256 input gradations (up to 192 levels in table values), and 512 floors of input. The key is converted to one-fourth (the table value is 256 levels).

  A pixel having a dark level of 512 gradations or more, that is, a dark level of 50% or more of all gradations has a large correction amount, which may cause image quality degradation. A pixel to be processed.

  Although the example of this normalization table has three refraction points, it may be a curvilinear curve having no refraction points. However, it should be considered that the coefficient of the high gradation portion of the histogram (see FIG. 3) is 1: 1.

3) Writing to frame memory for dark image correction data
The 8-bit correction coefficient data for one frame is stored in the correction data storage nonvolatile memory 13f via the dark image correction memory normalization table 13a constituting the normalization table. The correction coefficient data for one frame stored in the correction data storing nonvolatile memory 13f is written into the dark image correction data frame memory 13b in the initial setting process at the time of shooting in normal use.

2. Normal imaging processing
The initial setting process during normal imaging is performed according to the following procedure.
1) Reading frame memory for dark image correction data
The readout timing of the dark image correction data frame memory 13b needs to coincide with the imaging synchronization signal output from the imaging device. However, in consideration of the delay time for performing the normalization table inverse transformation, the dark image correction calculation process reads out the coordinates that match.

2) Normalization table inverse transformation
Inverse conversion at the time of normalization table conversion is performed to obtain a calculation value for dark image correction calculation processing.

  In the normalization table conversion, as shown by a solid line (8-bit correction value) in FIG. 8, the gradation is reduced to 1/2 (up to 128 levels in the table value) up to 128 gradations and 1/2 (up to 256 gradations). (Up to 192 levels with table values) Since up to 512 gradations were converted to one-fourth (256 values with table values), in inverse conversion, 8 bits read from the frame memory 13b for dark image correction data As shown in FIG. 9, the data is realized by multiplying up to 128 gradations by 1 times, up to 192 gradations by 2 times, and up to 256 gradations by 4 times. That is, a value that can be corrected for each gradation up to 128 gradations, further corrected every 2 gradations up to 256 gradations, and corrected every 4 gradations up to 512 gradations is generated. Yes.

  A pixel having a dark level higher than that is set as a target pixel of the pixel defect correction process described above as a defective pixel.

  FIG. 9 shows an example of table conversion values for inversely converting 8-bit data read from the dark image correction data frame memory 13b. As described above, since the pixel defect correction process is performed for a value exceeding 512 gradations, this conversion table is also a value for correcting up to 512 gradations (in FIG. 9, the vertical axis represents normalized table output gradation data, The horizontal axis is conversion table input data).

3) Exposure time correction processing
The dark level generally changes depending on the exposure time, exposure standby time, ambient temperature, and the like of the image sensor. Here, processing for correcting a dark level that varies with a change in exposure time is performed.

  Regarding the relationship between the exposure time and the dark level, as shown in FIG. 10, as the exposure time becomes longer, the dark level also increases due to the influence of the dark current of the image sensor. Therefore, by correcting the exposure time as a coefficient, it is possible to suppress the dark level fluctuation due to the change of the exposure time (in FIG. 10, the vertical axis is gradation and the horizontal axis is exposure time μsec).

  This dark level fluctuation varies as an offset depending on the exposure time. As a correction value, the exposure time correction is added to or subtracted from the data (10 bits) obtained from the dark image correction inverse conversion table 13c. Thus, dark image correction data is obtained.

4) Dark Image Correction Processing An image with a dark level corrected can be obtained by subtracting the imaging output signal output from the clamp circuit 12 and the dark image correction data described above.

  The image signal whose dark level has been corrected by the dark level correction circuit 13 in this way is input to the highlight level correction circuit 14 at the next stage.

[Highlight level correction processing and exposure time correction processing]
The highlight level correction process is divided into a prior correction value generation process and a normal imaging process.
[Correction value generation processing]
The correction value generation process is performed according to the following procedure.
1) Highlight image capture
The highlight image is captured in a state where dark level correction processing has been performed (described above) first, and then for each effective pixel of the image sensor 11, the output of each pixel of the photoelectric conversion unit 11a is uniform and constant. The image (highlight image) is taken in the light irradiation state. In this example, in photographing with a 10-bit signal, as shown in FIG. 11, photographing is performed with the irradiation light amount adjusted so that the highlight level becomes a level having an average value of 744 gradations.

2) Normalization table conversion In the highlight level correction processing by the normalization table 14a for the highlight image correction memory, pixels that exceed a certain gradation that is positive or negative with respect to the reference luminance gradation (here, 744 gradation). Is treated as a pixel defect, and a table for effectively correcting fluctuation gradations within that range is generated. Here, as shown in FIG. 11, pixels having a luminance variation of −30% (520 gradations) to + 30% (967 gradations) of the effective gradation variation with respect to the reference level of 744 gradations are sufficient. (In FIG. 11, the vertical and horizontal axes are gradations).

[Normal imaging processing]
The normal imaging process is performed according to the following procedure.
In order to correct the highlight portion, correction using a multiplier is effective.

  In highlight level correction by the normalization table 14a for highlight image correction memory, gain correction is performed so that the gradation to be corrected approaches the reference gradation. For example, as shown in FIG. 12, when the luminance level of the correction target pixel is 520 gradations when the reference level is 744 gradations, approximately 744 gradations are obtained by multiplying by about 1.43 by multiplication. Can do. Similarly, in the case of 967 gradations, approximately 0.77 gradation results in approximately 744 gradations. Therefore, the highlight level correcting multiplier needs to be able to perform an operation of 0.77 to 1.43 times at least (in FIG. 12, the vertical axis and the horizontal axis are gradations).

  Similarly to the dark level correction described above, table data having a correction coefficient of 8 bits is generated in order to eliminate waste of the correction frame memory.

  The minimum magnification required for this multiplier is 744 gradations / 967 gradations = 0.77 times, but in consideration of the margin, 744 gradations / 1023 gradations = 0.73 times can be calculated. It can be so. If the calculation coefficient is 8 bits, 0.73 × 256 = 186, which is the minimum gain coefficient. Similarly, for the maximum magnification, 1.44 × 256 = 369 is the maximum coefficient. However, since this numerical value cannot be handled with 8 bits, the offset is subtracted from these coefficients so that they can be accommodated with 8 bits coefficients (normalization table (ENC) and inverse conversion table (DEC) and offset shown in FIG. 13). See base value corresponding to minutes). In this case, if the minimum coefficient 186 is considered to be 0 in the coefficient table, the maximum coefficient 369-minimum coefficient 186 = 183, so that the number of 256 tables or less that can be expressed by a coefficient of 8-bit value is obtained. At this time, the relationship between the highlight reference level (reference (highlight)), the offset (base value), the normalization table (ENC), and the inverse conversion table (DEC) is corrected by 10-bit linear gradation correction. This is shown in FIG. 13 in comparison with the coefficient (10 bits). The normalization table (ENC) is realized by the highlight image correction memory normalization table 14a, and the reverse conversion table (DEC) is realized by the highlight image correction reverse conversion table 13c.

  As described above, with respect to an image signal for each pixel configured with the number of bits exceeding 1 byte, the dark level and highlight level gradation correction coefficients configured with the number of bits not exceeding 1 byte are retained, A simple hardware configuration is provided by providing a luminance correction circuit for the image sensor that corrects the gradation of the image signal output from the image sensor 11 using this gradation correction coefficient in the order of dark level and highlight level for each pixel. Can enable luminance correction of multi-value gradation by high-speed processing, thereby obtaining a high-definition high-quality captured image without luminance unevenness.

  DESCRIPTION OF SYMBOLS 11 ... Image pick-up element, 11a ... Photoelectric conversion part (image sensor part), 11b ... Analog signal pre-processing part, 11c ... A / D conversion part, 11d ... Horizontal / vertical drive part, 11e ... Sensor synchronous signal generation part, 12 ... Clamp circuit (V-CLAMP), 13 ... Dark level correction circuit, 13a ... Dark image correction memory normalization table, 13b ... Dark image correction data frame memory, 13c ... Dark image correction reverse conversion table, 13d ... Exposure time Correction processing unit, 13e ... dark image signal correction calculation unit, 13f ... nonvolatile memory for storing correction data, 14 ... highlight level correction circuit, 14a ... normalization table for highlight image correction memory, 14b ... highlight image correction data Frame memory, 14c... Inverse conversion table for highlight image correction, 14d... Exposure time correction processing unit, 14e. Highlight image signal correction calculation unit, 14f: correction data storage nonvolatile memory, 15: pixel defect correction circuit, 15a: defective pixel coordinate storage nonvolatile memory, 15b: defective pixel coordinate reproduction memory, 15c: pixel defect correction Processing unit, 16 ... post-processing circuit, 20 ... CPU, 21 ... camera synchronization signal generation circuit, 22 ... exposure time control circuit, 23, 24, 25 ... memory control circuit.

Claims (9)

A clamp circuit that holds a DC voltage level of a dark signal that is a reference of an image signal output from an image sensor that constitutes an area image sensor;
A gradation range on the low gradation side set in advance for an image signal for each pixel, which is provided at the subsequent stage of the clamp circuit and is configured by the number of gradation bits n exceeding 1 byte output from the image sensor. to the biased level distribution, it is composed of the smaller number of gradation bits than the gradation number of bits n of the image signal m (m <n), and the correction coefficient of low gradation side more than the correction coefficient of the high tone A dark level correction circuit that corrects the gradation of the image signal for each pixel using the distributed nonlinear gradation correction coefficient;
The gradation bit provided at the subsequent stage of the dark level correction circuit, wherein the level distribution is biased to a predetermined gradation range on the high gradation side with respect to the image signal for each pixel output from the image sensor A highlight level correction circuit for correcting the gradation of the image signal for each pixel using a gradation correction coefficient of several m ;
Provided in the subsequent stage of the highlight level correction circuit, for the pixel defect coordinate-designated by the defect image coordinate holding means for holding the defective pixel of the image sensor as a coordinate value, the luminance value is set to the luminance value of the surrounding pixels. and pixel defect correction processing unit for interpolating processing based,
A luminance correction circuit for an image sensor, comprising: The dark level correction circuit includes:
For all the effective pixels of the image sensor, a non-linear gradation correction coefficient that is configured with the number of gradation bits m for each pixel and in which the correction coefficient on the low gradation side is distributed more than the correction coefficient on the high gradation side. A frame memory for dark image correction data that stores and outputs gradation correction coefficients of all the effective pixels in synchronization with the photographing operation ;
And dark image correction conversion table for converting the gradation correction coefficient for the number of gradation bits m outputted from the dark image correction data frame memory to the number of gradation bits n,
A dark image signal correction calculation unit that performs gradation correction by subtraction processing using the gradation correction coefficient output from the dark image correction conversion table for the image signal output from the imaging element;
The brightness correction circuit for an image sensor according to claim 1 , comprising: The dark level correction circuit includes:
Generating said composed gradation bits m, and non-linear gradation correction coefficient correction coefficient low gradation side is allocated more than the correction coefficient of the high tone based on the image signal output from the imaging device A dark image correction memory normalization table ,
The gradation correction coefficient generated by the dark image correction memory normalization table is stored, and the gradation correction coefficient is output to the dark image correction data frame memory for initial setting processing at the time of shooting. Sex memory,
The luminance correction circuit for an image sensor according to claim 2 , further comprising: The dark level correction circuit includes:
An exposure time control circuit for outputting an exposure time correction coefficient according to a dark level variation with respect to an exposure time when the image sensor is shielded;
The brightness correction circuit for an image sensor according to claim 2 , wherein the dark image signal correction calculation unit performs gradation correction including an exposure time correction coefficient output from the exposure time control circuit. The highlight level correction circuit includes:
For all the effective pixels of the image sensor, the gradation correction coefficient of the gradation bit number m in which the level distribution is biased in the gradation range on the high gradation side set for each pixel is stored and synchronized with the photographing operation. A frame memory for highlight image correction data for outputting gradation correction coefficients of all effective pixels ;
A highlight image correction conversion table for converting the gradation correction coefficient of the gradation bit number m output from the highlight image correction data frame memory into the gradation bit number n ;
A highlight image signal correction calculator for tone correction by multiplication processing using the gradation compensation coefficient output an image signal output from the imaging element from the conversion table for the highlight image correction,
The brightness correction circuit for an image sensor according to claim 1 , comprising: The highlight level correction circuit includes:
Calculation magnification on the low gradation side and high gradation side of the gradation correction range centered on a preset reference gradation composed of the gradation bit number m based on the image signal output from the image sensor A normalization table for highlight image correction memory that generates gradation correction coefficients with different values ,
The highlight correction coefficient generated in the highlight image correction memory normalization table is stored, and the gradation correction coefficient is output to the highlight image correction data frame memory for initial setting processing at the time of shooting. Non-volatile memory for image correction;
The luminance correction circuit for an image sensor according to claim 5 , further comprising: The dark image correction nonvolatile memory gradation correction factors stored in the to claim 3, characterized in that it is generated based on the dark level of the histogram taken while shielding the imaging surface of the imaging element A brightness correction circuit of the image pickup device described. The gradation correction coefficient stored in the non-volatile memory for highlight image correction is based on a highlight level histogram that is obtained by entering predetermined non-saturated uniform light on the imaging surface of the image sensor. The brightness correction circuit for an image sensor according to claim 6 , wherein the brightness correction circuit is generated. The highlight level correction circuit includes:
An exposure time control circuit that outputs an exposure time correction coefficient according to a highlight level variation with respect to an exposure time when the image sensor is shielded;
6. The luminance correction circuit for an image sensor according to claim 5, wherein the highlight image signal correction calculation unit performs gradation correction including an exposure time correction coefficient output from the exposure time control circuit.

Images