JP2013062601A - Imaging apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus which divides an image signal from an imaging element into plural regions and sorts the regions to image processing LSIs and which, even if a joined optical black part (OB part) includes continuous defective pixels, is capable of satisfactorily correcting the defective pixels.SOLUTION: The imaging apparatus comprises: plural image processing LSIs for performing image processing on the image signal from the imaging element comprising an effective imaging part and the OB part; and an image signal division part which divides the image signal of the effective imaging part from the imaging element into plural regions, adds the image signal of the OB part corresponding to one of the divided regions of the image signal of the effective imaging part, and sorts the regions of the image signal to the image processing LSIs. Each image processing LSI comprises: a defective pixel detection part which detects a defective pixel in the OB part including continuous defective pixels; and a pixel correction part which corrects the detected defective pixel. Consequently, even if there are continuous defective pixels at the boundary of joined OB part, the defective pixels can satisfactorily be corrected.

Description

  The present invention relates to an imaging apparatus and an image processing method, and relates to a defect pixel correction technique in a system that performs image processing by dividing a video signal output from an imaging device into a plurality of regions.

  In recent years, the number of pixels that can be recorded in an imaging apparatus such as a digital video camera has increased. This is also closely related to the standard of the monitor that displays the video recorded by the imaging device. That is, in the monitor standard, there is a shift from a standard called SD (Standard Definition) to a standard called HD (High Definition), and a shift to a monitor having a further resolution is planned in the future. In connection with such changes in monitor standards, an increase in the number of pixels that can be recorded in an imaging apparatus is also required. In order to realize a high pixel count in an imaging apparatus, it is necessary to deal with a lens and an imaging device, an image processing LSI that digitally processes a video signal, a video output LSI that outputs the signal to the outside, and the like.

  However, as the number of pixels increases, there is a concern that the amount of information of the video signal output from the image sensor in one frame increases and the image processing LSI that digitally processes the video signal becomes large. As one solution to this problem, a technique is disclosed in which a plurality of image processing LSIs are provided and video signals are processed in parallel to increase the system speed and load distribution (see, for example, Patent Document 1).

JP 20036631 A

  When a plurality of image processing LSIs are provided and the video signal from the image sensor is divided into a plurality of areas and distributed to each image processing LSI for processing such as removal of fixed pattern noise from the image sensor Requires an optical black signal of the image sensor. For this reason, when the video signal is divided into a plurality of regions, the signal of the optical black portion is also divided so as to correspond to each region and processed by the image processing LSI.

  However, when the optical black part is divided corresponding to each area, when the optical black part corresponding to each area is joined, adjacent pixels at the joined boundary may be different from the actual image sensor. As a result, even if the required specification of the image sensor prohibits the continuous existence of defective pixels in the horizontal direction in the optical black portion, the defective pixels continuously exist in the optical black portion after joining. There is a problem that it sometimes happens. For this reason, parameters related to defective pixel correction must be relaxed compared to a system in which defective pixels are not continuous.However, if the parameters are relaxed, the removal rate of defective pixels decreases, and the accuracy of black level clamp processing and the like deteriorates. This significantly affects the image quality of the imaging device.

  An object of the present invention is to divide a video signal from an image sensor into a plurality of regions and distribute the image signals to each image processing LSI for processing, and even if defective pixels are continuously present in the joined optical black portion, An object of the present invention is to provide an imaging apparatus that can correct defective pixels satisfactorily.

  An image pickup apparatus according to the present invention includes an image pickup device having an effective image pickup portion including a plurality of pixels that perform photoelectric conversion according to an incident light amount, an optical black portion including a light-shielded pixel, and a video signal output from the image pickup device. A plurality of image processing means for performing image processing on the image sensor, and dividing the video signal of the effective imaging unit output from the imaging device into a plurality of regions, and corresponding to each region of the video signal of the optical black unit Video signal dividing means for adding the video signal of the corresponding optical black part to the divided video signal of the effective image pickup part and distributing the video signal to the plurality of image processing means for each region, Image processing means includes defective pixel detection means for detecting defective pixels in the optical black portion including continuous defective pixels; and the defective pixels And having a pixel correction means for correcting the defective pixel in the optical black portion detected by means out.

  According to the present invention, even when defective pixels are continuous at the boundary of the joined optical black portions, the defective pixels can be corrected satisfactorily, and image processing that requires the use of optical black portion signals can be performed with high accuracy. It can be carried out.

It is a figure which shows the structural example of the imaging device which concerns on embodiment of this invention. It is a figure which shows an example of distribution of a video signal. It is a figure which shows the structural example of a defective pixel correction | amendment part. It is a flowchart which shows the defective pixel correction process in 1st Embodiment. It is a flowchart which shows the defective pixel correction process in 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
A first embodiment of the present invention will be described.
FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to the first embodiment of the present invention. FIG. 1 shows a configuration from an image sensor to an image processing LSI in the image pickup apparatus according to the first embodiment. The image pickup apparatus shown in FIG. 1 includes an image pickup element 101, a line memory 102, a video signal dividing unit 103, and a plurality of image processing LSIs 104.

  The image sensor 101 is an image sensor corresponding to a high number of pixels, receives incident light from a lens, converts it into a video signal by photoelectric conversion, and outputs it. The image sensor 101 is, for example, a CCD (Charge Coupled Device) image sensor, a CMOS image sensor, or the like. The image pickup device 101 includes an image signal that directly outputs an analog image signal, an image signal that internally performs analog-to-digital conversion processing, and outputs digital data such as LVDS (Low voltage differential signaling). The imaging element 101 has a plurality of pixels including photoelectric conversion elements, and these are arrayed two-dimensionally (in the row direction and the column direction). In addition, the imaging element 101 includes an effective imaging unit (effective pixel unit) that includes a plurality of pixels that perform photoelectric conversion according to the amount of incident light, and an optical black unit that includes light-shielded pixels. Here, the image sensor 101 in the present embodiment prohibits the defective pixels from being continuously present before being divided by the video signal dividing unit 103.

  The line memory 102 is used when the video signal from the image sensor 101 is distributed to a plurality of image processing LSIs 104. The video signal dividing unit 103 receives the video signal output from the image sensor 101, divides it into a plurality of areas, and distributes it to the image processing LSI 104. The video signal dividing unit 103 divides the video signal obtained from the effective image pickup unit (effective pixel unit) of the image sensor 101 into a plurality of regions and obtains it from the optical black unit of the image sensor 101 so as to correspond to each region. The received video signal is divided. Then, the video signal dividing unit 103 adds an optical black signal corresponding to the region to the divided video signal, and outputs the signal to the image processing LSI 104.

  FIG. 2 is a diagram illustrating an example of distribution of video signals by the video signal dividing unit 103. In FIG. 2, an output for one frame from the image sensor 101 is schematically shown, and a case where a video signal is divided into four regions is shown as an example. In FIG. 2, a horizontal optical black (HOB) unit 201 and a vertical optical black (VOB) unit 202 correspond to light-shielded pixel areas and output a black level without reacting to light. The effective imaging unit 203 corresponds to a pixel region that receives incident light, photoelectrically converts actual light, and outputs a level corresponding to the incident light.

  The video signal dividing unit 103 divides the effective imaging unit 203 shown in FIG. 2 into four regions A to D, and adds an optical black unit corresponding to each of the regions A to D in the horizontal and vertical directions. That is, the video signal dividing unit 103 adds the regions E, F, and H of the optical black units 201 and 202 to the region A of the effective imaging unit 203. Similarly, the areas E, G, and H of the optical black sections 201 and 202 are added to the area B of the effective imaging section 203, and the optical black section 201 is added to the area C of the effective imaging section 203. , 202, area E, area F, and area I are added. In addition, the area E, the area G, and the area I of the optical black portions 201 and 202 are added to the area D of the effective imaging unit 203.

  In this manner, the video signal dividing unit 103 adds the signals of the corresponding optical black units 201 and 202 to the divided regions of the effective imaging unit 203 and distributes them to the image processing LSI 104 for each region. . Here, the video signal from the image sensor 101 shown in FIG. 2 is sequentially output from the upper left to the lower right in the figure. The line memory 102 stores a signal of the optical black portion (region E, region F, region G, region H, region I). Then, the video signal dividing unit 103 distributes the signals of the effective imaging units (area A, area B, area C, area D) in real time, and at the same time, reads out the signals of each area of the optical black section from the line memory 102. Is distributed.

  The image processing LSI 104 performs various types of image processing on the video signal in the area allocated by the video signal dividing unit 103. The image processing LSI 104 performs correction processing related to defective pixels of the image sensor 101, removal of fixed pattern noise of the image sensor 101, black level clamp processing, and the like. Further, the image processing LSI 104 performs various image processing in the imaging apparatus such as noise reduction and gamma correction. The image processing LSI 104 includes a processing unit 105 that performs various image processing such as noise reduction and gamma correction on a video signal, and a defective pixel correction unit that performs correction processing related to a defective pixel in the optical black portion of the image sensor 101. 106. In addition, the image processing LSI 104 includes a storage circuit that stores setting values necessary for each correction and image processing.

  In the imaging apparatus according to the present embodiment, the video signal output from the imaging device 101 corresponding to the high pixel is divided into a plurality of areas by the video signal dividing unit 103, and the plurality of image processing LSIs 104 are provided for the video signals in each area. Are processed in parallel. Thereby, the processing speed concerning the image processing of the video signal from the image sensor 101 can be improved.

  FIG. 3 is a block diagram illustrating a configuration example of the defective pixel correction unit 106. In FIG. 3, an area determination unit 301 determines whether or not an input video signal is a signal of a pixel in an optical black part. The defective pixel detection unit 302 detects defective pixels in the optical black portion. The defective pixel detection unit 302 determines whether or not the target pixel is a defective pixel based on the signal level of the target pixel and the signal levels of the left and right pixels adjacent to the target pixel. It is determined whether or not is a continuous defective pixel, and defective pixels are detected. The pixel correction unit 303 replaces and corrects the signal level of the defective pixel detected by the defective pixel detection unit 302.

Next, the defective pixel correction process in the first embodiment will be described. FIG. 4 is a flowchart illustrating a defective pixel correction process in the optical black portion of the image sensor 101 according to the first embodiment. The defective pixel correction process shown in FIG. 4 is executed by the defective pixel correction unit 106 of the image processing LSI 104.
First, in step S11, the image processing LSI 104 receives a video signal corresponding to each image processing LSI that is divided into a plurality of areas by the video signal dividing unit 103 and distributed to each area.

  Next, in step S12, the region determination unit 301 of the defective pixel correction unit 106 determines whether or not the pixel signal in the input video signal is a signal of the pixel in the optical black portion. This is because the area of the optical black portion is determined in advance from the specifications of the image sensor 101, so information indicating the area of the optical black portion is stored in advance in the image processing LSI, and the determination is performed with reference to the information. As a result of the determination in step S12, if it is determined that the signal is an optical black pixel, the process proceeds to step S13. If it is determined that the signal is not an optical black pixel, the process proceeds to step S18.

  In step S13, the defective pixel detection unit 302 of the defective pixel correction unit 106 detects the signal level of the target pixel and pixels adjacent to the target pixel in one direction (here, the left and right pixels adjacent to the target pixel). Compare with the average value of the signal level. As a result of the comparison, if the absolute value of the difference between the signal level of the target pixel and the average value of the signal levels of the adjacent left and right pixels is greater than or equal to the first threshold value stored in advance, the defective pixel detection unit 302 determines that the target pixel is a defective pixel, and proceeds to step S14. On the other hand, when the absolute value of the difference between the signal level of the target pixel and the average value of the signal levels of adjacent left and right pixels is smaller than the first threshold, the defective pixel detection unit 302 determines that the pixel is a defective pixel. It is determined that there is not, and the process proceeds to step S18.

  In step S14, the defective pixel detection unit 302 compares the signal level of the target pixel determined to be a defective pixel in step S13 with the signal level of each of the adjacent left and right pixels. When the absolute value of the difference between the signal level of the target pixel and the signal level of any one of the adjacent left and right pixels is equal to or smaller than a predetermined second threshold value, the process proceeds to step S15, and the defective pixel detection unit 302 It is determined that the target pixel is a continuous defective pixel with the compared pixel, and the process proceeds to step S16. In step S <b> 16, the pixel correction unit 303 of the defective pixel correction unit 106 performs correction processing on the pixels that are determined to be continuous defective pixels. Here, since the image sensor 101 has no defective pixels, it is obvious that the adjacent pixels of the continuous defective pixels are not defective pixels. Therefore, the pixel correction unit 303 replaces the signal level of the continuous defective pixel with the average value of the signal levels of the adjacent pixels of the continuous defective pixel, and proceeds to step S18.

  On the other hand, if the absolute value of the difference between the signal level of the target pixel and the signal level of each of the adjacent left and right pixels is greater than the second threshold as a result of the comparison in step S14, the defective pixel detection unit 302 , It is determined that the defective pixels are not continuous. In step S17, the pixel correction unit 303 performs correction by replacing the signal level of the target pixel with the average value of the signal levels of the adjacent left and right pixels, and the process proceeds to step S18.

  In step S18, the defective pixel correction unit 106 determines whether or not the target pixel is the last pixel in the readout area. That is, it is determined whether or not the pixel is the lower right pixel in each area in FIG. As a result, if the target pixel is not the last pixel in the readout area, the process returns to step S12 with the next pixel as the target pixel (step S19). On the other hand, if the target pixel is the last pixel in the readout area, the process proceeds to the top left top pixel in the next frame, and the process shown in FIG. 4 is newly repeated.

  According to the first embodiment, even in a video signal divided into a plurality of regions and distributed to each image processing LSI 104, even when continuous defective pixels that are prohibited in the image sensor 101 itself occur in the optical black portion, A defective pixel can be corrected satisfactorily. In this way, even if defective pixels continue at the boundary of the joined optical black portions, the defective pixels can be corrected well, and the black level clamping process of the image sensor, etc. without affecting the subsequent processing. The image processing can be performed with high accuracy.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.
Since the configuration of the imaging apparatus in the second embodiment is the same as that of the first embodiment described above, a description thereof will be omitted, and the defective pixel correction process in the second embodiment will be described below. FIG. 5 is a flowchart illustrating a defective pixel correction process in the optical black portion of the image sensor 101 according to the second embodiment. The defective pixel correction process shown in FIG. 5 is executed by the defective pixel correction unit 106 of the image processing LSI 104.

  In FIG. 5, the processing from step S21 to step S24 is the same as the processing from step S11 to step S14 shown in FIG.

  As a result of the comparison in step S24, if the absolute value of the difference between the signal level of the target pixel and the signal level of any of the adjacent left and right pixels is equal to or smaller than a predetermined second threshold value, the process proceeds to step S25. . In step S25, the defective pixel detection unit 302 determines that the target pixel is a continuous defective pixel with the compared pixel. In subsequent step S26, the pixel correction unit 303 stores information (hereinafter referred to as an address) indicating the positions of the two pixels determined to be continuous defective pixels in the storage circuit, and proceeds to step S28.

  If the absolute value of the difference between the signal level of the target pixel and the signal level of each of the adjacent left and right pixels is greater than the second threshold as a result of the comparison in step S24, the defective pixel detection unit 302 , It is determined that the defective pixels are not continuous. In step S27, the pixel correction unit 303 stores the address of the target pixel determined to be a defective pixel in the storage circuit, and proceeds to step S28.

  In step S28, the defective pixel correction unit 106 determines whether or not the target pixel is the last pixel in the readout area. As a result, if the target pixel is not the last pixel in the readout area, the process returns to step S22 with the next pixel as the target pixel (step S29). On the other hand, if the target pixel is the last pixel in the readout area, the process proceeds to step S30.

  Here, when proceeding to step S30, the above-described processing is performed up to the last pixel in the readout region, and addresses of all defective pixels are extracted. In step S30, the pixel correction unit 303 performs correction by replacing the signal levels of all pixels corresponding to the extracted defective pixel addresses with predetermined black levels determined by the image processing LSI 104, and performs processing. Exit.

  According to the second embodiment, in a video signal divided into a plurality of regions and distributed to each image processing LSI 104, defective pixels can be corrected well even when consecutive defective pixels occur in the optical black portion. . In this way, even if defective pixels continue at the boundary of the joined optical black portions, the defective pixels can be corrected well, and the black level clamping process of the image sensor, etc. without affecting the subsequent processing. The image processing can be performed with high accuracy. Further, in the second embodiment, since defective pixels are replaced with a uniformly defined black level, the black level error becomes larger than in the first embodiment, but if a defective pixel is extracted in one frame, it is different. Since it is not necessary to perform this operation again in the frame of, processing can be reduced.

(Other embodiments of the present invention)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

DESCRIPTION OF SYMBOLS 101 ... Image sensor, 102 ... Line memory, 103 ... Video signal division part, 104 ... Image processing LSI, 105 ... Processing part, 106 ... Defect pixel correction part, 301 ... Area determination part, 302 ... Defect pixel detection part, 303 ... Pixel correction unit

Claims (7)

An imaging device having an effective imaging unit including a plurality of pixels that perform photoelectric conversion according to the amount of incident light and an optical black unit that includes light-shielded pixels; and
A plurality of image processing means for performing image processing on a video signal output from the image sensor;
The video signal of the effective imaging unit output from the imaging device is divided into a plurality of regions, the video signal of the optical black unit is divided corresponding to each region, and the divided video signals of the effective imaging unit are divided. Video signal dividing means for adding a video signal of the corresponding optical black portion to each of the image black and distributing the image signal to the plurality of image processing means for each region;
The image processing means includes
A defective pixel detecting means for detecting defective pixels in the optical black portion including continuous defective pixels;
An image pickup apparatus comprising: a pixel correction unit that corrects a defective pixel in the optical black portion detected by the defective pixel detection unit.   The defective pixel detection means is configured to calculate an average of the signal level of the target pixel and the signal level of the adjacent pixel based on the signal level of the target pixel and the signal level of the pixel adjacent to the target pixel in one direction. If the absolute value of the difference from the value is greater than or equal to a first threshold, the target pixel is detected as a defective pixel, and the signal level of the target pixel detected as the defective pixel and any adjacent pixel The imaging apparatus according to claim 1, wherein the target pixel is detected as a continuous defective pixel when an absolute value of a difference from the signal level is equal to or smaller than a second threshold value.   The pixel correction means replaces the signal level of a defective pixel detected as a non-continuous defective pixel with an average value of the signal levels of pixels adjacent to the defective pixel, and detects the defective pixel detected as a continuous defective pixel. 3. The imaging apparatus according to claim 2, wherein the signal level is replaced with an average value of signal levels of pixels adjacent to the continuous defective pixels.   In one frame, defective pixels in the optical black portion including consecutive defective pixels are detected, information indicating the position of the detected defective pixels is stored, and stored in frames subsequent to the frame. The imaging apparatus according to claim 1, wherein the defective pixel is corrected using information indicating a position of the defective pixel.   5. The image pickup apparatus according to claim 4, wherein the pixel correction unit replaces the signal level of the pixel corresponding to the stored information indicating the position of the defective pixel with a predetermined black level.   6. The imaging according to claim 1, wherein the image processing performed by the image processing means includes at least one of removal of fixed pattern noise of the imaging device and black level clamping processing. apparatus. An image pickup device having an effective image pickup portion including a plurality of pixels that perform photoelectric conversion according to the amount of incident light and an optical black portion having light-shielded pixels, and a plurality of image processing performed on a video signal output from the image pickup device An image processing method in an imaging device comprising:
The video signal of the effective imaging unit output from the imaging device is divided into a plurality of regions, the video signal of the optical black unit is divided corresponding to each region, and the divided video signals of the effective imaging unit are divided. A video signal dividing step of adding a video signal of the corresponding optical black part to each of the plurality of image processing means for each region, and
An image processing step of performing image processing by the plurality of image processing means on the video signal distributed in the video signal dividing step;
The image processing step includes
A defective pixel detection step of detecting defective pixels in the optical black portion including continuous defective pixels;
A pixel correction step of correcting a defective pixel in the optical black portion detected in the defective pixel detection step.

Images