US6940549B2 - Image sensor signal defect correction - Google Patents
Image sensor signal defect correction Download PDFInfo
- Publication number
- US6940549B2 US6940549B2 US09/886,063 US88606301A US6940549B2 US 6940549 B2 US6940549 B2 US 6940549B2 US 88606301 A US88606301 A US 88606301A US 6940549 B2 US6940549 B2 US 6940549B2
- Authority
- US
- United States
- Prior art keywords
- defect
- difference signals
- intensity difference
- pixels
- intensity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related, expires
Links
- 230000007547 defect Effects 0.000 title claims abstract description 74
- 238000012937 correction Methods 0.000 title claims abstract description 34
- 238000000034 method Methods 0.000 claims abstract description 40
- 238000005070 sampling Methods 0.000 claims description 5
- 230000035945 sensitivity Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010348 incorporation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/68—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to defects
Definitions
- the invention relates to the correction of defects in image sensors, and is applicable to digital signal processors, digital still cameras and to digital video cameras. It has particular relevance to image sensors for consumer electronics and for medical systems, and is suitable for incorporation in a digital signal processor.
- Defects in images can take the form of single pixels, clusters or columns which have a small sensitivity difference compared to the rest of the image.
- a defect pixel may have a higher or a lower output signal than its neighboring pixels at the same illustration. This can be caused by defects in the sensor itself or by defects on the cover class of the image sensor that arise because the pixels around the edges of a sensor block will generally have lower sensitivity.
- this sensor has to be built up out of different blocks. The signals from these blocks are stitched together. At the borders of these blocks the sensors have a slightly different sensitivity than the neighboring pixels. This results in a so-called stitch line, which often appears as a visible line dividing the image. For example, in a known image sensor, a stitch line may appear as a double column with a gain difference above 1% and frequently up to 3%.
- EP 0 519 719 describes a method of reducing edge defects in images generated using arrays of pick-up elements.
- An image splitter is employed to separate the images corresponding to adjacent elements in the array so as to reduce the edge effects.
- JP-A 61-260773 discloses a defect pixel correction in dependence on a plurality of differences across a defect.
- EP-A 0 685 964 discloses a CCD-defect compensating method for a camcorder, including the steps of: sequentially delaying input pixel data; operating the values of pixels preceding and behind two stages from an nth pixel, and comparing the operated value with a reference level value to detect the presence or absence of a defect; averaging the pixel data two stages preceding and behind the nth pixel having the defect, to thereby obtain the nth pixel data; deciding the position of the defect to thereby generate a defect compensating control signal; and selectively compensating only for the pixel data averaged in the averaging step or the pixel data having the defect and delayed in the pixel delaying step, and outputting the compensated data with the original (externally input) pixel data.
- Defect correction can be accomplished by a correction that uses a (suitably weighted) average value of neighboring pixels that are not defect, the so-called average correction technique that is known to skilled persons in this technical field.
- the performance of the average correction technique is not good enough to completely eliminate the wide range of possible defects, including stitch lines and clusters, over a wide range of circumstances.
- the invention provides a defect correction as defined in the independent claims.
- Advantageous embodiments are defined in the dependent claims.
- the severity of the defect is large then average correction is used. If the severity is small, then the defect pixel value is not corrected. For in-between values, the level of good nearby pixels is assessed and the defect pixel value is not corrected for relatively small level differences, while the average correction is used for relatively large differences.
- FIG. 1 is a block flow diagram schematically illustrating one embodiment of a method according to the present invention
- FIG. 2 schematically illustrates one stage of the method of FIG. 1 ;
- FIG. 3 schematically illustrates one example applied to another stage of the method of FIG. 1 ;
- FIG. 4 schematically illustrates another example applied to the stage of FIG. 3 .
- Level signal 1 L 1 , level signal 2 L 2 and calibration information 3 CD 3 are determined in dependence upon the application concerned and the customer requirements.
- Digital Image data from a sensor 4 S 4 is supplied and pixel sampling is effected at sampler 5 .
- At least two and preferably three pixels are sampled on each side of a defect. Differences between the intensities of the preferably pixels are then calculated at difference calculator 6 .
- Four difference values D 1 , D 2 , D 3 , D 4 are calculated; two different values D 1 , D 2 generated by differences in intensity across the defect and a difference value D 3 , D 4 relating to each side of the defect.
- the difference signals D 1 , D 2 , and D 3 , D 4 are then compared at comparator 7 to the level signals L 1 and L 2 , respectively. If any one of the difference signals is higher than its respective level signal then the image scene is categorized as “high frequency” (AC), but if all difference signals are lower than the two level signals then the image scene is categorized “low frequency” (DC). This is termed the “DC Test”.
- each block following the image sensor S 4 may be seen as a method step carried out by an image processor in a software implementation.
- FIG. 2 schematically illustrates the calculated difference values. Pixels are labeled as a, b, c on one side of a defect cluster DEF and x, y, z on the opposite side.
- the cluster DEF comprises two pixels m, n.
- the following differences are calculated: D 1 : c ⁇ y D 2 : b ⁇ x D 3 : a ⁇ c D 4 : x ⁇ z
- D 1 and D 2 represent differences between pixels on opposite sides of the defect cluster DEF whereas D 3 and D 4 represent the differences between pixels which lie on respectively the same sides of the defect DEF.
- FIG. 3 shows one particular example applied to the method of the invention.
- the intensities of the pixels a, b, c and x, y, z are represented by the bar graph and the differences D 1 to D 4 are illustrated by the respective arrows which are reproduced in the lower graph which also shows the values of level 1 and level 2 for comparison.
- D 3 and D 4 are above level 2 and thus this image is treated for correction as a “high frequency” (AC) image.
- the size of the defect is then assessed. If the size of the defect is large then the average correction technique is applied. If the defect is small then no correction is done. If the defect is of medium size then the intensity level of good adjacent pixels is assessed and either the average correction is used or the defect pixel value is not corrected as explained above.
- This correction method gives a good correction capability across a wide range of defects and defect sizes and can substantially reduce or eliminate the small defect lines hitherto seen.
- Typical results of the method of the invention are achieved on images of a checkered shirt. This is a high frequency (AC) image and a stitch line tends to be very obvious in such scenes: a 3% small defect column would be typical. When the average correction technique is used, the small defect lines are still very evident. When the method of the invention is used and level correction (small) is applied with for superior results: the small defect line is barely visible.
- AC high frequency
- the method of the invention is applied to a scene with cluster defects.
- a raster of 24% clusters in the air and in the grill of a classic car is present in the original image. It appeared that the cluster defects in the image, particularly in the grill, are substantially reduced by means of the level correction method according to the invention, thus illustrating that the method of the invention can correct rather large clusters which previous technologies could not.
- the levels at which defects are categorized as large, medium or small is determined during calibration of the device in production.
- any reference signs placed between parentheses shall not be construed as limiting the claim.
- the word “comprising” does not exclude the presence of elements or steps other than those listed in a claim.
- the word “a” or “an” preceding an element does not exclude the presence of a plurality of such elements.
- the invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Image Input (AREA)
- Image Processing (AREA)
- Picture Signal Circuits (AREA)
Abstract
Description
D1: c−y
D2: b−x
D3: a−c
D4: x−z
Claims (13)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP00202200 | 2000-06-23 | ||
EP00202200.2 | 2000-06-23 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20020012476A1 US20020012476A1 (en) | 2002-01-31 |
US6940549B2 true US6940549B2 (en) | 2005-09-06 |
Family
ID=8171687
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/886,063 Expired - Fee Related US6940549B2 (en) | 2000-06-23 | 2001-06-21 | Image sensor signal defect correction |
Country Status (5)
Country | Link |
---|---|
US (1) | US6940549B2 (en) |
EP (1) | EP1297687A1 (en) |
JP (1) | JP2004501575A (en) |
KR (1) | KR20020033766A (en) |
WO (1) | WO2001099412A1 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020015111A1 (en) * | 2000-06-30 | 2002-02-07 | Yoshihito Harada | Image processing apparatus and its processing method |
US20070076296A1 (en) * | 1995-02-03 | 2007-04-05 | Dowski Edward R Jr | Wavefront coded imaging systems |
US20080075354A1 (en) * | 2006-09-25 | 2008-03-27 | Nokia Corporation | Removing singlet and couplet defects from images |
US20100141807A1 (en) * | 2003-01-16 | 2010-06-10 | Alex Alon | Camera with image enhancement functions |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7342681B2 (en) * | 2001-07-13 | 2008-03-11 | Transpacific Ip, Ltd | High-speed calibration method and system for an image-capture apparatus |
US7477781B1 (en) | 2002-10-10 | 2009-01-13 | Dalsa Corporation | Method and apparatus for adaptive pixel correction of multi-color matrix |
US20050280721A1 (en) * | 2004-06-18 | 2005-12-22 | Van Der Heide Auke | Method for providing combined gain compensation and error correction of a camera pickup array and an apparatus for implementing the method |
US7676084B2 (en) * | 2006-06-19 | 2010-03-09 | Mtekvision Co., Ltd. | Apparatus for processing dead pixel |
US7649555B2 (en) * | 2006-10-02 | 2010-01-19 | Mtekvision Co., Ltd. | Apparatus for processing dead pixel |
Citations (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS61260773A (en) | 1985-05-14 | 1986-11-18 | Mitsubishi Electric Corp | Picture defect compensation device |
US4701784A (en) * | 1984-06-01 | 1987-10-20 | Matsushita Electric Industrial Co., Ltd. | Pixel defect correction apparatus |
EP0519719A2 (en) | 1991-06-20 | 1992-12-23 | Canon Kabushiki Kaisha | Arrangement of a plurality of image sensors in a video camera |
EP0685964A1 (en) | 1994-06-02 | 1995-12-06 | Lg Electronics Inc. | CCD-defect compensating method and apparatus for camcorder |
EP0778543A2 (en) * | 1995-12-04 | 1997-06-11 | Eastman Kodak Company | A gradient based method for providing values for unknown pixels in a digital image |
JPH10150570A (en) * | 1996-09-20 | 1998-06-02 | Dainippon Screen Mfg Co Ltd | Method, device for specifying defective pixel in digital image and recording medium |
US5805216A (en) * | 1994-06-06 | 1998-09-08 | Matsushita Electric Industrial Co., Ltd. | Defective pixel correction circuit |
US6104839A (en) * | 1995-10-16 | 2000-08-15 | Eastman Kodak Company | Method and apparatus for correcting pixel values in a digital image |
JP2001028711A (en) * | 1999-07-14 | 2001-01-30 | Fuji Photo Film Co Ltd | Correction device for defect pixel data for solid-state image pickup element |
US20010036305A1 (en) * | 1999-12-30 | 2001-11-01 | Sung-Chun Jun | Detecting and compensating defective pixels in image sensor on real time basis |
JP2001307079A (en) * | 2000-04-26 | 2001-11-02 | Seiko Epson Corp | Image processor, image processing method and recording medium |
GB2364461A (en) * | 2000-05-24 | 2002-01-23 | Hewlett Packard Co | Correcting defective pixels in an image |
US20030063203A1 (en) * | 2001-10-03 | 2003-04-03 | Olympus Optical Co., Ltd. | Fault pixel correcting apparatus |
US6593961B1 (en) * | 1998-10-30 | 2003-07-15 | Agilent Technologies, Inc. | Test efficient method of classifying image quality of an optical sensor using three categories of pixels |
US6704458B2 (en) * | 1999-12-29 | 2004-03-09 | Eastman Kodak Company | Method and apparatus for correcting heavily damaged images |
US6707493B1 (en) * | 1998-06-30 | 2004-03-16 | Hyundai Electronics Industries Co., Ltd. | Method and apparatus for error detection and correction in image sensor |
US6806902B1 (en) * | 1999-06-08 | 2004-10-19 | Chrontel, Inc. | System and method for correcting bad pixel data in a digital camera |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS61260772A (en) * | 1985-05-14 | 1986-11-18 | Mitsubishi Electric Corp | Picture defect compensation device |
US4858013A (en) * | 1987-03-19 | 1989-08-15 | Mitsubishi Denki Kabushiki Kaisha | Solid state imaging device with adaptive pixel correction |
-
2001
- 2001-06-15 JP JP2002504132A patent/JP2004501575A/en not_active Withdrawn
- 2001-06-15 KR KR1020027002258A patent/KR20020033766A/en not_active Application Discontinuation
- 2001-06-15 EP EP01947376A patent/EP1297687A1/en not_active Withdrawn
- 2001-06-15 WO PCT/EP2001/006881 patent/WO2001099412A1/en not_active Application Discontinuation
- 2001-06-21 US US09/886,063 patent/US6940549B2/en not_active Expired - Fee Related
Patent Citations (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4701784A (en) * | 1984-06-01 | 1987-10-20 | Matsushita Electric Industrial Co., Ltd. | Pixel defect correction apparatus |
JPS61260773A (en) | 1985-05-14 | 1986-11-18 | Mitsubishi Electric Corp | Picture defect compensation device |
EP0519719A2 (en) | 1991-06-20 | 1992-12-23 | Canon Kabushiki Kaisha | Arrangement of a plurality of image sensors in a video camera |
EP0685964A1 (en) | 1994-06-02 | 1995-12-06 | Lg Electronics Inc. | CCD-defect compensating method and apparatus for camcorder |
US5696554A (en) * | 1994-06-02 | 1997-12-09 | Lg Electronics, Inc. | CCD-defect compensating method and apparatus for camcorder |
US5805216A (en) * | 1994-06-06 | 1998-09-08 | Matsushita Electric Industrial Co., Ltd. | Defective pixel correction circuit |
US6104839A (en) * | 1995-10-16 | 2000-08-15 | Eastman Kodak Company | Method and apparatus for correcting pixel values in a digital image |
EP0778543A2 (en) * | 1995-12-04 | 1997-06-11 | Eastman Kodak Company | A gradient based method for providing values for unknown pixels in a digital image |
JPH10150570A (en) * | 1996-09-20 | 1998-06-02 | Dainippon Screen Mfg Co Ltd | Method, device for specifying defective pixel in digital image and recording medium |
US6707493B1 (en) * | 1998-06-30 | 2004-03-16 | Hyundai Electronics Industries Co., Ltd. | Method and apparatus for error detection and correction in image sensor |
US6593961B1 (en) * | 1998-10-30 | 2003-07-15 | Agilent Technologies, Inc. | Test efficient method of classifying image quality of an optical sensor using three categories of pixels |
US6806902B1 (en) * | 1999-06-08 | 2004-10-19 | Chrontel, Inc. | System and method for correcting bad pixel data in a digital camera |
JP2001028711A (en) * | 1999-07-14 | 2001-01-30 | Fuji Photo Film Co Ltd | Correction device for defect pixel data for solid-state image pickup element |
US6704458B2 (en) * | 1999-12-29 | 2004-03-09 | Eastman Kodak Company | Method and apparatus for correcting heavily damaged images |
US20010036305A1 (en) * | 1999-12-30 | 2001-11-01 | Sung-Chun Jun | Detecting and compensating defective pixels in image sensor on real time basis |
JP2001307079A (en) * | 2000-04-26 | 2001-11-02 | Seiko Epson Corp | Image processor, image processing method and recording medium |
GB2364461A (en) * | 2000-05-24 | 2002-01-23 | Hewlett Packard Co | Correcting defective pixels in an image |
US6724945B1 (en) * | 2000-05-24 | 2004-04-20 | Hewlett-Packard Development Company, L.P. | Correcting defect pixels in a digital image |
US20030063203A1 (en) * | 2001-10-03 | 2003-04-03 | Olympus Optical Co., Ltd. | Fault pixel correcting apparatus |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070076296A1 (en) * | 1995-02-03 | 2007-04-05 | Dowski Edward R Jr | Wavefront coded imaging systems |
US20020015111A1 (en) * | 2000-06-30 | 2002-02-07 | Yoshihito Harada | Image processing apparatus and its processing method |
US7133072B2 (en) * | 2000-06-30 | 2006-11-07 | Canon Kabushiki Kaisha | Image processing apparatus having an image correction circuit and its processing method |
US20100141807A1 (en) * | 2003-01-16 | 2010-06-10 | Alex Alon | Camera with image enhancement functions |
US8126287B2 (en) * | 2003-01-16 | 2012-02-28 | DigitalOptics Corporation International | Camera with image enhancement functions |
US20080075354A1 (en) * | 2006-09-25 | 2008-03-27 | Nokia Corporation | Removing singlet and couplet defects from images |
US7796806B2 (en) * | 2006-09-25 | 2010-09-14 | Nokia Corporation | Removing singlet and couplet defects from images |
Also Published As
Publication number | Publication date |
---|---|
EP1297687A1 (en) | 2003-04-02 |
US20020012476A1 (en) | 2002-01-31 |
KR20020033766A (en) | 2002-05-07 |
JP2004501575A (en) | 2004-01-15 |
WO2001099412A1 (en) | 2001-12-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7199824B2 (en) | Intelligent blemish control algorithm and apparatus | |
US6965395B1 (en) | Methods and systems for detecting defective imaging pixels and pixel values | |
US6625325B2 (en) | Noise cleaning and interpolating sparsely populated color digital image using a variable noise cleaning kernel | |
JP4247514B2 (en) | Method and system for correcting defective pixels | |
US7659930B2 (en) | Edge enhancement method and apparatus for Bayer images, and color image acquisition system | |
US7286179B2 (en) | Defective pixel filtering for digital imagers | |
US20030179418A1 (en) | Producing a defective pixel map from defective cluster pixels in an area array image sensor | |
US20050249417A1 (en) | Edge detecting method | |
JP2004207896A (en) | Pixel defect detecting/correcting apparatus and method therefor | |
US6940549B2 (en) | Image sensor signal defect correction | |
JPH06225267A (en) | Method and apparatus for motion compensated image | |
US7460180B2 (en) | Method for false color suppression | |
US20030007099A1 (en) | Motion adaptive noise reduction method and system | |
US9826174B2 (en) | Image processing apparatus and method | |
US6987577B2 (en) | Providing a partial column defect map for a full frame image sensor | |
JP3227815B2 (en) | Solid-state imaging device | |
JP4288954B2 (en) | Defect detection circuit and defect detection method | |
US6674906B1 (en) | Method and apparatus for detecting edges in a mixed image | |
JP2006166194A (en) | Pixel defect detection circuit and pixel defect detection method | |
JP2000101924A (en) | Defect detection correction device in image input device | |
JPH11262025A (en) | Image input device and image correction method therefor | |
JP2004173060A (en) | Noise elimination method, image pickup device, and noise elimination program | |
JP2004350104A (en) | Signal processing apparatus and electronic camera | |
US8358850B2 (en) | Determination method of edge direction | |
JP3081046B2 (en) | Image noise determination method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: KONINKLIJKE PHILIPS ELECTRONICS N.V., NETHERLANDS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:DILLEN, BARTHOLOMEUS GOVERDINA MARIA HENRICUS;VAN DER SIJDE, ARJEN GERBEN;LANGEN, RALPH;REEL/FRAME:012198/0122;SIGNING DATES FROM 20010806 TO 20010808 |
|
AS | Assignment |
Owner name: IPG ELECTRONICS 503 LIMITED Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KONINKLIJKE PHILIPS ELECTRONICS N.V.;REEL/FRAME:022203/0791 Effective date: 20090130 Owner name: IPG ELECTRONICS 503 LIMITED, GUERNSEY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KONINKLIJKE PHILIPS ELECTRONICS N.V.;REEL/FRAME:022203/0791 Effective date: 20090130 |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
AS | Assignment |
Owner name: PENDRAGON WIRELESS LLC, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:IPG ELECTRONICS 503 LIMITED;REEL/FRAME:028594/0224 Effective date: 20120410 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Free format text: PAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
REMI | Maintenance fee reminder mailed | ||
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.) |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20170906 |