WO2012063265A4 - Method and apparatus for detecting the bad pixels in sensor array and concealing the error - Google Patents

Method and apparatus for detecting the bad pixels in sensor array and concealing the error Download PDF

Info

Publication number
WO2012063265A4
WO2012063265A4 PCT/IN2011/000775 IN2011000775W WO2012063265A4 WO 2012063265 A4 WO2012063265 A4 WO 2012063265A4 IN 2011000775 W IN2011000775 W IN 2011000775W WO 2012063265 A4 WO2012063265 A4 WO 2012063265A4
Authority
WO
WIPO (PCT)
Prior art keywords
pixels
pixel
bad
noise map
level
Prior art date
Application number
PCT/IN2011/000775
Other languages
French (fr)
Other versions
WO2012063265A3 (en
WO2012063265A9 (en
WO2012063265A2 (en
Inventor
Sudipta Mukhopadhyay
Abhishek Kumar Tripathi
Original Assignee
Indian Institute Of Technology, Kharagpur
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Indian Institute Of Technology, Kharagpur filed Critical Indian Institute Of Technology, Kharagpur
Priority to KR1020137014825A priority Critical patent/KR101559724B1/en
Publication of WO2012063265A2 publication Critical patent/WO2012063265A2/en
Publication of WO2012063265A3 publication Critical patent/WO2012063265A3/en
Publication of WO2012063265A9 publication Critical patent/WO2012063265A9/en
Publication of WO2012063265A4 publication Critical patent/WO2012063265A4/en
Priority to IL226318A priority patent/IL226318A0/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/67—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/67—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response
    • H04N25/671—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/68—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to defects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/68—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to defects
    • H04N25/683—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to defects by defect estimation performed on the scene signal, e.g. real time or on the fly detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Image Processing (AREA)

Abstract

A method of detection of bad pixel in sensor array and correction of detected bad pixels by inpainting and also to image sensor devices involving such sensor arrays adapted for identification and correction of the bad pixels. Importantly, image sensors which are provided to convert optical images to electrical signals would benefit from such possible detection and correction of pixel errors thereby providing for better quality of output. The method and the device enables providing for better quality cameras in particular digital cameras involving CCD image sensors or CMOS sensors. It monitors the health of the sensor arrays and increase its life, without involving any additional hardware cost. The invention targets all possible varieties of defective pixels including dead pixels, stuck pixels, hot and cold pixels and therefore is adapted to serve as a complete solution to such art of handling image corruption even for high numbers of bad pixels.

Claims

AMENDED CLAIMS
received by the International Bureau on 12 November 2012 (12.11.12)
We claim:
1. A method for detecting the bad pixels in sensor array comprising generating error/noise map involving intensity based detection of location information of bad pixels of sensors using local statistics of the digital image involving values signifying the location of different variety of bad pixel including anyone of more of dead, stuck, hot and cold pixels and normal pixel respectively.
2. A method for detecting the bad pixels in sensor array as claimed in claim 1 comprising
generating said error/noise map comprising providing a noise map involving intensity based detection of location information of bad pixels of sensors using local statistics of the digital image and subjecting the location of the thus identified bad pixels to further evaluation of its local statistics to confirm that the pixels are bad or not and thereby generate a final noise map with 1 and 0 values which signify the location of bad pixel and normal pixel respectively.
3. A method for detecting the bad pixels in sensor array as claimed in anyone of claims 1 to 2 comprising assessing number of bad pixels and its subgroups to determine health of a sensor array and/or generating error map over a several subsequent images and involving combinations thereof to further confirm error detection.
4. A method for detecting the bad pixels in sensor array as claimed in anyone of claims 1 or 3 comprising step of authentication of the images captured by a camera by comparing with the error/noise map thus generated with images taken from a camera.
5. A method for detecting the bad pixels in sensor array and concealing of the error comprising
(i) generating noise map involving intensity based detection of location information of bad pixels of sensors using local statistics of the digital image involving values signifying the location of bad pixel and normal pixel respectively;
(ii) carrying out an adaptive inpainting procedure for correcting only the detected bad pixels in said noise map in relation to a noisy image comprising the step of replacing the bad pixels by the median value (intensity) of normal pixels within a specified window around said detected bad pixel.
6. A method for detecting the bad pixels in sensor array and concealing of the error comprising
(i) generating said noise map comprises providing a noise map involving intensity based detection of location information of bad pixels of sensors using local statistics of the digital image and subjecting the location of the thus identified bad pixels to further evaluation of its local statistics to confirm that the pixels are bad or not and thereby generate a final noise map with 1 and 0 values which signify the location of bad pixel and normal pixel respectively;
(ii) an adaptive inpainting procedure for correcting only the detected bad pixels in said final noise map in relation to a noisy image comprising replacing the bad pixels by the median value(intensity) of normal pixels within a specified window around said detected bad pixel.
7. A method for detecting the bad pixels in sensor array and concealing of the error as claimed in anyone of claims 5 or 6 wherein said bad pixels include dead, stuck, hot and cold pixels.
8. A method as claimed in claim 7 wherein carrying out the step of detection of said dead pixel comprises (I) imposing a Wi X W2 window for each noisy image and centering the same on the current pixel to determine the maximum value (sme¾) and minimum value (sm/n) within the window, thereafter generating the first stage noise map V following :
Figure imgf000004_0001
wherein sy is the intensity value of the pixel at location (i,j) and Smll, is the global minimum intensity value of the image and wherein in this first stage noise map, r(i,j) = 1 means corresponding pixel in noisy image is probably dead and r(i,j) = 0 means corresponding pixel in noisy image is normal; followed by
27
(II) generating a final noise map Y comprising
Level 1) imposing a x w2 window around the dead pixel in noisy image, such that w2 are much smaller than previous window size; if more than wmIn normal pixels such as when pixels which have corresponding values in first stage noise map 0 are found, thereafter calculating the distance measure d between the normal pixels and the central dead pixel and going to Level 3; if the number of normal pixels is not more than wmln then going to Level 2; where it is assumed for simulation that d is mean of absolute distance (MAD) evaluated by following equation;
Figure imgf000005_0001
where e,- is the distance of i-th pixel from the central pixel.
Level 2) setting the w, = w, + pl( i=l,2, pi are small positive integers and going to Level 1 and increasing the window to a pre determined fixed size;
Level 3) if the corresponding d is less than the threshold δ (δ is a small positive value), replacing the corresponding pixel in temporary noise map r to 0 else leave it and move to next dead pixel and go to Level 1.
9. A method as claimed in claim 7 , wherein carrying out the step of detection of stuck pixels comprises (I) imposing a W( X W2 window for each noisy image and centering the same on the current pixel to determine the maximum value (smaj() and minimum value (sm,„) within the window, thereafter generating the first stage noise map Y following :
Figure imgf000005_0002
wherein is the intensity value of the pixel at location (i,j) and Smalt is the global maximum intensity value of the image and in this first stage noise map, r(i,j) = 1 means corresponding pixel in noisy image is probably stuck and r(i,j) = 0 means corresponding pixel in noisy image is normal followed by
28
(II) generating a final noise map Y comprising
Level 1 : Imposing a x w2 window around the stuck pixel in noisy image, such that w2 are much smaller than previous window size and if more than wmln normal pixels are found, then calculating distance measure d between the normal pixels and the central stuck pixel and go to Level 3 and if the number of normal pixels are not more than wmln then go to Level 2. Here it is assumed that d is MAD.
Level 2 : Setting the = wi + p,, i=l,2, Pi are small positive integers and going to Level 1, Increasing the window up to a pre determined fixed size.
Level 3 : If the corresponding d is less than the threshold δ (δ is a small positive), replacing the corresponding pixel in temporary noise map ' r' to 0 else leave it and move to next stuck pixel and going to Level 1, whereby the noise map V is final noise map.
A method as claimed in claim 7 wherein carrying out the step of detection of said hot and cold pixels comprises (I) imposing a W, X W2 window for each noisy image and centering the same on the current pixel, thereafter generating the first stage noise map Y following :
Figure imgf000006_0001
where Sy is the intensity value of the pixel at location (i,j). Smax and Smi„ are the global maximum and minimum intensity value respectively. In first stage noise map, r(i,j) = 1 means corresponding pixel in noisy image is probably hot or cold and r(i,j) = 0 means corresponding pixel in noisy image is normal.
(II) generating a final noise map Y comprising
29
Level 1 : Imposing a wt x w2 window around the hot (cold) pixel in noisy image, such that w1( w2 are much smaller than previous window size and if more than wmln normal pixels are found, then calculating distance measure d between the normal pixels and the central hot (cold) pixel and going to Level 3. If the number of normal pixels are not more than wmln then going to Z.eve/ 2 wherein it is assumed that the distance measure d is MAD.
Level 2 : Setting the w, = w, + pi( i=l,2, pi are small positive integers and going to Level 1. Increasing the window up to a predetermined fixed size.
Level 3 : taking minimum of distance of the central hot (cold) pixel from both end preferably such that if image is 8 bit then T3 = min , 255 - and If the corresponding d is less than the threshold qT3l replacing the corresponding pixel in temporary noise map to 0 else leaving it and moving to next hot(cold) pixel and going to Level iwith q being a constant (0 < q < 1) to thereby generate the final noise map V.
11. A method as claimed in anyone of claims 5 to 10 wherein said method is adapted to be applied for images of any bit depth.
12. A method as claimed in anyone of claims 5 to 11, wherein the said step of correcting the pixel error by inpainting comprising the steps of :
(1) imposing a w3 x w4 window around the bad pixel in noisy image to find out a set S having all those noisy image pixels under current window for which corresponding value in noisy map are 0; if S is not a null set then going to Step 2, otherwise going to Step 3,
(2) replacing the current pixel with median of the set S and proceeding to the next bad pixel and going to Step 1,
30
(3) setting the w, = + Aw,, i=3,4 Aw, are small positive integers and going to Stepl and increasing the window up to a pre determined maximum size.
13. A method as claimed in anyone of claims 5 to 12, wherein said bad pixel detection procedure followed by bad pixel inpainting is adapted to achieve high PSNR even with image corruption for high number of bad pixels.
14. A method as claimed in anyone of claims 5 to 13, wherein in said bad pixel detection is performed online or offline and once detected the inpainting may be applied online/offline depending on application demand.
15. An image sensor system comprising a sensor array adapted to detect bad pixels and carry out inpainting of bad pixels comprising :
(i) means adapted for generating noise map involving intensity based detection of location information of bad pixels of sensors using local statistics of the digital image involving values signifying the location of bad pixel and normal pixel respectively;
(ii) means adapted for carrying out an adaptive inpainting procedure for correcting only the detected bad pixels in said noise map in relation to a noisy image comprising the step of replacing the bad pixels by the median value (intensity) of normal pixels within a specified window around said detected bad pixel.
16. An image sensor system as claimed in claim 15 wherein said sensor array is adapted to detect variety of bad pixels selected from dead pixel, stuck pixel, cold and hot pixels following any of the methods as claimed in claims 5 to 15.
17. An image sensor system as claimed in claim 15 or 16, which is adapted to convert optical/thermal any other signal including thermal , micro-wave, ultrasound, x-ray to electrical signals.
31

STATEMENT UNDER ARTICLE 19 OF PCT

Amended claims are directed to further clarify and qualify such inventive aspects of the present invention residing in the method and apparatus for detecting the bad pixels in sensor array and concealing the error. The detection procedure is based on the detection of location information of bad pixels of sensors using local statistics involving values signifying the location signifying the location of different variety of bad pixel including anyone or more of dead, stuck, hot and cold pixels and normal pixels respectively and subjecting the location of the thus identified bad pixels to further evaluation of its local statistics to confirm that the pixels are bad or not and thereby executing an adaptive inpainting procedure for correcting the detected bad pixels in the said detection process by replacing the bad pixels by the median value (intensity) of normal pixels within a specified window around said detected bad pixel.

The amendments in the claims are directed to qualify the claimed invention and do not extend beyond the disclosure and directions in the international application as filed.

PCT/IN2011/000775 2010-11-12 2011-11-11 Method and apparatus for detecting the bad pixels in sensor array and concealing the error WO2012063265A2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020137014825A KR101559724B1 (en) 2010-11-12 2011-11-11 Method and Apparatus for Detecting the Bad Pixels in Sensor Array and Concealing the Error
IL226318A IL226318A0 (en) 2010-11-12 2013-05-12 Method and apparatus for detecting the bad pixels in sensor array and concealing the error

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IN1279KO2010 2010-11-12
IN1279/KOL/2010 2010-11-12

Publications (4)

Publication Number Publication Date
WO2012063265A2 WO2012063265A2 (en) 2012-05-18
WO2012063265A3 WO2012063265A3 (en) 2012-07-26
WO2012063265A9 WO2012063265A9 (en) 2012-10-04
WO2012063265A4 true WO2012063265A4 (en) 2012-12-27

Family

ID=45974475

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IN2011/000775 WO2012063265A2 (en) 2010-11-12 2011-11-11 Method and apparatus for detecting the bad pixels in sensor array and concealing the error

Country Status (3)

Country Link
KR (1) KR101559724B1 (en)
IL (1) IL226318A0 (en)
WO (1) WO2012063265A2 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2750391B1 (en) 2012-12-31 2017-03-22 Nokia Technologies Oy Method, apparatus and computer program product for processing of images
DE102013209165A1 (en) * 2013-05-17 2014-11-20 Arnold & Richter Cine Technik Gmbh & Co. Betriebs Kg PIXEL MAPPING PROCEDURE
US20150172576A1 (en) * 2013-12-05 2015-06-18 Aselsan Elektronik Sanayi Ve Ticaret Anonim Sirketi Real time dynamic bad pixel restoration method
KR102206996B1 (en) * 2019-01-29 2021-01-25 주식회사 디알텍 Method for processing radiation image and radiographic apparatus

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5047863A (en) 1990-05-24 1991-09-10 Polaroid Corporation Defect correction apparatus for solid state imaging devices including inoperative pixel detection
US5499114A (en) 1994-10-31 1996-03-12 Eastman Kodak Company Digital image scanning apparatus with pixel data compensation for bad photosites
US6618084B1 (en) * 1997-11-05 2003-09-09 Stmicroelectronics, Inc. Pixel correction system and method for CMOS imagers
US6665009B1 (en) 1998-05-20 2003-12-16 Omnivision Technologies, Inc. On-chip dead pixel correction in a CMOS imaging sensor
US7068854B1 (en) * 1999-12-29 2006-06-27 Ge Medical Systems Global Technology Company, Llc Correction of defective pixels in a detector
US7015961B2 (en) * 2002-08-16 2006-03-21 Ramakrishna Kakarala Digital image system and method for combining demosaicing and bad pixel correction
US8009209B2 (en) * 2005-09-30 2011-08-30 Simon Fraser University Methods and apparatus for detecting defects in imaging arrays by image analysis
JP4604078B2 (en) * 2007-11-22 2010-12-22 アキュートロジック株式会社 Defective pixel correction method, defective pixel correction program, and defective pixel correction device

Also Published As

Publication number Publication date
WO2012063265A3 (en) 2012-07-26
KR101559724B1 (en) 2015-10-13
IL226318A0 (en) 2013-07-31
WO2012063265A9 (en) 2012-10-04
WO2012063265A2 (en) 2012-05-18
KR20130096293A (en) 2013-08-29

Similar Documents

Publication Publication Date Title
US8000559B2 (en) Method of correcting image distortion and apparatus for processing image using the method
US10348989B2 (en) Image processing device, image processing method, and image processing system
US7622715B2 (en) Method and apparatus for capturing and analyzing thermo-graphic images of a moving object
Goljan et al. Sensor-fingerprint based identification of images corrected for lens distortion
US8355064B2 (en) Noise reduction for machine vision systems
WO2012063265A4 (en) Method and apparatus for detecting the bad pixels in sensor array and concealing the error
US9092845B2 (en) Image processing device with defect correcting circuit and image processing method
WO2019142540A1 (en) Solid-state imaging device and information processing method of solid-state imaging device
JP2009296174A5 (en)
US8810696B2 (en) Blemish detection method
JP6091285B2 (en) Image processing apparatus, inspection system using the same, sheet image correction method, and sheet inspection method using the same
JP2011065097A5 (en) Image processing apparatus, imaging apparatus, and control method
US20140071298A1 (en) Blemish detection sytem and method
US9681083B2 (en) Method and system to detect a light-emitting diode
JP2012070319A (en) Image processing method, image processing apparatus, and image processing program
JP2010093471A (en) Imaging device, semiconductor integrated circuit for the same, and method of determining defective pixel
JP2009290653A (en) Defective image correcting apparatus, imaging apparatus, defective pixel correcting method, and program
TWI528817B (en) Image capture devices and image processing method thereof
US20060279646A1 (en) Pixel defect detection method for solid-state image pickup device
JP2008283620A (en) Solid-state imaging apparatus
JP2010011161A (en) Image capturing apparatus and image sensor temperature measuring method
Yang et al. Identification and replacement of defective pixel based on Matlab for IR sensor
JP2006210417A (en) Method of checking solid-state imaging device
CN103686098B (en) Digital camera shooting device and image processing method thereof
JP6036998B2 (en) Imaging apparatus, image correction method, and image correction program

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 226318

Country of ref document: IL

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 20137014825

Country of ref document: KR

Kind code of ref document: A

122 Ep: pct application non-entry in european phase

Ref document number: 11833611

Country of ref document: EP

Kind code of ref document: A2