JPWO2018037674A1 - Image pickup apparatus, abnormal pixel detection interpolation method, and image pickup apparatus adjustment method - Google Patents
Image pickup apparatus, abnormal pixel detection interpolation method, and image pickup apparatus adjustment method Download PDFInfo
- Publication number
- JPWO2018037674A1 JPWO2018037674A1 JP2018535474A JP2018535474A JPWO2018037674A1 JP WO2018037674 A1 JPWO2018037674 A1 JP WO2018037674A1 JP 2018535474 A JP2018535474 A JP 2018535474A JP 2018535474 A JP2018535474 A JP 2018535474A JP WO2018037674 A1 JPWO2018037674 A1 JP WO2018037674A1
- Authority
- JP
- Japan
- Prior art keywords
- pixel
- dark current
- interpolation
- white
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims description 16
- 238000001514 detection method Methods 0.000 title description 87
- 230000002159 abnormal effect Effects 0.000 title description 2
- 238000003384 imaging method Methods 0.000 claims abstract description 169
- 238000012937 correction Methods 0.000 claims abstract description 95
- 238000009825 accumulation Methods 0.000 claims abstract description 47
- 230000007547 defect Effects 0.000 claims description 140
- 238000012545 processing Methods 0.000 claims description 40
- 230000003287 optical effect Effects 0.000 claims description 7
- 238000000926 separation method Methods 0.000 claims description 5
- 230000000903 blocking effect Effects 0.000 claims description 4
- 238000003705 background correction Methods 0.000 claims description 3
- 230000035945 sensitivity Effects 0.000 abstract description 26
- 206010027146 Melanoderma Diseases 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 37
- 230000002093 peripheral effect Effects 0.000 description 11
- 230000008859 change Effects 0.000 description 10
- 241000669298 Pseudaulacaspis pentagona Species 0.000 description 9
- 230000008569 process Effects 0.000 description 9
- 238000006243 chemical reaction Methods 0.000 description 8
- 230000015654 memory Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000009467 reduction Effects 0.000 description 5
- 230000032683 aging Effects 0.000 description 4
- 241000316887 Saissetia oleae Species 0.000 description 3
- 230000017525 heat dissipation Effects 0.000 description 3
- 230000007774 longterm Effects 0.000 description 3
- 241000519995 Stachys sylvatica Species 0.000 description 2
- 230000001934 delay Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000010422 painting Methods 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 239000004753 textile Substances 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 210000003127 knee Anatomy 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/68—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to defects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/63—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to dark current
- H04N25/633—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to dark current by using optical black pixels
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Studio Devices (AREA)
- Solid State Image Pick-Up Elements (AREA)
Abstract
感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する)の暗電流と正常画素の暗電流とを確実に識別して検出し、完全黒キズを補間する。撮像装置において、撮像素子温度に6℃でおよそ2倍となる正常な遮光画素の暗電流を検出する手段を有し、該検出した正常な遮光画素の暗電流に蓄積時間を逆比例させておよそ1秒から16384秒遮光蓄積して、遮光(OB)画素信号補正前の撮像信号で、感度が異常に低い撮像画素(以下、完全黒キズと称する)の暗電流とを正常画素の暗電流とを識別して検出し、通常撮像時は、完全黒キズは周囲画素の中央値で補間する。The dark current of an imaging pixel (hereinafter referred to as a perfect black flaw) having an abnormally low sensitivity and a dark current and the dark current of a normal pixel are reliably identified and detected to interpolate the perfect black flaw. The image pickup apparatus has means for detecting dark current of a normal light-shielded pixel which approximately doubles at 6 ° C. to the image pickup element temperature, and makes the accumulation time inversely proportional to the detected dark current of the normal light-shielded pixel. The dark current of an imaging pixel (hereinafter referred to as a perfect black spot) with an abnormally low sensitivity in the imaging signal before light-blocked (OB) pixel signal correction, with 1 sec to 16384 sec dark light accumulation and dark current of a normal pixel. Is detected, and during normal imaging, perfect black spots are interpolated with the median of surrounding pixels.
Description
本発明は、テレビジョンカメラなどの撮像装置に係わり、特に撮像素子の異常画素検出と補間に関するものである。 The present invention relates to an imaging device such as a television camera, and more particularly to abnormal pixel detection and interpolation of an imaging device.
従来のテレビジョンカメラの固撮像素子の画素欠陥検出方法としては、撮像光を分光して得られた複数分光を分光光毎に同一撮像位置もしくは近傍撮像位置で撮像し出力した映像信号どうしを比較した結果に応じて、前記分光光のうちいずれの分光光を撮像した撮像素子で画素欠陥が発生しているか否かを検出し、補間するものがある(特許文献1参照)。
従来の撮像装置の全体構成図の図11を参照。図11は、雑音低減、ゲイン補正およびアナログ−デジタル変換のAFE(Analog Front End)を集積したCMOS(Complementary Metal Oxide Semiconductor)撮像素子であり、映像信号処理で遮光画素信号(以下、OBと称する)補正を行う従来例である。As a pixel defect detection method for a solid-state image sensor of a conventional television camera, image signals obtained by imaging a plurality of spectra obtained by dispersing imaging light at the same imaging position or a nearby imaging position for each spectral light are compared According to the result of the measurement, it is detected whether or not a pixel defect is generated in an image pickup element which has picked up any of the spectral light, and it is interpolated (see Patent Document 1).
See FIG. 11 of the overall configuration of a conventional imaging device. FIG. 11 shows a CMOS (Complementary Metal Oxide Semiconductor) image sensor integrated with AFE (Analog Front End) of noise reduction, gain correction and analog-digital conversion, and a light-shielded pixel signal (hereinafter referred to as OB) in video signal processing This is a conventional example in which correction is performed.
しかしながら、撮像素子で撮像した映像信号には、複数の分光した映像信号毎にランダムノイズ成分が重畳されるため、画素欠陥検出信号に検出誤差を与える事になり、ランダムノイズより小さなレベルの画素欠陥の検出が困難であった。そのため用途がランダムノイズより画素欠陥信号が比較的大きい長時間露光型カメラ等の一部に限られていた(特許文献2参照)。 However, since a random noise component is superimposed on each of a plurality of divided video signals on a video signal picked up by the image pickup element, a detection error is given to the pixel defect detection signal, and a pixel defect of a level smaller than random noise Detection was difficult. Therefore, the application is limited to a part of a long time exposure type camera etc. whose pixel defect signal is relatively larger than random noise (see Patent Document 2).
さらに、撮像素子は一般に、6℃温度上昇で暗電流は2倍程度となっていて、撮像素子温度の指数関数に比例して暗電流は大きく変動する。そのため、画素欠陥の中で感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する)の暗電流と、正常画素の暗電流とを確実に識別して検出することが困難である。暗電流が多い撮像素子温度であれば、画素欠陥の中で暗電流が異常に多い撮像画素(以下、白キズと称する)と正常画素の暗電流との識別は比較的容易である。 Furthermore, in the imaging device, the dark current is generally doubled when the temperature rises by 6 ° C., and the dark current largely fluctuates in proportion to the exponential function of the imaging device temperature. Therefore, among the pixel defects, the dark current of the imaging pixel (hereinafter, referred to as a perfect black flaw) having a sensitivity that is abnormally low and the dark current is only a leakage current and the dark current of the normal pixel are reliably identified and detected. It is difficult. If the image pickup device temperature has a large amount of dark current, it is relatively easy to distinguish an image pickup pixel having a large amount of dark current among pixel defects (hereinafter referred to as a white blemish) from the dark current of a normal pixel.
本発明は、感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する)の暗電流と、6℃温度上昇で暗電流は2倍程度と撮像素子温度の指数関数に比例して暗電流は大きく変動する正常画素の暗電流とを確実に識別して検出して、通常撮像時の完全黒キズを補間することを目的とする。 In the present invention, the dark current of an imaging pixel (hereinafter referred to as a perfect black scratch) having an abnormally low sensitivity and a dark current only as low as the leakage current, and an increase in temperature by 6 ° C. It is an object of the present invention to reliably identify and detect the dark current of a normal pixel which fluctuates largely in proportion to the function, and to interpolate a completely black flaw at the time of normal imaging.
本発明の撮像装置は、(色分解光学系及び3個以上の撮像素子あるいはR,G1,G2,Bのベイヤ配列のオンチップカラーフィルタ付撮像素子あるいはモノクロ)撮像素子と、該撮像素子の(各赤緑青の原色映像信号あるいは(モノクロの)輝度の)映像信号の第一の所定のレベルより大きい暗電流の画素ごとに独立に周囲画素の信号レベルを用いて補間する手段と、映像信号処理で遮光画素映像信号の代表値を有効画素映像信号から減算する遮光画素信号(以下、OBと称する)補正をする映像信号処理手段とを有する撮像装置であって、(撮像素子温度が6℃上昇でおよそ2倍となる)正常画素の暗電流を検出する手段(OB画素の最小値からN番目からN+M番目の平均値を算出する手段)または撮像素子の温度を検出する手段(温度センサ)を有し、該検出した撮像素子温度に対応する(6℃上昇でおよそ2倍となる)正常画素の暗電流に蓄積時間を逆比例させて(およそ1秒から16384秒)遮光蓄積する手段と、映像信号処理のOB補正前に、第二の所定のレベルより小さい暗電流の画素(感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する))を(正常画素の暗電流と識別して)検出する手段と、撮影時に第二の所定のレベルより小さい暗電流の完全黒キズの画素ごとに独立に周囲画素の信号レベルを用いて補間する手段と、を有することを特徴とする。
即ち、撮像素子と、該撮像素子の映像信号の画素ごとに遮光時の暗電流を検出する手段と、第一の所定のレベルより大きい暗電流の画素ごとに独立に周囲画素の信号レベルを用いて補間する手段と、映像信号処理で遮光画素映像信号の代表値を有効画素映像信号から減算する遮光画素信号(以下、OBと称する)補正をする映像信号処理手段と、を有する撮像装置において、
正常画素の暗電流を検出する手段(OB画素の最小値からN番目からN+M番目平均値を算出する手段)または撮像素子の温度を検出する手段を有し、該検出した撮像素子温度に対応する正常画素の暗電流に蓄積時間を逆比例させて遮光蓄積する手段と、
映像信号処理のOB補正前に、第二の所定のレベルより小さい暗電流の画素(以下、完全黒キズと称する)を前記正常画素の暗電流と識別して検出する手段と、
撮影時に第二の所定のレベルより小さい暗電流の完全黒キズの画素ごとに独立に周囲画素の信号レベルを用いて補間する手段と、を有することを特徴とする。The imaging device according to the present invention includes an imaging device (a color separation optical system and an imaging device with three or more imaging devices or an on-chip color filter with monochrome arrangement of R, G1, G2, B, or monochrome), and Means for independently interpolating using the signal level of surrounding pixels for each pixel of dark current larger than the first predetermined level of each red, green, and blue primary color video signal or (monochrome) luminance video signal, and video signal processing An imaging device having video signal processing means for correcting a light-shielded pixel signal (hereinafter referred to as OB) for subtracting a representative value of the light-shielded pixel video signal from the effective pixel video signal; Means that detects the dark current of normal pixels (means for calculating the Nth to N + Mth average values from the minimum value of OB pixels) or means for detecting the temperature of the imaging device (temperature Sensor, and the dark current of a normal pixel corresponding to the detected image sensor temperature (approximately doubled by 6 Means and dark current pixels smaller than the second predetermined level before OB correction of the video signal processing (imaging pixels whose sensitivity is abnormally low and the dark current is only a leak current degree (hereinafter, referred to as complete black flaws)) Means (detecting the dark current of a normal pixel) and means for interpolating using signal levels of surrounding pixels independently for each completely black scratched pixel of dark current smaller than the second predetermined level at the time of photographing And.
That is, the image sensor, means for detecting dark current at the time of light shielding for each pixel of the image signal of the image sensor, and signal levels of surrounding pixels independently for each pixel of dark current larger than the first predetermined level An image pickup apparatus comprising: means for interpolation; and video signal processing means for correcting a light-shielded pixel signal (hereinafter referred to as OB) for subtracting a representative value of the light-shielded pixel video signal from an effective pixel video signal in video signal processing;
Means for detecting dark current of normal pixels (means for calculating Nth to N + Mth average values from the minimum value of OB pixels) or means for detecting the temperature of the imaging device, corresponding to the detected imaging device temperature A means for making the accumulation time inversely proportional to the dark current of the normal pixel and performing light-shielded accumulation;
A means for identifying and detecting a dark current pixel smaller than a second predetermined level (hereinafter referred to as a perfect black flaw) from the dark current of the normal pixel before OB correction of the video signal processing;
And means for performing interpolation using signal levels of surrounding pixels independently for every completely black scratch pixel of dark current smaller than a second predetermined level at the time of photographing.
また、本発明の撮像装置は、上述の撮像装置であって、垂直走査周期のおよそ15倍以上(の該検出した撮像素子温度に対応する(6℃上昇でおよそ2倍となる)正常画素の暗電流に蓄積時間を逆比例させて垂直走査周期のおよそ15倍からおよそ245760倍で垂直走査周期約60Hzとして0.25秒から4096秒の可変時間又は垂直走査周期のおよそ60倍で垂直走査周期約60Hzとしておよそ1秒の固定時間)の遮光蓄積時間で暗電流が異常に多い画素欠陥(以下、白キズと称する)の検出を行う手段と、上記検出した撮像素子温度が6℃でおよそ2倍となる正常画素の暗電流に蓄積時間を逆比例させて(およそ1秒から16384秒)遮光蓄積して(ランダムノイズの影響を除去して、)OB補正前に白キズ補間を行う手段と、白キズ補間後に第二の所定のレベルより小さい暗電流の画素(感度が異常に低い撮像画素(以下、完全黒キズと称する))を(正常画素の暗電流と識別して)検出する手段と、撮影時に完全黒キズごとに独立に周囲画素の信号レベルを用いて補間する手段と、完全黒キズ補間後にOB補正を行う手段と、を有することを特徴とする。
即ち、上述の撮像装置において、
垂直走査周期に基づく前記検出した撮像素子温度に対応する正常画素の暗電流に蓄積時間を逆比例させて垂直走査周期の任意の倍数の範囲で垂直走査周期の所定の可変時間又は垂直走査周期の所定の遮光蓄積時間で暗電流が所定以上の画素欠陥(以下、白キズと称する)の検出を行う手段と、
上記検出した撮像素子温度に対応する正常画素の暗電流に蓄積時間を逆比例させて遮光蓄積してOB補正前に白キズ補間を行う手段と、
白キズ補間後に前記完全黒キズを前記正常画素の暗電流と識別して検出する手段と、
撮影時に前記完全黒キズごとに独立に周囲画素の信号レベルを用いて補間する手段と、
完全黒キズ補間後にOB補正を行う手段と、を有することを特徴とする。
さらに、上述の撮像装置であって、白キズ検出の蓄積での白キズ基準レベル又は標準撮像時の白キズ補間レベルと、判定レベルを入れ替えて、白キズの映像信号タイミング(アドレス)判定と標準撮像時の白キズ補間判定とを同一手段で行うことを特徴とする。
また、上記撮像装置において、白キズ補間後に完全黒キズ補間するので、(撮像信号の周囲画素信号及び撮像信号、又は白キズ補間信号の周囲画素信号及び白キズ補間信号の様に)入力信号を変えて、白キズのレベル以下判定時の白キズの映像信号タイミング(アドレス)、又は完全黒キズの映像信号タイミング(アドレス)の様に)制御信号を変えることにより、同一回路で白キズ補間と完全黒キズ補間と2回処理することで、白キズ補間と完全黒キズ補間とを同一回路で行う手段とを有することを特徴とする。An imaging apparatus according to the present invention is the above-described imaging apparatus, which has approximately 15 times or more of the vertical scanning cycle (of the normal pixel corresponding to the detected imaging element temperature (approximately doubled by an increase of 6.degree. C.)). Vertical scan period with variable time of about 25 to 4096 seconds or about 60 times the vertical scanning period, with the storage time being inversely proportional to dark current, about 15 times to about 245760 times the vertical scanning period and about 60 Hz for the vertical scanning period A means for detecting a pixel defect (hereinafter referred to as a white defect) having an abnormally large dark current in a light-blocking accumulation time of about 60 Hz and a fixed time of about 1 second), and the detected image sensor temperature is about 2 at 6 ° C. Means of performing white defect interpolation before OB correction (in order to eliminate the influence of random noise) by making the accumulation time inversely proportional (approximately 1 second to 16384 seconds) to the dark current of the normal pixel to be doubled A means for detecting a pixel whose dark current is smaller than the second predetermined level after white blemish interpolation (imaged pixel having an extremely low sensitivity (hereinafter referred to as a complete black blemish)) And means for performing interpolation using signal levels of surrounding pixels independently for each complete black defect at the time of shooting, and means for performing OB correction after complete black defect interpolation.
That is, in the above-described imaging device,
The dark current of the normal pixel corresponding to the detected image sensor temperature based on the vertical scanning cycle makes the storage time inversely proportional to a predetermined variable time or vertical scanning cycle of the vertical scanning cycle within the range of any multiple of the vertical scanning cycle A means for detecting a pixel defect (hereinafter referred to as a white defect) having a dark current of a predetermined value or more in a predetermined light shielding accumulation time;
A means for performing the white defect interpolation before the OB correction by making the accumulation time inversely proportional to the dark current of the normal pixel corresponding to the detected image pickup device temperature and performing the shading correction before the OB correction;
A means for discriminating and detecting the completely black flaw from the dark current of the normal pixel after white flaw interpolation;
A means for performing interpolation using signal levels of surrounding pixels independently for each of the completely black flaws at the time of photographing;
And means for performing OB correction after perfect black-and-whites interpolation.
Furthermore, in the above-described imaging device, the white defect reference level at the accumulation of white defect detection or the white defect interpolation level at the time of standard imaging and the determination level are interchanged, and the image signal timing (address) determination and standard of white defects is performed. The white defect interpolation determination at the time of imaging is performed by the same means.
Further, in the above imaging apparatus, since perfect black scale interpolation is performed after white scale interpolation, (like peripheral pixel signals and image pickup signals of an image pickup signal or surrounding pixel signals of white scale interpolation signal and white scale interpolation signal) By changing the control signal of the white scratch video signal timing (address) at the time of determination below the white scratch level or the video signal timing (address) of a completely black scratch, the white scratch interpolation and the same circuit can be performed. It is characterized by having means for performing white defect interpolation and complete black defect interpolation with the same circuit by performing complete black defect interpolation and processing twice.
また、本発明の撮像装置の調整方法は、色分解光学系及び3個以上の撮像素子あるいはR,G1,G2,Bのベイヤ配列のオンチップカラーフィルタ付撮像素子と、該撮像素子の各赤緑青の原色映像信号の画素ごとに遮光時の暗電流を検出する手段と、第一の所定のレベルより大きい暗電流の画素ごとに独立に周囲画素の信号レベルを用いて補間する手段と、映像信号処理で遮光画素映像信号の代表値を有効画素映像信号から減算する遮光画素信号(以下、OBと称する)補正をする映像信号処理手段とを有する撮像装置であって、撮像装置を組立調整後の(ヒートランまたは高温エージング等)撮像素子温度が40℃以上の状態時に、およそ1秒から16秒遮光蓄積し、映像信号処理のOB補正前に、第二の所定のレベルより小さい暗電流の画素(感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する))を(正常画素の暗電流と識別して)検出し、撮影時に完全黒キズの画素ごとに独立に周囲画素の信号レベルを用いて補間することを特徴とする。
即ち、本発明の撮像装置の調整方法は、色分解光学系及び3個以上の撮像素子あるいはR,G1,G2,Bのベイヤ配列のカラーフィルタ付撮像素子と、該撮像素子の各赤緑青の原色映像信号の画素ごとに遮光時の暗電流を検出する手段と、第一の所定のレベルより大きい暗電流の画素ごとに独立に周囲画素の信号レベルを用いて補間する手段と、映像信号処理で遮光画素映像信号の代表値を有効画素の映像信号から減算する遮光画素信号(以下、OBと称する)補正をする映像信号処理手段とを有する撮像装置において、
撮像装置を撮像素子温度が所定以上の温度の状態時に、所定時間範囲にて遮光蓄積し、
映像信号処理のOB補正前に、第二の所定のレベルより小さい暗電流の画素(以下、完全黒キズと称する)を正常画素の暗電流と識別して検出し、
撮影時に前記完全黒キズの画素ごとに独立に周囲画素の信号レベルを用いて補間することを特徴とする。Further, according to the adjustment method of an image pickup device of the present invention, a color separation optical system and an image pickup element with on-chip color filter of Bayer arrangement of three or more image pickup elements or R, G1, G2, B, and each red of the image pickup element A means for detecting dark current at the time of light shielding for each pixel of the green and blue primary color video signals; a means for independently interpolating using signal levels of surrounding pixels for each pixel of dark current larger than the first predetermined level; An imaging apparatus comprising: video signal processing means for correcting a light shielding pixel signal (hereinafter referred to as OB) for subtracting a representative value of a light shielding pixel video signal from an effective pixel video signal in signal processing; (Heat run or high temperature aging, etc.) When the imaging device temperature is 40 ° C. or more, dark current is accumulated for about 1 to 16 seconds, and dark current smaller than the second predetermined level before OB correction of video signal processing Detects pixels (imaged pixels whose sensitivity is low and dark current is leak current only) (identified as dark current of the normal pixel) (detected as dark current of normal pixel), The interpolation is performed independently using signal levels of surrounding pixels.
That is, according to the adjustment method of the image pickup apparatus of the present invention, a color separation optical system and an image pickup element with three or more image pickup elements or a color filter with Bayer arrangement of R, G1, G2 and B, and red, green and blue of each of the image pickup elements A means for detecting dark current at the time of light shielding for each pixel of the primary color video signal, a means for independently interpolating using the signal level of surrounding pixels for each pixel of dark current larger than the first predetermined level, and video signal processing And an image signal processing unit for correcting a light-shielded pixel signal (hereinafter referred to as "OB") for subtracting the representative value of the light-shielded pixel video signal from the video signal of the effective pixel.
When the image pickup device temperature is a predetermined temperature or more, the image pickup device is light-shielded and accumulated in a predetermined time range,
Before OB correction of video signal processing, a pixel having a dark current smaller than a second predetermined level (hereinafter, referred to as a completely black flaw) is identified and detected as a dark current of a normal pixel,
At the time of photographing, interpolation is performed using signal levels of surrounding pixels independently for each pixel of the completely black flaw.
本発明によれば、感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する)信号も自動検出でき、かつ完全黒キズの周囲画素を用いた補間が可能となる。
また、白キズ補間後完全黒キズ補間後に映像信号処理で遮光画素信号(以下、OBと称する)補正するので、OB補正は白キズと完全黒キズの影響を受けないで安定に動作するので、映像信号の黒が安定し、テレビカメラのワイドダイナミックレンジ化が容易になる。According to the present invention, it is possible to automatically detect an imaging pixel (hereinafter, referred to as a complete black flaw) signal whose sensitivity is low and dark current is only a leakage current, and interpolation using surrounding pixels of complete black flaw is possible. Become.
In addition, since the light-shielded pixel signal (hereinafter referred to as OB) correction is performed in the video signal processing after the perfect black scratch interpolation after white scratch interpolation, the OB correction operates stably without being affected by the white scratch and the perfect black scratch. The black of the video signal is stabilized, and the wide dynamic range of the television camera can be easily made.
以下、本発明の実施形態について図面を参照して詳細に説明する。
図1Aと図1Bと図1Cと図1Dとは本発明の撮像装置の全体構成の一実施例を示すブロック図であり、雑音低減、ゲイン補正およびアナログ−デジタル変換のAFEを集積したCMOS撮像素子で、映像信号処理で遮光画素の映像信号の代表値を有効画素映像信号から減算するOB補正を行う。
図1Aのテレビジョンカメラ30はガンマ後マトリクスの映像信号処理で、R,G,Bの3板撮像素子を用いたカラー撮像装置である。
図1Bは本発明の撮像装置の全体構成の他の一実施例を示すブロック図で、R,G1,G2,Bの4板撮像素子を用いたカラー撮像装置である。
図1Cは本発明の撮像装置の全体構成の他の一実施例を示すブロック図で、R,G1,G2,Bのベイヤ配列のオンチップカラーフィルタ付撮像素子を用いたカラー撮像装置である。
図1Dは本発明の撮像装置の全体構成の他の一実施例を示すブロック図で、モノクロ撮像素子を用いたモノクロ撮像装置である。
なお、本発明は、撮像素子をCMOSに限定するものではなく、CCD(Charge Coupled Device)等であってもよい。Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIGS. 1A, 1B, 1C, and 1D are block diagrams showing an embodiment of the entire configuration of the imaging apparatus of the present invention, and are CMOS imaging elements in which AFE of noise reduction, gain correction, and analog-digital conversion is integrated. Then, in the video signal processing, OB correction for subtracting the representative value of the video signal of the light-shielded pixel from the effective pixel video signal is performed.
The
FIG. 1B is a block diagram showing another embodiment of the entire configuration of the imaging apparatus of the present invention, which is a color imaging apparatus using four-plate imaging elements of R, G1, G2, and B. In FIG.
FIG. 1C is a block diagram showing another embodiment of the entire configuration of the imaging apparatus of the present invention, which is a color imaging apparatus using an imaging element with an on-chip color filter in Bayer arrangement of R, G1, G2 and B.
FIG. 1D is a block diagram showing another embodiment of the entire configuration of the imaging apparatus of the present invention, which is a monochrome imaging apparatus using a monochrome imaging device.
The present invention does not limit the imaging device to CMOS, and may be a CCD (Charge Coupled Device) or the like.
図1Aと図1Bにおいて、被写体からの入射光はレンズ部31で結像され、プリズム部32で赤色光と緑色光および青色光に分解され、各々CMOS撮像素子部33R,33G,33G1、33G2、33Bで光電変換される。光電変換されたR/G/Bの信号は、CMOS撮像素子内で雑音低減、ゲイン補正およびアナログ−デジタル変換とを行い、白キズ完全黒キズ検出補正機能付映像信号処理部の35に送られ、色補正、輪郭補正、ガンマ補正、ニー補正等の各種映像信号処理が行われる。
In FIGS. 1A and 1B, incident light from an object is imaged by the
図1Aと図1Bと図1Cと図1Dにおいて、温度センサ20はCMOS撮像素子部33R,33G,33G1,33G2,33B,58,59の近傍に設けて温度を検出する。
撮像素子制御54は、CPU(Central Processing Unit)39の指示に従いCMOS撮像素子部33R,33G,33G1,33G2,33B,58,59の蓄積(露光)や読出しを制御する。
レンズ部31はCPU39の指示に従い遮光または標準撮像を光学絞りまたは可変光学減衰(以下、絞りと称する)で制御する。
また、50は白キズ検出補間、51は完全黒キズ検出補間、52は暗電流算出OB補正、53はガンマ色輪郭補正である。
図1Bと図1Cとにおいて、57はパラレル−シリアル変換部(P/S)であり、パラレル−シリアル変換部(P/S)の57により、G1とG2の撮像信号からGの撮像信号が生成される。
図1Dにおいて、59はモノクロ撮像素子で、60はガンマ輪郭補正であり、モノクロの輝度(Y)信号のみが出力される。In FIGS. 1A, 1B, 1C, and 1D, the
The
The
Further, 50 is white defect detection interpolation, 51 is complete black defect detection interpolation, 52 is dark current calculation OB correction, and 53 is gamma color contour correction.
In FIGS. 1B and 1C,
In FIG. 1D, 59 is a monochrome imaging device, 60 is gamma contour correction, and only monochrome luminance (Y) signal is output.
デジタル信号処理部5では各種映像信号処理などが施された後、
Y=0.2126R+0.7152G+0.0722B Pb=0.5389(B−Y) Pr=0.6350(R−Y)の計算式により、R/G/Bから輝度信号(Y)と色差信号(Pb/Pr)に変換する。そしてパラレル−シリアル変換部(P/S)37でシリアル映像信号に変換され、外部に出力される。After various video signal processing and the like are performed in the digital
Y = 0.2126 R + 0.7152 G + 0.0722 B Pb = 0.5389 (B−Y) Pr = 0.6350 (R−Y) From the R / G / B to the luminance signal (Y) and the color difference signal (Pb) Convert to / Pr). Then, it is converted into a serial video signal by the parallel-serial converter (P / S) 37 and output to the outside.
CPU39は、テレビジョンカメラ30の各部を制御する。また、ビューファインダまたはモニタディスプレイの画像表示部40は撮像装置の設定用メニューや暗電流が異常に多い撮像画素(以下、白キズと称する)と正常画素と感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する)との自動補間操作や任意の画素の手動での周囲画素での補間操作を表示する。
ビューファインダまたはモニタディスプレイの40では被写体の映像にメニュー画面を重畳し、ユーザーはメニュー画面を見ながら白キズと完全黒キズとの自動検出補間操作や任意の画素の手動での周囲画素での補間操作を表示する。The
In the viewfinder or monitor
(実施例1、検出と補間の概要) 以下、本発明の一実施例を図1Aと図1Bと図1Cと図1D、図2、図3A、図3Bを用いて説明する。
ここで、図2は、暗電流が異常に多い白キズと正常画素と感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する)の遮光時の、映像信号処理でOB画素の映像信号の代表値を有効画素映像信号から減算するOB補正前の撮像信号(水平走査線)での完全黒キズ映像信号の撮像装置起動時から内部温度上昇の飽和までの変化を示す模式図模式図であり、(a)は遮光時のOB補正前の撮像信号(水平走査線)での感度が異常に低い撮像画素(以下、完全黒キズと称する)映像信号の撮像装置の起動時から内部温度上昇の飽和までの変化を示す模式図であり、(b)は 撮像素子の温度と蓄積時間と暗電流との白キズの映像信号の撮像装置起動時から内部温度上昇の飽和までの変化を示す模式図である。
撮像素子は一般に、6℃温度上昇で暗電流は2倍程度となっている。放熱に工夫しているカメラでは、一般に、内部温度上昇がおよそ2時間で飽和し12℃程度となっている。そのため、周囲温度が一定であっても、内部温度上昇12℃で、起動時に比べ、内部温度上昇の飽和時には4倍となる。通常はその中間となる。
Here, FIG. 2 shows a video signal at the time of light shielding of an imaging pixel (hereinafter referred to as a perfect black defect) in which the dark current is abnormally high and the white pixel and the normal pixel have an abnormally low sensitivity. Change from the time the image pickup device starts up of the complete black scratch image signal to the saturation of the internal temperature rise in the imaging signal (horizontal scanning line) before OB correction that subtracts the representative value of the video signal of OB pixel from the effective pixel video signal (A) is an imaging device for an imaging pixel (hereinafter, referred to as complete black flaw) image signal having an abnormally low sensitivity in an imaging signal (horizontal scanning line) before OB correction at the time of light shielding. (B) is a schematic diagram showing a change from the start of the image to the saturation of the internal temperature rise, and (b) shows the temperature rise of the image signal of the image sensor temperature, the storage time and the dark current It is a schematic diagram which shows the change to saturation.
In general, in the imaging device, the dark current is approximately doubled when the temperature rises by 6 ° C. In cameras that devise heat dissipation, the internal temperature rise generally saturates in about 2 hours and reaches about 12 ° C. Therefore, even if the ambient temperature is constant, the internal temperature rise is 12 ° C., and the internal temperature rise is quadrupled at the time of saturation of the internal temperature rise as compared with the start time. Usually it is in the middle.
図3Aは、遮光時の遮光画素映像信号の代表値を有効画素映像信号から減算する遮光画素信号(以下、OBと称する)補正をする前の撮像信号の撮像素子の温度と蓄積時間と暗電流との白キズと正常画素と完全黒キズの映像信号の撮像装置起動時から内部温度上昇の飽和までの変化と撮像素子の14bit出力の垂直走査周期単位の蓄積時間による出力飽和を示す模式図である。垂直走査周期は一般に約60Hzや50HZが主である。約120Hzや150Hzや約180Hzや約240Hz等の高速も約24Hzや約30Hz等の低速もある。以下、垂直走査周期は約60Hzで垂直走査周期と蓄積時間との関係を説明する。 FIG. 3A shows the temperature, the accumulation time, and the dark current of the image pickup element of the image pickup signal before the light shielding pixel signal (hereinafter referred to as OB) correction that subtracts the representative value of the light shielding pixel video signal at the light shielding time from the effective pixel video signal. It is a schematic diagram showing the output saturation by the accumulation time of the vertical scanning cycle unit unit of the change from the image pickup device startup of the white scratch and normal pixel and the complete black scratch image signal to saturation of internal temperature rise and 14 bit output of the image sensor. is there. The vertical scanning period is generally about 60 Hz or 50 Hz. There are also high speeds such as about 120 Hz, 150 Hz, about 180 Hz and about 240 Hz, and low speeds such as about 24 Hz and about 30 Hz. Hereinafter, the vertical scanning cycle is about 60 Hz, and the relationship between the vertical scanning cycle and the accumulation time will be described.
図3Bは、遮光時のOB補正前の撮像信号の撮像素子の温度と蓄積時間の比と暗電流の比と撮像素子の温度と完全黒キズを検出する遮光蓄積時間を示す模式図である。
漏れ電流相当の暗電流しかない完全黒キズを検出するため、白キズ検出の蓄積時間は白キズが飽和しない中時間(垂直走査周期の約60Hzの60倍のおよそ1秒)であるのに対し、完全黒キズ検出の蓄積時間は撮像素子の温度に6℃で2倍の正常画素の暗電流に反比例させて長時間(1秒から163894秒)とすることにより、正常画素の暗電流と、漏れ電流相当の暗電流しかない完全黒キズの暗電流と確実に識別する。FIG. 3B is a schematic view showing the ratio of the ratio of the temperature and storage time of the image pickup device before the OB correction at the time of light shielding to the storage time and the ratio of dark current, and the temperature of the image pickup device and the blackout detection time.
The accumulation time of white flaw detection is a medium time (about 60 seconds of
白キズ検出と異なり、完全黒キズ信号検出は困難なので、検出時の長時間蓄積が許容される。
具体的には、撮影直前の検出は、撮像素子温度22℃で128秒程度が低温の実用限界であるが、撮像装置を組立調整後のヒートランまたは高温エージング等なら、最高使用温度45℃で放熱に工夫しているカメラで内部温度上昇約12℃で撮像素子温度57℃なら完全黒キズ検出の遮光蓄積時間は約2秒となる。
最高使用温度45℃で放熱に工夫していないカメラは内部温度上昇約19℃で撮像素子温度が64℃となり、完全黒キズ検出の遮光蓄積時間は垂直走査周期の約60Hzの60倍の約1秒となる。
最高使用温度40℃で特に放熱に工夫しているカメラは内部温度上昇約8℃で撮像素子温度が48℃となり、完全黒キズ検出の遮光蓄積時間は約16秒となる。ヒートランまたは高温エージング等での完全黒キズ検出の遮光蓄積時間は約1秒から約16秒なら、原価上昇はほとんどない。Unlike white blemish detection, since full black blemish signal detection is difficult, long-time accumulation at the time of detection is allowed.
Specifically, the detection limit immediately before shooting is a practical limit at a low temperature of about 128 seconds at an imaging device temperature of 22 ° C, but if it is a heat run after assembly adjustment of the imaging device or high temperature aging, etc. If the image pickup device temperature is 57 ° C. and the internal temperature rise is about 12 ° C., the light shielding accumulation time for complete black flaw detection is about 2 seconds.
Cameras not designed for heat dissipation at the maximum use temperature of 45 ° C have an internal temperature rise of about 19 ° C and the image pickup device temperature is 64 ° C, and the dark storage detection time of complete black flaw detection is about 1 of 60 times of about 60 Hz of the vertical scanning cycle It will be seconds.
The camera that devises heat dissipation particularly at the maximum use temperature of 40 ° C. has an internal temperature rise of about 8 ° C., and the image pickup device temperature becomes 48 ° C. There is almost no increase in cost if the dark storage time of complete black flaw detection in heat run or high temperature aging is about 1 second to about 16 seconds.
本発明の撮像装置の全体構成の一実施例を示すブロック図の図1Aと図1Bと図1Cと図1Dの33R,33G,33G1,33G2,33Bまたは58または59のCMOS撮像素子の近傍に、温度センサ20を設けてCMOS撮像素子の温度を検出し、完全黒キズ検出の遮光蓄積時間は撮像素子の温度に6℃で2倍の正常画素の暗電流に反比例させても良い。暗電流算出OB補正52で遮光された正常画素の暗電流の代表値を算出しても良い。
正常画素の暗電流が少ない低温時において、正常画素の暗電流と、完全黒キズの暗電流と確実に識別するために、完全黒キズ検出の遮光蓄積時間は撮像素子の温度に6℃で2倍の正常画素の暗電流に正確に反比例させるために、暗電流算出OB補正52で遮光された正常画素の暗電流の代表値を算出する方法は、後述の実施例3で説明する。1A, 1B, 1C, 1D and 33R, 33G,
In order to reliably distinguish between the dark current of a normal pixel and the dark current of a completely black flaw at low temperature when the dark current of the normal pixel is small, the light blocking accumulation time of the completely black flaw detection is 6 ° C. at the temperature of the image pickup element A method of calculating the representative value of the dark current of the normal pixel shielded by the dark current
以上のように本発明は、色分解光学系及び3個以上の撮像素子あるいはR,G1,G2,Bのベイヤ配列のオンチップカラーフィルタ付撮像素子あるいはモノクロ撮像素子と、該撮像素子の各赤緑青の原色映像信号あるいは(モノクロの)輝度映像信号の画素ごとに遮光時の暗電流を検出する手段と、第一の所定のレベルより大きい暗電流の画素ごとに独立に周囲画素の信号レベルを用いて補間する手段と、映像信号処理で遮光画素映像信号の代表値を有効画素映像信号から減算する遮光画素信号(以下、OBと称する)補正をする映像信号処理手段とを有する撮像装置において、
撮像装置を組立調整後の(ヒートランまたは高温エージング等)撮像素子温度が40℃以上の状態時に、垂直走査周期の約60Hzの60倍のおよそ1秒から16秒(できれば周囲温度40℃以上、内部温度上昇11℃以上、撮像素子温度が51℃以上でおよそ4秒)遮光蓄積して映像信号処理のOB補正前に又はOB補正を中止して、第二の所定のレベルより小さい暗電流の画素(感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する))を(正常画素の暗電流と識別して)検出し、撮影時に第二の所定のレベルより小さい暗電流の完全黒キズの画素ごとに独立に周囲画素の信号レベルを用いて補間することを特徴とする撮像装置の調整方法である。As described above, according to the present invention, a color separation optical system and an imaging element with three or more imaging elements or an on-chip color filter with a Bayer arrangement of R, G1, G2 and B, or a monochrome imaging element, and each red of the imaging element Means for detecting dark current at the time of light blocking for each pixel of the primary color video signal of green and blue or luminance video signal (monochrome), and signal levels of surrounding pixels independently for each pixel of dark current larger than the first predetermined level An image pickup apparatus comprising: means for interpolation using; and video signal processing means for correcting a light-shielded pixel signal (hereinafter referred to as OB) for subtracting a representative value of a light-shielded pixel video signal from an effective pixel video signal in video signal processing.
When the imaging device temperature is 40 ° C or higher after assembling and adjusting the imaging device (heat run or high temperature aging, etc.), approximately 1 to 60 seconds of about 60 Hz of the vertical scanning cycle (preferably
また、上記撮像装置において、撮像素子温度に対応する(6℃上昇でおよそ2倍となる)正常画素の暗電流を検出する手段(OB画素の最小値からNからN+M番目の平均値を算出する手段)または撮像素子の温度を検出する手段(温度センサ)と、該検出した撮像素子温度に対応する(6℃でおよそ2倍となる)正常画素の暗電流に反比例させて長時間(1秒から163894秒)遮光蓄積し読出を行い、第二の所定のレベルより小さい暗電流の画素(感度が異常に低い撮像画素(以下、完全黒キズと称する))を(正常画素の暗電流と識別して)検出する手段と、撮影時に完全黒キズの画素ごとに独立に周囲画素の信号レベルを用いて補間する手段と、を有することを特徴とする撮像装置である。(具体策は実施例3で説明する。) In the above imaging apparatus, means for detecting dark current of a normal pixel corresponding to the temperature of the imaging element (approximately doubling at 6 ° C. increase) (N to N + Mth average values are calculated from the minimum value of OB pixels Means) or means (temperature sensor) for detecting the temperature of the imaging device, and inversely proportional to the dark current of a normal pixel corresponding to the detected imaging device temperature (approximately doubled at 6.degree. C.) for a long time (1 second And storing the readout and reading out the dark current smaller than the second predetermined level (identifying an imaging pixel whose sensitivity is abnormally low (hereinafter referred to as a complete black defect)) as the dark current of a normal pixel And an imaging device characterized in that it comprises means for detecting and means for interpolating independently using signal levels of surrounding pixels for each completely black flawed pixel at the time of photographing. (Specific measures will be described in the third embodiment.)
(実施例2、検出と補間の詳細) 以下、本発明の他の1実施例を図1Aと図1Bと図1Cと図4〜図10を用いて説明する。
図4は本発明の一実施例の白キズ検出と補間の詳細構成図である。
図5は本発明の一実施例の完全黒キズ検出と補間の詳細構成図である。Second Embodiment Details of Detection and Interpolation Another embodiment of the present invention will be described below with reference to FIGS. 1A, 1 B, 1 C, and 4 to 10.
FIG. 4 is a detailed block diagram of white defect detection and interpolation according to an embodiment of the present invention.
FIG. 5 is a detailed block diagram of complete black blemish detection and interpolation according to an embodiment of the present invention.
図6は本発明の一実施例を説明するための模式図である。
図6(A)は本発明の一実施例であり、遮光時のOB補正前の完全黒キズを周囲8画素の中央値で補間する模式図(標準撮像信号での完全黒キズの周囲正常画素での補間)である。
図6(B)は本発明の一実施例であり、遮光時のOB補正前の完全黒キズを周囲8画素の中央値で補間する模式図(標準撮像信号での完全黒キズの周囲黒キズを含む正常画素での補間)である。
図6(C)は本発明の一実施例であり、遮光時のOB補正前の撮像信号での白キズを周囲8画素の中央値で補間する模式図(標準撮像信号での白キズの周囲正常画素での補間)である。
図6(D)は本発明の一実施例であり、遮光時のOB補正前の撮像信号での白キズを周囲8画素の中央値で補間する模式図(標準撮像信号での白キズの周囲白キズを含む正常画素での補間)である。FIG. 6 is a schematic view for explaining an embodiment of the present invention.
FIG. 6A is an embodiment of the present invention, and is a schematic diagram in which a complete black defect before OB correction at the time of light shielding is interpolated with the median of eight pixels around (a normal pixel around a complete black defect in a standard imaging signal Interpolation).
FIG. 6B is an embodiment of the present invention, a schematic diagram in which a perfect black defect before OB correction at the time of light shielding is interpolated by the median of eight pixels around (a surrounding black defect of a perfect black defect in a standard imaging signal Interpolation at normal pixels including
FIG. 6C is an embodiment of the present invention, and is a schematic diagram in which white defects in an image pickup signal before OB correction at the time of light shielding are interpolated with a median of 8 pixels around (white defects in a standard image pickup signal Interpolation at normal pixels).
FIG. 6D is an embodiment of the present invention, and is a schematic diagram in which white flaws in an image pickup signal before OB correction at the time of light shielding are interpolated with the median of eight surrounding pixels (around white flaws in a standard image pickup signal Interpolation at a normal pixel including white flaws).
図7は本発明の一実施例の白キズの周囲8画素の中央値を算出し、白キズを補間する詳細構成図である。
図8は本発明の一実施例の完全黒キズの周囲8画素の中央値を算出し、完全黒キズを補間する詳細構成図である。
図9は本発明の一実施例の白キズ検出と完全黒キズ検出とのフローチャートであり、図10は発明の一実施例の白キズ補間と完全黒キズ補間とのフローチャートである。FIG. 7 is a detailed block diagram of calculating medians of eight pixels around white defects and interpolating white defects according to an embodiment of the present invention.
FIG. 8 is a detailed block diagram for calculating the median of eight pixels around a complete black defect according to an embodiment of the present invention and interpolating the complete black defect.
FIG. 9 is a flowchart of white flaw detection and complete black flaw detection according to one embodiment of the present invention, and FIG. 10 is a flowchart of white flaw interpolation and complete black flaw interpolation according to one embodiment of the invention.
図4は本発明の一実施例の暗電流が異常に多い撮像画素(いわゆる白キズ)検出と補間の詳細構成図であり、図5は本発明の一実施例の感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する)検出の詳細構成図である。
図4と図5において、Rの白キズ補間信号、Gの白キズ補間信号、Bの白キズ補間信号から完全黒キズ検出の長時間蓄積での完全黒キズ基準レベルを減算し、完全黒キズの映像信号タイミング(アドレス)を判定し、標準撮像時に、標準撮像時の完全黒キズ補間レベル判定はしないで、完全黒キズの映像信号タイミング(アドレス)で完全黒キズの周囲画素での補間を行う。FIG. 4 is a detailed configuration diagram of imaging pixels (so-called white flaws) detection and interpolation with abnormally high dark current according to an embodiment of the present invention, and FIG. 5 is abnormally low in sensitivity with an embodiment according to the present invention. Is a detailed configuration diagram of an imaging pixel (hereinafter, referred to as a complete black flaw) detection having only a leakage current level.
In FIG. 4 and FIG. 5, the complete black defect reference level in the long time accumulation of complete black defect detection is subtracted from the white defect interpolation signal of R, the white defect interpolation signal of G, and the white defect interpolation signal of B to complete black defect Video signal timing (address) is determined, and during standard imaging, complete black flaw interpolation level determination during standard imaging is not performed, and interpolation at completely black flaw peripheral pixels is performed with complete black flaw video signal timing (address) Do.
図4と図5において、12,13,14は減算器であり、10は白キズの検出と補間部、11は黒キズの検出と補間部であり、15は白キズの映像信号タイミング(アドレス)判定と標準撮像時の白キズ補間判定、16は白キズの周囲画素での補間、17は黒キズの映像信号タイミング(アドレス)判定、18は黒キズの周囲画素での補間である。
In FIGS. 4 and 5,
本発明の撮像装置の全体構成の一実施例を示すブロック図の図1Aと図1Bと図1Cと図4と図5において、CPU39は、レンズ31に絞りを閉じてCMOS撮像素子33R,33G,33Bを遮光する指示をする。次にCPU39は、遮光撮像素子制御54へおよそ1秒の中時間蓄積を指示し、撮像素子制御54は白キズ検出用中時間間欠パルスを発生し、CMOS撮像素子33R,33G,33Bに供給する。そして、遮光中時間蓄積したR,G,Bの撮像信号と白キズ検出の中時間蓄積での白キズ基準レベルとの12,13,14の減算器差分から、15は白キズの映像信号タイミング(アドレス)判定を行う。
In FIGS. 1A, 1B, 1C, 4 and 5 of the block diagram showing one embodiment of the entire configuration of the imaging apparatus of the present invention, the
図1Aと図1Bと図1Cと図4と図5において、次にCPU39は、撮像素子制御54へ正常画素の暗電流に反比例させて長時間(1秒から163894秒)蓄積を指示し、撮像素子制御54は完全黒キズ検出用長時間間欠パルスを発生し、CMOS撮像素子33R,33G,33Bに供給する。そして、遮光長時間蓄積したR,G,Bの撮像信号を白キズの映像信号タイミング(アドレス)に基づき16で白キズの周囲画素での補間を行ったR,G,Bの補間信号と完全黒キズ検出の長時間蓄積での完全黒キズ基準レベルとの12,13,14の減算器差分から、17は完全黒キズの映像信号タイミング(アドレス)判定を行う。
1A, 1B, 1C, 4 and 5, next, the
標準撮像時に、CPU39は、レンズ31に絞りを開かせ、遮光撮像素子制御54へ標準撮像を指示し、撮像素子制御54は標準撮像パルスを発生し、CMOS撮像素子33R,33G,33Bに供給する。標準撮像したR,G,Bの撮像信号と標準撮像時の白キズ補間レベルとの12,13,14の減算器差分から、15は白キズの映像信号タイミング(アドレス)判定を行う。16は白キズ補間レベル以下の撮像信号で白キズの映像信号タイミング(アドレス)で白キズの周囲画素での補間を行う。16で白キズの周囲画素での補間を行ったR,G,Bの補間信号を18は完全黒キズの映像信号タイミング(アドレス)に基づき、完全黒キズを周囲画素の中央値で補間を行う。標準撮像時の完全黒キズ補間レベル判定はしない。
At the time of standard imaging, the
また、図4において、白キズ検出の中時間(垂直走査周期のおよそ15倍以上(の該検出した撮像素子温度に対応する(6℃上昇でおよそ2倍となる)正常画素の暗電流に蓄積時間を逆比例させて垂直走査周期のおよそ15倍からおよそ245760倍で垂直走査周期約60Hzとして0.25秒から4096秒の可変時間又は垂直走査周期のおよそ60倍で垂直走査周期約60Hzとしておよそ1秒の固定時間)の)遮光蓄積時間での白キズ基準レベル又は標準撮像時の白キズ補間レベルと、判定レベルを入れ替えて、白キズの映像信号タイミング(アドレス)判定と標準撮像時の白キズ補間判定とを同一手段で行うことが可能であり、回路の小型化と低価格化が実現できる。 In addition, in FIG. 4, the middle time of the white defect detection (approximately 15 times or more of the vertical scanning cycle (of which corresponding to the detected image pickup device temperature (approx. Inverse time, approximately 15 times to approximately 245760 times the vertical scanning period and approximately 60 Hz for the vertical scanning period, or approximately 60 times as vertical scanning period at approximately 60 times the variable time of 0.25 seconds to 4096 seconds or approximately 60 Hz for the vertical scanning period The white defect reference level for 1 sec fixed time) and the white defect interpolation level at standard imaging or the white defect interpolation level at standard imaging are replaced with the determination level to determine the video signal timing (address) of white defects and white at standard imaging The defect interpolation determination can be performed by the same means, and the circuit can be downsized and the cost can be reduced.
図6(A)は本発明の一実施例であり、遮光時のOB補正前の完全黒キズを周囲8画素の中央値で補間する模式図(標準撮像信号での完全黒キズの周囲正常画素での補間)であり、中央の完全黒キズ撮像信号0が、周囲の周囲正常画素撮像信号32,48.56,64.80,96,102,128の中央値64又は80で補間される。
FIG. 6A is an embodiment of the present invention, and is a schematic diagram in which a complete black defect before OB correction at the time of light shielding is interpolated with the median of eight pixels around (a normal pixel around a complete black defect in a standard imaging signal Interpolation), and the center completely black scratch
図6(B)は本発明の一実施例であり、遮光時のOB補正前の完全黒キズを周囲8画素の中央値で補間する模式図(標準撮像信号での完全黒キズの周囲黒キズを含む正常画素での補間)であり、中央の完全黒キズ撮像信号0が、周囲の完全黒キズ撮像信号0を除く周囲の周囲正常画素撮像信号32,48.56,64.80,102,128の中央値64で補間される。
FIG. 6B is an embodiment of the present invention, a schematic diagram in which a perfect black defect before OB correction at the time of light shielding is interpolated by the median of eight pixels around (a surrounding black defect of a perfect black defect in a standard imaging signal Interpolation at normal pixels), and the center completely black scratched
図6(C)は本発明の一実施例であり、遮光時のOB補正前の撮像信号での白キズを周囲8画素の中央値で補間する模式図(標準撮像信号での白キズの周囲正常画素での補間)であり、中央の白キズ撮像信号1024が、周囲の周囲正常画素撮像信号32,48.56,64.80,96,102,128の中央値64又は80で補間される。
FIG. 6C is an embodiment of the present invention, and is a schematic diagram in which white defects in an image pickup signal before OB correction at the time of light shielding are interpolated with a median of 8 pixels around (white defects in a standard image pickup signal Interpolation at normal pixels), and the center white scratch
図6(D)は本発明の一実施例であり、遮光時のOB補正前の撮像信号での白キズを周囲8画素の中央値で補間する模式図(標準撮像信号での白キズの周囲白キズを含む正常画素での補間)であり、中央の白キズ撮像信号1024が、周囲の周囲正常画素撮像信号32,48.56,80,96,102,128の中央値80で補間される。
ここで、完全黒キズ補間は白キズ補間後に行う。FIG. 6D is an embodiment of the present invention, and is a schematic diagram in which white flaws in an image pickup signal before OB correction at the time of light shielding are interpolated with the median of eight surrounding pixels (around white flaws in a standard image pickup signal Interpolation at a normal pixel including white spots), and the white
Here, complete black defect interpolation is performed after white defect interpolation.
図7と図8は本発明の一実施例の周囲8画素の中央値を算出し、補間する詳細構成図である。
図7と図8において、9は周囲画素信号選択部であり、16は白キズの周囲画素での補間、18は黒キズの周囲画素での補間、19は周囲画素中央値検出部、21,22,23,24,25,26,27,28は比較部、29は映像信号切替器である。FIG. 7 and FIG. 8 are detailed configuration diagrams for calculating and interpolating the median value of eight surrounding pixels according to an embodiment of the present invention.
In FIG. 7 and FIG. 8, 9 is a surrounding pixel signal selection part, 16 is an interpolation in the surrounding pixel of a white defect, 18 is an interpolation in a surrounding pixel of a black defect, 19 is a surrounding pixel center value detecting part, 21,
図7と図8の周囲画素信号選択部9において、5と6はラインメモリ、7は映像信号切替器、8は遅延器である。ラインメモリ5と6は0Hの撮像信号から1H,2Hの撮像信号を生成し、0Hの白キズ補間信号から1H2Hの白キズ補間信号を生成する。
映像信号切替器7は、0H,1H,2Hの撮像信号を選択して撮像信号の周囲画素信号を生成し、0H,1H,2Hの白キズ補間信号を選択して白キズ補間信号の周囲画素信号を生成する。ラインメモリ5と6の替わりに図示しないフレームメモリでも良い。In the surrounding pixel
The video
図7と図8において、遅延器8は周囲画素中央値検出部19の遅延分撮像信号を遅延させ、白キズ補間信号を遅延させる。そして、周囲画素中央値検出部19は、撮像信号の周囲画素信号から21〜28の比較部で撮像信号または白キズ補間信号の周囲画素中央値を検出し、白キズ補間信号の周囲画素信号から21〜28の比較部で白キズ補間信号の周囲画素中央値を検出する。
In FIG. 7 and FIG. 8, the
そして、図4の白キズの映像信号タイミング(アドレス)判定と標準撮像時の白キズ補間判定15からの白キズのレベル以下判定時の白キズの映像信号タイミング(アドレス)に応じて白キズの周囲画素での補間部の切替器29で、撮像信号を撮像信号の周囲画素中央値に補間する。また、図4の黒キズの映像信号タイミング(アドレス)判定17からの完全黒キズの映像信号タイミング(アドレス)に応じて完全黒キズの周囲画素での補間部の切替器29で、白キズ補間信号を白キズ補間信号の周囲画素中央値に補間する。
Then, the white scratch image signal timing (address) of FIG. 4 and the white scratch level below the white scratch level from the white scratch interpolation determination 15 at standard imaging and the white scratch image signal timing (address) determination in FIG. The image pickup signal is interpolated to the peripheral pixel center value of the image pickup signal by the
図7と図8において、周囲画素に複数の白キズが存在していたとしても、白キズ補間後に完全黒キズ補間するので、完全黒キズ補間は、白キズの影響を受けない。
また、図7と図8において、白キズ補間後に完全黒キズ補間するので、(撮像信号の周囲画素信号及び撮像信号、又は白キズ補間信号の周囲画素信号及び白キズ補間信号の様に)入力信号を変えて、白キズのレベル以下判定時の白キズの映像信号タイミング(アドレス)、又は完全黒キズの映像信号タイミング(アドレス)の様に)制御信号を変えることにより、同一回路で白キズ補間と完全黒キズ補間と2回処理することで、白キズ補間と完全黒キズ補間とを同一回路で行うことが可能であり、回路の小型化と低価格化が実現できる。In FIG. 7 and FIG. 8, even if there are a plurality of white flaws in surrounding pixels, complete black flaw interpolation is performed after white flaw interpolation, so complete black flaw interpolation is not affected by white flaws.
Further, in FIGS. 7 and 8, since perfect black scale interpolation is performed after white scale interpolation, input (like peripheral pixel signal and image pickup signal of imaging signal or surrounding pixel signal of white scale interpolation signal and white scale interpolation signal) White flaws in the same circuit by changing the signal and changing the control signal of the white scratch video signal timing (address) or white black scratch video signal timing (address) when the white scratch level is determined or less By performing the interpolation and the complete black defect interpolation twice, it is possible to perform the white defect interpolation and the complete black defect interpolation in the same circuit, and it is possible to realize the miniaturization and the cost reduction of the circuit.
図9は本発明の一実施例の白キズ検出と感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する)の検出とのフローチャートであり、図10は発明の一実施例の白キズ補間と感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する)の補間とのフローチャートである。 FIG. 9 is a flowchart of white blemish detection according to an embodiment of the present invention and detection of an imaging pixel (hereinafter, referred to as a perfect black blemish) in which the sensitivity is abnormally low and the dark current is only a leakage current. 3 is a flowchart of white defect interpolation of one embodiment and interpolation of an imaging pixel (hereinafter, referred to as complete black defect) in which the sensitivity is abnormally low and the dark current is only a leakage current level.
図9において、CPU39は、白キズ検出と感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する)の検出との開始後に、およそ1秒遮光蓄積し読出を行い(S901)、画素の映像信号は白キズレベルを超えているかを判定(S902)し、“NO”なら(S904)に進み、“YES”なら白キズの映像タイミング(アドレス)を記憶(S903)して(S904)に進む。
In FIG. 9, the
次に、CPU39は、撮像素子の温度上昇6℃で2倍の正常画素の暗電流に反比例させて長時間(1秒から163894秒)遮光蓄積し読出を行い(S904)、画素の映像信号は白キズの映像タイミング(アドレス)かを判定(S905)し、“NO”なら(S908)に進み、“YES”なら周囲8画素の中央値を算出(S906)し、周囲8画素の中央値で補間(S907)して(S908)に進む。
次に、CPU39は、画素の映像信号は黒キズレベルを下回っているかを判定(S908)し、“NO”なら終了に進み、“YES”なら黒キズの映像タイミング(アドレス)を記憶(S909)し、終了に進む。
以上で、白キズと完全黒キズとの検出の処理が終了となる。Next, the
Next, the
This is the end of the process of detecting white flaws and completely black flaws.
図10において、CPU39は、白キズ補間と感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する)の補間の開始後に、標準撮像で読出を行い(S1001)、画素の映像信号は白キズ補間のレベルを以下かを判定(S1002)し、“NO”なら(S1006)に進み、“YES”なら白キズの映像タイミング(アドレス)かを判定(S1003)し、“NO”なら(S1006)に進み、“YES”なら周囲8画素の中央値を算出(S1004)し、周囲8画素の中央値で補間(S1005)して(S1006)に進む。
次に、CPU39は、画素の映像信号は黒キズの映像タイミング(アドレス)かを判定(S1006)し、“NO”なら終了に進み、“YES”なら周囲8画素の中央値を算出(S1007)し、周囲8画素の中央値で補間(S1008)して終了に進む。
以上で、白キズと完全黒キズとの検出の補間が終了となる。In FIG. 10, the
Next, the
Above, interpolation of detection of a white flaw and a complete black flaw ends.
つまり、実施例1の撮像装置において、垂直走査周期のおよそ15倍以上(該検出した撮像素子温度に対応する(6℃上昇でおよそ2倍となる)正常画素の暗電流に蓄積時間を逆比例させて垂直走査周期のおよそ15倍からおよそ245760倍で垂直走査周期約60Hzとして0.25秒から4096秒の可変時間又は垂直走査周期のおよそ60倍で垂直走査周期約60Hzとしておよそ1秒の固定時間)の中時間の遮光蓄積時間で暗電流が異常に多い画素欠陥(以下、白キズと称する)の検出を行う手段と、(撮像素子温度が6℃上昇でおよそ2倍となる)正常画素の暗電流を検出する手段(OB画素の最小値からNからN+M番目の平均値を算出する手段)または撮像素子の温度を検出する手段(温度センサ)と、該検出した撮像素子温度が6℃上昇でおよそ2倍となる正常画素の暗電流に蓄積時間を逆比例させて(およそ1秒から16384秒)蓄積して(ランダムノイズの影響を除去して、)OB補正前に(又はOB補正を中止して、)白キズ補間を行う手段と、白キズ補間後に第二の所定のレベルより小さい暗電流の画素(感度が異常に低い撮像画素(以下、完全黒キズと称する)を(正常画素の暗電流と識別して)検出する手段と、撮影時に完全黒キズごとに独立に周囲画素の信号レベルを用いて補間する手段と、完全黒キズ補間後にOB補正を行う手段と、を有することを特徴とする撮像装置である。 That is, in the image pickup apparatus according to the first embodiment, the accumulation time is inversely proportional to the dark current of a normal pixel which is about 15 times or more of the vertical scanning cycle (corresponding to the detected image sensor temperature (approx. If the vertical scanning period is about 60 times, the vertical scanning period is about 60 times that of the vertical scanning period when the vertical scanning period is about 60 Hz, or if the vertical scanning period is about 60 Hz when the vertical scanning period is about 60 Hz A means for detecting a pixel defect (hereinafter referred to as a white blemish) in which dark current is abnormally high at light-blocking accumulation time in the middle of time), and normal pixels (the image sensor temperature increases approximately Means for detecting the dark current (mean for calculating the N + Mth average value from N from the minimum value of OB pixels) or means for detecting the temperature of the imaging element (temperature sensor), and the detected imaging element The accumulation time is inversely proportional (approximately 1 second to 16384 seconds) to the dark current of a normal pixel, which is approximately doubled at a 6 ° C. rise (before removing the influence of random noise) before OB correction A means for performing white defect interpolation (or stopping OB correction), and a pixel of dark current smaller than a second predetermined level after white defect interpolation (image pickup pixel having an abnormally low sensitivity (hereinafter referred to as complete black defect) Means for detecting the dark current of a normal pixel), means for interpolating using signal levels of surrounding pixels independently for every completely black flaw during photographing, and means for performing OB correction after completely black scratch interpolation And an image pickup apparatus characterized by comprising:
また、上記撮像装置において、白キズ検出の蓄積での白キズ基準レベル又は標準撮像時の白キズ補間レベルと、判定レベルを入れ替えて、白キズの映像信号タイミング(アドレス)判定と標準撮像時の白キズ補間判定とを同一手段で行う手段とを有することを特徴とする撮像装置である。 Further, in the above imaging apparatus, the white defect reference level at the accumulation of white defect detection or the white defect interpolation level at standard imaging and the determination level are interchanged, and the white defect video signal timing (address) determination and standard imaging are performed. Means for performing the white defect interpolation determination by the same means.
また、上記撮像装置において、白キズ補間後に完全黒キズ補間するので、(撮像信号の周囲画素信号及び撮像信号、又は白キズ補間信号の周囲画素信号及び白キズ補間信号の様に)入力信号を変えて、白キズのレベル以下判定時の白キズの映像信号タイミング(アドレス)、又は完全黒キズの映像信号タイミング(アドレス)の様に)制御信号を変えることにより、同一回路で白キズ補間と完全黒キズ補間と2回処理することで、白キズ補間と完全黒キズ補間とを同一回路で行う手段とを有することを特徴とする撮像装置である。 Further, in the above imaging apparatus, since perfect black scale interpolation is performed after white scale interpolation, (like peripheral pixel signals and image pickup signals of an image pickup signal or surrounding pixel signals of white scale interpolation signal and white scale interpolation signal) By changing the control signal of the white scratch video signal timing (address) at the time of determination below the white scratch level or the video signal timing (address) of a completely black scratch, the white scratch interpolation and the same circuit can be performed. An imaging apparatus is characterized by including means for performing white defect interpolation and complete black defect interpolation in the same circuit by performing complete black defect interpolation and processing twice.
(実施例3) 図12Aは、本発明の一実施例のOB補正52の一例であり、OB画素信号の代表値検出部の一例のOB画素信号の最小値からN(4)番目からN(4)+M(3)番目の平均検出部48と減算器4からなる。
図12Bは、本発明の一実施例のOB補正の一例のOB画素信号の代表値検出部の一例となるOB画素信号の最小値からN(2)番目からN(2)+M(1)番目の平均検出部とOB補正との、暗電流算出OB補正である。Example 3 FIG. 12A is an example of the
FIG. 12B shows N (2) th to N (2) + M (1) th from the minimum value of the OB pixel signal as an example of the representative value detection unit of the OB pixel signal according to an example of the OB correction of the embodiment of the present invention. Dark current calculation OB correction of the average detection unit and the OB correction.
図12Cは、本発明の一実施例のOB補正の一例のOB画素信号の代表値検出部の一例となるH−OB画素信号の最小値からN(4)番目からN(4)+M(3)番目の平均検出部とH−OB補正との、暗電流算出OB補正である。
図12Dは、本発明の一実施例のOB補正の一例のOB画素信号の代表値検出部の一例となるOB画素信号の最小値からN(4)番目からN(4)+M(3)番目の平均検出部とV−OB補正との、暗電流算出OB補正である。FIG. 12C shows N (4) th to N (4) + M (3) from the minimum value of the H-OB pixel signal as an example of the representative value detection unit of the OB pixel signal according to an example of the OB correction according to an embodiment of the present invention. Dark current calculation OB correction of the second average detection unit and the H-OB correction.
FIG. 12D shows N (4) th to N (4) + M (3) th from the minimum value of the OB pixel signal as an example of the representative value detector of the OB pixel signal according to an example of the OB correction according to the embodiment of the present invention. Dark current calculation OB correction of the average detection unit and the V-OB correction.
図12Aから図12DのV−OB補正において、遅延部55は必須ではないが、有効画素信号を遅延部55で遅延させ、有効画素後のV-OBで補正の方が安定する。
V−OB補正は縦筋補正やH Shading補正も兼ねる方が小型の信号処理回路で黒が安定した映像信号が出力でき、テレビカメラのワイドダイナミックレンジ化が容易になる。
図12Aから図12Dにおいて、OB画素信号の最小値からN(4)番目からN(4)+M(3)番目の平均検出部48は、21,22,23,24,25,26,27,28との比較部と、43,44,45との加算器と、46の1/4(2bitシフト)、47は最小値から4から4+3番目検出部からなる。In the V-OB correction of FIGS. 12A to 12D, although the
If V-OB correction is also used for vertical streak correction and H Shading correction, a compact signal processing circuit can output a video signal with stable black, and it becomes easy to achieve wide dynamic range of a television camera.
12A to 12D, the N (4) th to N (4) + M (3) th
図12Aと図12Cと図12Dにおいて、OB画素信号の最小値からN(4)番目からN(4)+M(3)番目の平均検出部47は、白キズと完全黒キズを除外するために、OB画素信号から8個の比較部で最小値からN(4)番目からN(4)+M(3)番目を検出し、加算し2bitシフトして1/4化することにより白キズと完全黒キズとを除外したOB画素信号を加算平均する。
12A, 12C, and 12D, the N (4) th to N (4) + M (3) th
図12Bにおいて、OB画素信号の最小値からN(2)番目からN(2)+M(1)番目の平均検出部47は、白キズと完全黒キズを除外するために、OB画素信号から8個の比較部で最小値からN(2)番目からN(2)+M(1)番目を検出し、加算し1bitシフトして1/2化することにより白キズと完全黒キズとを除外したOB画素信号を加算平均する。
In FIG. 12B, the N (2) -th to N (2) + M (1) -th
テレビカメラの撮像素子は、2K,4K,8Kと高画素になるに従い、有効画素に比較してOB画素が少なくなる。そのため、OB画素の白キズと完全黒キズの影響を受けやすくなる。
しかし、本発明では、OB画素の白キズと完全黒キズの影響を受けないで、撮像素子の温度が6℃上昇すると約2倍に増加するOB画素信号の正常画素の暗電流を検出することができる。その結果、検出した(撮像素子温度に6℃でおよそ2倍となる)正常画素の暗電流に蓄積時間を逆比例させることができる。
また、検出したOB画素信号の正常画素の暗電流の信号を撮像有効画素信号から減算すれば、OB画素の白キズと完全黒キズの影響を受けないで、映像信号処理で安定にOB補正をすることができ、映像信号の黒が安定し、テレビカメラのワイドダイナミックレンジ化が容易になる。As the image pickup element of the television camera becomes as high as 2K, 4K, and 8K, the number of OB pixels decreases as compared with the effective pixels. Therefore, it becomes susceptible to the influence of the white flaws and the complete black flaws of the OB pixel.
However, in the present invention, the dark current of the normal pixel of the OB pixel signal that increases about twice when the temperature of the image pickup device rises by 6 ° C. is detected without being affected by the white flaws and the complete black flaws of the OB pixels. Can. As a result, it is possible to make the accumulation time inversely proportional to the dark current of the detected normal pixel (which approximately doubles at 6 ° C. with the image sensor temperature).
Also, if the dark current signal of the normal pixel of the detected OB pixel signal is subtracted from the imaging effective pixel signal, the OB correction can be stably performed by the video signal processing without being affected by the white flaw and the complete black flaw of the OB pixel. The black of the video signal is stabilized, and the wide dynamic range of the television camera is facilitated.
撮像素子の温度が6℃上昇すると約2倍に増加するOB画素信号の代表値を検出することにより、温度センサを別途設けなくても、撮像素子の温度を検出することができる。
検出した(撮像素子温度が6℃上昇でおよそ2倍となる)正常画素の暗電流に蓄積時間を逆比例させることにより、正常画素の暗電流が少ない低温時の正常画素の暗電流と、漏れ電流相当の暗電流しかない異常低感度画素(完全黒キズ)の暗電流と確実に識別し、完全黒キズを確実に検出する。
白キズ検出と異なり、感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する)信号検出は困難なので、低温度における検出時の長時間蓄積が許容される。By detecting the representative value of the OB pixel signal that increases approximately twice when the temperature of the imaging device rises by 6 ° C., it is possible to detect the temperature of the imaging device without separately providing a temperature sensor.
By making the accumulation time inversely proportional to the dark current of a normal pixel that has been detected (the image sensor temperature increases by approximately 6 ° C.), the dark current of a normal pixel at low temperature with a small dark current of the normal pixel The dark current of an abnormally low sensitivity pixel (complete black flaw) having only a dark current equivalent to the current is reliably identified, and a complete black flaw is reliably detected.
Unlike white blemish detection, it is difficult to detect an imaging pixel (hereinafter referred to as complete black blemish) signal whose sensitivity is abnormally low and dark current is only about a leakage current, so long-term accumulation at detection at low temperature is allowed.
つまり、実施例1の撮像装置において、映像信号処理の中時間(およそ1秒の固定時間又は該検出した撮像素子温度が6℃上昇でおよそ2倍となる正常画素の暗電流に蓄積時間を逆比例させておよそ0.25秒から4096秒可変時間)の遮光蓄積時間で暗電流が異常に多い画素欠陥(以下、白キズと称する)の検出を行う手段と、(撮像素子温度が6℃上昇でおよそ2倍となる)正常画素の暗電流を検出する手段(OB画素の最小値からNからN+M番目の平均値を算出する手段)または撮像素子の温度を検出する手段(温度センサ)と、該検出した撮像素子温度が6℃上昇でおよそ2倍となる正常画素の暗電流に蓄積時間を逆比例させて(およそ1秒から16384秒)蓄積して(ランダムノイズの影響を除去して、)OB補正前に(又はOB補正を中止して、)白キズ補間を行う手段と、白キズ補間後に第二の所定のレベルより小さい暗電流の画素(感度が異常に低い撮像画素(以下、完全黒キズと称する))を(正常画素の暗電流と識別して)検出する手段と、撮影時に完全黒キズごとに独立に周囲画素の信号レベルを用いて補間する手段と、完全黒キズ補間後にOB補正を行う手段と、を有することを特徴とする撮像装置である。 That is, in the image pickup apparatus of the first embodiment, the accumulation time is reversed to the dark current of the normal pixel in which the fixed time of about 1 second (the fixed time of about 1 second or the detected image pickup device temperature rises about A means for detecting a pixel defect (hereinafter referred to as a white defect) having an abnormally large amount of dark current in a light-blocking accumulation time in proportion to about 0.25 seconds to 4096 seconds variable time) (image sensor temperature rises by 6 ° C. Means for detecting dark current of normal pixels (mean for calculating an average value from N to N + Mth from the minimum value of OB pixels) or means for detecting the temperature of the imaging device (temperature sensor); The accumulation time is reversely proportional to the dark current of a normal pixel in which the detected image pickup device temperature is approximately doubled at a rise of 6 ° C. (approximately 1 second to 16384 seconds), and the influence of random noise is removed ) Before OB correction (also A means for performing white defect interpolation by stopping OB correction, and a pixel of dark current smaller than the second predetermined level after white defect interpolation (image pickup pixel having an abnormally low sensitivity (hereinafter referred to as complete black defect)) (Detects the dark current of a normal pixel), interpolates using signal levels of surrounding pixels independently for every completely black flaw at the time of photographing, and performs OB correction after completely black flaw interpolation And an image pickup apparatus characterized by comprising:
また、実施例2と実施例3の組合せは、上記撮像装置において、映像信号処理の中時間(およそ1秒の固定時間又は該検出した撮像素子温度が6℃上昇でおよそ2倍となる正常画素の暗電流に蓄積時間を逆比例させておよそ0.25秒から4096秒可変時間)の遮光蓄積時間で白キズ検出を行う手段と、該白キズ検出後で、白上記撮像素子温度に蓄積時間を逆比例させておよそ4秒から4096秒蓄積して(ランダムノイズの影響を除去して、)OB補正前又はOB補正を中止して、白キズ補間を行い、第二の所定のレベルより小さい暗電流の画素(感度が異常に低い撮像画素(以下、完全黒キズと称する))を(正常画素の暗電流と識別して)検出する手段と、撮影時に完全黒キズごとに独立に周囲画素の信号レベルを用いて補間する手段とを有することを特徴とする撮像装置である。 In the above-described imaging device, the combination of the second embodiment and the third embodiment is the normal time during which the image signal processing is performed during the middle time (fixed time of approximately 1 second or the detected imaging element temperature is approximately doubled by A means for performing white flaw detection in a light-blocking accumulation time (approximately 0.25 seconds to 4096 seconds variable time) with the accumulation time being inversely proportional to the dark current, and the white accumulation sensor temperature after the white flaw detection. Is inversely proportional and accumulated for about 4 seconds to 4096 seconds (with removing the influence of random noise) before OB correction or before OB correction to perform white defect interpolation, smaller than the second predetermined level Means for detecting pixels of dark current (image pickup pixels with extremely low sensitivity (hereinafter referred to as complete black blemishes) (identified as dark current of normal pixels) and surrounding pixels independently for every completely black blemish at the time of shooting Means to interpolate using the signal level of And an image pickup apparatus characterized in that
以上は、図1AのR,G,Bの3撮像素子を用いたカラーカメラを中心に説明したが、図1BのR,G1,G2,Bの4板式撮像素子を用いたカラーカメラでも、図1Cのオンチップカラーフィルタの撮像素子を用いたカラーカメラでも、図1Dのオンチップカラーフィルタのない撮像素子を用いたモノクロカメラでも、映像信号処理でOB補正をしていれば、構わない。 The above description has been focused on a color camera using three R, G, and B imaging devices in FIG. 1A, but a color camera using four, R, G1, G2, and B four-plate type imaging devices in FIG. It does not matter as long as OB correction is performed by image signal processing, whether it is a color camera using an imaging element of the 1C on-chip color filter or a monochrome camera using an imaging element without the on-chip color filter in FIG.
また、白キズ検出の中時間蓄積での白キズ基準レベル又は標準撮像時の白キズ補間レベルと、判定レベルを入れ替えて、白キズの映像信号タイミング(アドレス)判定と標準撮像時の白キズ補間判定とを同一手段で行うことが可能であり、回路の小型化と低価格化が実現できる。
さらに、入力信号を変えて、白キズのレベル以下判定時の白キズの映像信号タイミング(アドレス)、又は完全黒キズの映像信号タイミング(アドレス)の様に)制御信号を変えて同一回路で2回処理して、白キズ補間と完全黒キズ補間とを同一手段で行うことが可能であり、回路の小型化と低価格化が実現できる。In addition, the white defect reference level during medium time accumulation of white defect detection or the white defect interpolation level at the time of standard imaging and the determination level are interchanged, and the white scratch image signal timing (address) determination at white defects and the white defect interpolation at the standard imaging The determination can be performed by the same means, and the circuit can be miniaturized and the cost can be reduced.
Furthermore, change the input signal, change the video signal timing (address) of white flaws at the time of determination below the level of white flaws, or the video signal timing (address) of completely black flaws) It is possible to perform the round process and perform the white defect interpolation and the complete black defect interpolation by the same means, and it is possible to realize the miniaturization and the cost reduction of the circuit.
図7と図8において、周囲画素に複数の白キズが存在していたとしても、白キズ補間後に完全黒キズ補間するので、完全黒キズ補間は、白キズの影響を受けない。
また、白キズ補間後完全黒キズ補間後に映像信号処理で遮光画素信号(以下、OBと称する)補正するので、OB補正は白キズと完全黒キズの影響を受けないで安定に動作するので、映像信号の黒が安定し、テレビカメラのワイドダイナミックレンジ化が容易になる。
テレビカメラの撮像素子は、2K(1920H×1080V),4K(3840H×2160V),8K(7680H×4320V)と高画素になるに従い、有効画素に比較してOB画素が少なくなる。そのため、OB画素の白キズの影響を受けやすくなる。しかし、本発明では、OB画素の白キズの影響を受けないで、撮像素子の温度が6℃上昇すると約2倍に増加するOB画素信号の代表値を検出することができる。
従来は、白キズ検出と異なり、感度が異常に低く暗電流が漏れ電流程度しかない撮像画素(以下、完全黒キズと称する)信号検出と補間との自動化は困難で手動補間を行っていた。
本発明によれば、完全黒キズの検出と補間とが自動で可能で、CCD撮像素子に比べ、高画素化と高速読出しとが容易で安価なMOS撮像素子の完全黒キズが許容できない放送用カメラや高い信頼性が要求される(原子力発電所や新幹線等の)監視用カメラや(自動車塗装や織物等の確認の)産業用カメラ等への適用が加速される。In FIG. 7 and FIG. 8, even if there are a plurality of white flaws in surrounding pixels, complete black flaw interpolation is performed after white flaw interpolation, so complete black flaw interpolation is not affected by white flaws.
In addition, since the light-shielded pixel signal (hereinafter referred to as OB) correction is performed in the video signal processing after the perfect black scratch interpolation after white scratch interpolation, the OB correction operates stably without being affected by the white scratch and the perfect black scratch. The black of the video signal is stabilized, and the wide dynamic range of the television camera can be easily made.
As the image pickup element of the television camera has high pixels such as 2K (1920H × 1080V), 4K (3840H × 2160V), and 8K (7680H × 4320V), the number of OB pixels is smaller than that of effective pixels. Therefore, it becomes easy to receive the influence of the white flaw of OB pixel. However, according to the present invention, it is possible to detect a representative value of the OB pixel signal, which increases about twice when the temperature of the image pickup device rises by 6 ° C., without being affected by white defects of the OB pixel.
Conventionally, unlike white blemish detection, automation of imaging pixel (hereinafter referred to as complete black blemish) signal detection and interpolation in which the sensitivity is abnormally low and dark current is only a leakage current is difficult, and manual interpolation has been performed.
According to the present invention, it is possible to automatically detect and interpolate completely black spots, and it is easy to increase the number of pixels and to read at high speed compared to a CCD image sensor, and for broadcasts which can not tolerate complete black flaws of an inexpensive MOS image sensor. Applications to cameras, surveillance cameras (such as nuclear power plants and Shinkansen), and industrial cameras (such as confirmation of automobile painting and textiles) that require high reliability are accelerated.
本発明の実施形態である撮像装置は、感度が異常に低く暗電流が漏れ電流程度しかない撮像画素信号も自動検出でき、かつ完全黒キズの周囲画素を用いた補間が可能となる。 The imaging device according to the embodiment of the present invention can automatically detect an imaging pixel signal whose sensitivity is abnormally low and the dark current is only a leakage current, and interpolation using surrounding pixels of completely black spots becomes possible.
以上、本発明の一実施形態について詳細に説明したが、本発明は上述した実施形態に限定されるものではなく、本発明の趣旨を逸脱しない範囲で種々変更して実施することができる。 As mentioned above, although one embodiment of the present invention was described in detail, the present invention is not limited to the embodiment mentioned above, and can be variously changed and carried out in the range which does not deviate from the meaning of the present invention.
撮像素子温度に蓄積(露光)時間を逆比例して長時間蓄積することによって、完全黒キズを検出し、周囲画素で補間して高画質の映像信号を生成する用途の放送用カメラや高い信頼性が要求される(原子力発電所や新幹線等の)監視用カメラや(自動車の車体塗装や織物等の確認の)産業用カメラ等に適用できる。この出願は、2016年8月25日に出願された日本出願特願2016−164619を基礎として優先権の利益を主張するものであり、その開示の全てを引用によってここに取り込む。 A broadcast camera or high reliability that is used for detecting perfect black spots and interpolating with surrounding pixels to generate high-quality video signals by accumulating for a long time in inverse proportion to the accumulation (exposure) time to the image sensor temperature It can be applied to surveillance cameras (such as nuclear power stations and Shinkansen) and industrial cameras (for checking automobile body painting, textiles, etc.). This application claims the benefit of priority based on Japanese Patent Application No. 2016-164619 filed on August 25, 2016, the entire disclosure of which is incorporated herein by reference.
4:減算器、5,6:ラインメモリ、7:映像信号切替器、8:遅延器、9:周囲画素信号選択部、12,13,14:減算器、15:白キズの映像信号タイミング(アドレス)判定と標準撮像時の白キズ補間判定、16:白キズの周囲画素での補間、17:完全黒キズの映像信号タイミング(アドレス)判定、18:完全黒キズの周囲画素での補間、19:周囲画素中央値検出部、20:温度センサ、21,22,23,24,25,26,27,28:比較部、29:映像信号切替器、30:テレビジョンカメラ、31:レンズ、32:プリズム、33R,33G,33G1,33G2,33B:CMOS撮像素子、34:白キズ補間、35:白キズ完全黒キズ検出補間機能付映像信号処理部、36:MATRIX部、37:パラレル−シリアル変換部(P/S)、38:白キズ完全黒キズ検出補間部、39:CPU、40:ビューファインダ、41:白キズ検出補間機能付映像信号処理部、42:白キズ検出補間部、43,44,45:加算器、46:1/4(2bitシフト)、47:最小値から4から4+3番目検出部、48:OB画素信号の最小値からN(4)番目からN(4)+M(3)番目の平均検出部、49:OB補正、50:白キズ検出補間、51:完全黒キズ検出補間、52:暗電流算出OB補正、53:ガンマ色輪郭補正、54:撮像素子制御、55:遅延部、56:1/2(1bitシフト)、57:パラレル−シリアル変換部(P/S)、58:オンチップカラーフィルタ付撮像素子、59:モノクロ撮像素子、60:ガンマ輪郭補正、61:ラインメモリ、71:ライン加算平均部。 4: Subtractor 5, 6: Line memory 7: 7: Video signal switching device 8: Delay device 9: Surrounding pixel signal selection unit 12, 13, 14: Subtractor 15: Video signal timing of white flaws ( Address) Judgment and white scratch interpolation judgment at standard imaging, 16: Interpolation in surrounding pixels of white flaw, 17: Video signal timing (address) judgment of complete black flaw, 18: Interpolation in surrounding pixel of complete black flaw, 19: ambient pixel median value detection unit, 20: temperature sensor, 21, 22, 23, 24, 25, 26, 27, 28: comparison unit, 29: video signal switcher, 30: television camera, 31: lens, 32: Prism, 33R, 33G, 33G1, 33G2, 33B: CMOS image pickup device, 34: White scratch interpolation, 35: White scratch complete black scratch detection Video signal processing unit with interpolation function, 36: MATRIX section, 37: Parallel- Real conversion unit (P / S) 38: white flaws completely black flaw detection interpolation part 39: CPU 40: view finder 41: video signal processing part with white flaw detection interpolation function 42: white flaw detection interpolation part 43, 44, 45: Adder, 46: 1/4 (2-bit shift), 47: From the minimum value to 4 to 4 + 3rd detection unit, 48: From the minimum value of the OB pixel signal to N (4) to N (4) + M (3) th average detection unit, 49: OB correction, 50: white flaw detection interpolation, 51: complete black flaw detection interpolation, 52: dark current calculation OB correction, 53: gamma color contour correction, 54: image sensor control , 55: delay unit, 56: 1/2 (1 bit shift), 57: parallel-serial conversion unit (P / S), 58: imaging device with on-chip color filter, 59: monochrome imaging device, 60: gamma contour correction , 61: line memory, 71: la Emissions averaging section.
Claims (3)
正常画素の暗電流を検出する手段(OB画素の最小値からN番目からN+M番目平均値を算出する手段)または撮像素子の温度を検出する手段を有し、該検出した撮像素子温度に対応する正常画素の暗電流に蓄積時間を逆比例させて遮光蓄積する手段と、
映像信号処理のOB補正前に、第二の所定のレベルより小さい暗電流の画素(以下、完全黒キズと称する)を前記正常画素の暗電流と識別して検出する手段と、
撮影時に第二の所定のレベルより小さい暗電流の完全黒キズの画素ごとに独立に周囲画素の信号レベルを用いて補間する手段と、を有することを特徴とする撮像装置。Interpolation using an image sensor, means for detecting dark current at the time of light shielding for each pixel of the image signal of the image sensor, and signal levels of surrounding pixels independently for each pixel of dark current larger than the first predetermined level And an image signal processing means for correcting a light-shielded pixel signal (hereinafter referred to as OB) for subtracting a representative value of the light-shielded pixel video signal from the effective pixel video signal in the video signal processing.
Means for detecting dark current of normal pixels (means for calculating Nth to N + Mth average values from the minimum value of OB pixels) or means for detecting the temperature of the imaging device, corresponding to the detected imaging device temperature A means for making the accumulation time inversely proportional to the dark current of the normal pixel and performing light-shielded accumulation;
A means for identifying and detecting a dark current pixel smaller than a second predetermined level (hereinafter referred to as a perfect black flaw) from the dark current of the normal pixel before OB correction of the video signal processing;
An imaging device characterized by comprising means for interpolating independently using signal levels of surrounding pixels independently for every completely black flaw pixel of dark current smaller than a second predetermined level at the time of photographing.
垂直走査周期に基づく前記検出した撮像素子温度に対応する正常画素の暗電流に蓄積時間を逆比例させて垂直走査周期の任意の倍数の範囲で垂直走査周期の所定の可変時間又は垂直走査周期の所定の遮光蓄積時間で暗電流が所定以上の画素欠陥(以下、白キズと称する)の検出を行う手段と、
上記検出した撮像素子温度に対応する正常画素の暗電流に蓄積時間を逆比例させて遮光蓄積してOB補正前に白キズ補間を行う手段と、
白キズ補間後に前記完全黒キズを前記正常画素の暗電流と識別して検出する手段と、
撮影時に前記完全黒キズごとに独立に周囲画素の信号レベルを用いて補間する手段と、
完全黒キズ補間後にOB補正を行う手段と、を有することを特徴とする撮像装置。In the imaging device of claim 1,
The dark current of the normal pixel corresponding to the detected image sensor temperature based on the vertical scanning cycle makes the storage time inversely proportional to a predetermined variable time or vertical scanning cycle of the vertical scanning cycle within the range of any multiple of the vertical scanning cycle A means for detecting a pixel defect (hereinafter referred to as a white defect) having a dark current of a predetermined value or more in a predetermined light shielding accumulation time;
A means for performing the white defect interpolation before the OB correction by making the accumulation time inversely proportional to the dark current of the normal pixel corresponding to the detected image pickup device temperature and performing the shading correction before the OB correction;
A means for discriminating and detecting the completely black flaw from the dark current of the normal pixel after white flaw interpolation;
A means for performing interpolation using signal levels of surrounding pixels independently for each of the completely black flaws at the time of photographing;
And a means for performing OB correction after perfect black scratch interpolation.
撮像装置を撮像素子温度が所定以上の温度の状態時に、所定時間範囲にて遮光蓄積し、
映像信号処理のOB補正前に、第二の所定のレベルより小さい暗電流の画素(以下、完全黒キズと称する)を正常画素の暗電流と識別して検出し、
撮影時に前記完全黒キズの画素ごとに独立に周囲画素の信号レベルを用いて補間することを特徴とする撮像装置の調整方法。Dark current at the time of light blocking for each pixel of the red, green, and blue primary color image signals of the color separation optical system, the three or more image sensors, or the color filter with R, G1, G2, B Bayer arrangement Means for detecting the dark current pixels, and means for interpolating independently using the signal levels of surrounding pixels for each pixel of dark current larger than the first predetermined level; And an image pickup apparatus having video signal processing means for correcting a light-shielded pixel signal (hereinafter referred to as OB) to be subtracted from the video signal.
When the image pickup device temperature is a predetermined temperature or more, the image pickup device is light-shielded and accumulated in a predetermined time range,
Before OB correction of video signal processing, a pixel having a dark current smaller than a second predetermined level (hereinafter, referred to as a completely black flaw) is identified and detected as a dark current of a normal pixel,
A method of adjusting an image pickup apparatus, wherein interpolation is performed using signal levels of surrounding pixels independently for each of the completely black flaw pixels at the time of photographing.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2016164619 | 2016-08-25 | ||
JP2016164619 | 2016-08-25 | ||
PCT/JP2017/021624 WO2018037674A1 (en) | 2016-08-25 | 2017-06-12 | Imaging device, abnormal pixel detection and interpolation method and imaging device adjustment method |
Publications (2)
Publication Number | Publication Date |
---|---|
JPWO2018037674A1 true JPWO2018037674A1 (en) | 2019-06-20 |
JP6730436B2 JP6730436B2 (en) | 2020-07-29 |
Family
ID=61245738
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
JP2018535474A Active JP6730436B2 (en) | 2016-08-25 | 2017-06-12 | Imaging device and method of adjusting imaging device |
Country Status (2)
Country | Link |
---|---|
JP (1) | JP6730436B2 (en) |
WO (1) | WO2018037674A1 (en) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2000358195A (en) * | 1999-06-15 | 2000-12-26 | Matsushita Electric Ind Co Ltd | Video signal processor |
JP2003244547A (en) * | 2002-02-13 | 2003-08-29 | Sharp Corp | Solid-state image pickup device, method for converting failed pixel for the apparatus and defect correcting method |
JP2016029759A (en) * | 2014-07-25 | 2016-03-03 | キヤノン株式会社 | Defective pixel detection method for imaging device |
-
2017
- 2017-06-12 WO PCT/JP2017/021624 patent/WO2018037674A1/en active Application Filing
- 2017-06-12 JP JP2018535474A patent/JP6730436B2/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2000358195A (en) * | 1999-06-15 | 2000-12-26 | Matsushita Electric Ind Co Ltd | Video signal processor |
JP2003244547A (en) * | 2002-02-13 | 2003-08-29 | Sharp Corp | Solid-state image pickup device, method for converting failed pixel for the apparatus and defect correcting method |
JP2016029759A (en) * | 2014-07-25 | 2016-03-03 | キヤノン株式会社 | Defective pixel detection method for imaging device |
Also Published As
Publication number | Publication date |
---|---|
WO2018037674A1 (en) | 2018-03-01 |
JP6730436B2 (en) | 2020-07-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9071781B2 (en) | Image capturing apparatus and defective pixel detection method | |
US7129975B2 (en) | Addressable imager with real time defect detection and substitution | |
US7952621B2 (en) | Imaging apparatus having temperature sensor within image sensor wherin apparatus outputs an image whose quality does not degrade if temperature increases within image sensor | |
JP5526014B2 (en) | Imaging device | |
JP5541718B2 (en) | Imaging device and defective pixel detection method thereof | |
JP5855035B2 (en) | Solid-state imaging device | |
JP2011029887A (en) | Image pickup apparatus and control method | |
JP2008109504A (en) | Imaging device and correcting method | |
JP2007019577A (en) | Solid-state image pickup device | |
JP2008306565A (en) | Imaging apparatus and signal correcting method thereof | |
JP6991809B2 (en) | Imaging device | |
JPWO2018142555A1 (en) | Imaging device | |
JP2008148063A (en) | Imaging apparatus | |
JP2013150144A (en) | Imaging method and imaging apparatus | |
JP6730436B2 (en) | Imaging device and method of adjusting imaging device | |
JP2018182366A (en) | Imaging device and detecting correcting method of dark current fluctuation and imaging device adjusting method | |
JP2011114473A (en) | Pixel defect correction device | |
JP2009105582A (en) | Noise correction circuit, imaging device and noise correction method | |
JP5195289B2 (en) | Imaging device | |
JP6358607B2 (en) | Image processing apparatus and image processing method | |
JP4745680B2 (en) | Imaging apparatus and imaging method | |
JP2010178384A (en) | Image signal processing circuit, camera and image signal processing method | |
JP2006108918A (en) | Imaging apparatus | |
JP2004023231A (en) | Imaging device and portable telephone system provided with the imaging device | |
JP2003348456A (en) | Digital still camera |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20190206 |
|
A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20191107 |
|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20191216 |
|
TRDD | Decision of grant or rejection written | ||
A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20200625 |
|
A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20200702 |
|
R150 | Certificate of patent or registration of utility model |
Ref document number: 6730436 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 |
|
R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |