WO2022064609A1 - 撮像装置 - Google Patents

撮像装置 Download PDF

Info

Publication number
WO2022064609A1
WO2022064609A1 PCT/JP2020/036085 JP2020036085W WO2022064609A1 WO 2022064609 A1 WO2022064609 A1 WO 2022064609A1 JP 2020036085 W JP2020036085 W JP 2020036085W WO 2022064609 A1 WO2022064609 A1 WO 2022064609A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixel
same color
gain
image
line
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/036085
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
翔 加藤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kokusai Denki Electric Inc
Original Assignee
Hitachi Kokusai Electric Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi Kokusai Electric Inc filed Critical Hitachi Kokusai Electric Inc
Priority to JP2022551503A priority Critical patent/JP7352037B2/ja
Priority to PCT/JP2020/036085 priority patent/WO2022064609A1/ja
Priority to US18/021,013 priority patent/US12413871B2/en
Publication of WO2022064609A1 publication Critical patent/WO2022064609A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/67—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response
    • H04N25/671—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction
    • H04N25/677—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction for reducing the column or line fixed pattern noise
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/12—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths with one sensor only
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/67—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/67—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response
    • H04N25/671—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/67—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response
    • H04N25/671—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction
    • H04N25/673—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction by using reference sources
    • H04N25/674—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction by using reference sources based on the scene itself, e.g. defocusing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors

Definitions

  • the present invention relates to an image pickup device that uses a CMOS image sensor for photographing.
  • CMOS image sensors for standard industrial use or monitoring use is used, and in order to obtain a gain (sensitivity gain) that exceeds the image sensor specifications, for example, FPGA (Field Programmable Gate Array). And ISP (Image Signal Processor), additional sensitivity is being improved in combination with noise reduction processing.
  • FPGA Field Programmable Gate Array
  • ISP Image Signal Processor
  • Patent Document 1 discloses a technique of specifying a position where line-shaped fixed pattern noise is generated and correcting each pixel constituting the fixed pattern noise based on pixels of the same color around the fixed pattern noise.
  • the present invention has been made in view of the above-mentioned conventional circumstances, and suppresses fixed pattern noise generated by individual differences of a CMOS image sensor when providing a gain exceeding the specifications of the CMOS image sensor. It is an object of the present invention to provide an image pickup apparatus capable of capable.
  • the image pickup apparatus is configured as follows. That is, in an image pickup device that uses a CMOS image sensor for imaging, a memory that stores a gain that exceeds the gain obtained by the CMOS image sensor and starts to generate line-shaped fixed pattern noise in the captured image as a correction reference value, and CMOS imaging. It is characterized by having an image processing unit that performs processing for suppressing fixed pattern noise on the captured image when the gain when providing a gain exceeding the gain obtained by the element exceeds the correction reference value. And.
  • the captured image includes a plurality of pixel lines, and in each of the plurality of pixel lines, each pixel in the pixel line is arranged according to one of the plurality of color arrangement patterns, and the image processing unit is used to display fixed pattern noise. Processing for suppressing fixed pattern noise for captured images when pixel lines are not continuous with the same color arrangement pattern, or when pixel lines of fixed pattern noise are continuous with the same color arrangement pattern and the number of consecutive pixels is two. May be applied.
  • the image processing unit uses the pixels of the pixel line of the fixed pattern noise as the same color pixels of the pixel lines of the same color scheme on both sides of the pixel line. It may be corrected based on.
  • the image processing unit sets each pixel of the left pixel line of the two pixel lines to the pixel line. Correction is made based on the same color pixels of the pixel line of the same color pattern on the left side, and each pixel of the right pixel line of the two pixel lines is the same color pixel of the pixel line of the same color pattern on the right side of the pixel line. It may be corrected based on.
  • the image processing unit may correct each pixel of the fixed pattern noise pixel line based on the same color pixel at the same position in the pixel line of the same color scheme.
  • an image pickup device capable of suppressing fixed pattern noise generated by individual differences of the CMOS image pickup device when providing a gain exceeding the specifications of the CMOS image pickup device.
  • FIG. 1 It is a figure which shows the structural example of the image pickup apparatus which concerns on one Embodiment of this invention. It is a figure which shows the example of the fixed pattern noise included in a photographed image. It is a figure which shows the waveform example of the image level of the photographed image which contains the fixed pattern noise. It is a figure which shows the correction example when the fixed pattern noise is not continuous in the same color arrangement. It is a figure which shows the correction example when the fixed pattern noise has two lines continuous in the same color arrangement. It is a figure which shows the flowchart example about the preprocessing for correcting a fixed pattern noise.
  • the CMOS image pickup element is used for photographing, and the gain is added by FPGA or ISP so that the gain exceeding the gain (sensitivity gain) obtained inside the CMOS image pickup element can be obtained. It has an additional function. Further, the image pickup apparatus according to the present invention has an FPN suppressing function of suppressing fixed pattern noise (FPN) generated in a captured image when a gain is added.
  • FPN fixed pattern noise
  • FIG. 1 shows a configuration example of an image pickup apparatus according to an embodiment of the present invention.
  • the image pickup device 1 of this example includes a CMOS image pickup element 11, an image processing unit 12, and a memory 13.
  • a gain that exceeds the gain obtained by the CMOS image sensor 11 and starts to generate line-shaped fixed pattern noise in the captured image is stored in advance as a correction reference value.
  • This correction reference value is set according to the measurement result in advance, and may differ depending on the individual difference of the CMOS image sensor 11.
  • the image processing unit 12 suppresses fixed pattern noise (FPN) with respect to the captured image after gain adjustment when the gain when providing the gain exceeding the gain obtained by the CMOS image sensor 11 exceeds the correction reference value. Line correction is automatically applied.
  • FPN fixed pattern noise
  • FIG. 2 shows an example of FPN included in the captured image after the gain is added.
  • a line-shaped FPN 22 extending in the vertical direction is generated in a part of the captured image 21.
  • FIG. 3 shows an example of a video-level waveform of a captured image including FPN.
  • the horizontal axis (H Line) indicates the horizontal position (line position) of the captured image
  • the vertical axis (Intensity) indicates the image level (numerical value in the range of 0 to 100) of the pixel line at the horizontal position. It is shown.
  • the peak 33 in which the image level waveform 31 exceeds the threshold value 32 is generated. Therefore, the gain at the time when the peak 33 is detected is automatically or manually detected, and the gain (dB) at the time of detection is set in the memory 13 as a correction reference value.
  • the line correction (FPN suppression function) provided during the operation of the image pickup apparatus 1
  • ON / OFF is automatically switched in conjunction with the gain provided by the image pickup apparatus 1. .. That is, when the gain provided by the image pickup apparatus 1 exceeds the correction reference value, the line correction is turned on, and when the gain provided by the image pickup apparatus 1 is less than the correction reference value, the line correction is turned off.
  • FIG. 4 shows an example of line correction when the FPNs are not continuous in the same color arrangement.
  • FIG. 5 shows an example of line correction when two lines are continuous in the same color arrangement of FPN.
  • the captured image is composed of a plurality of pixel lines extending in the vertical direction. Further, in each of the plurality of pixel lines, each pixel in the pixel line is arranged according to any of the plurality of color arrangement patterns. In the examples of FIGS. 4 and 5, there is a first color scheme in which G (green) and B (blue) are alternately arranged, and a second color scheme in which R (red) and G (green) are alternately arranged.
  • the captured image is configured by alternately arranging the pixel lines of the first color scheme and the pixel lines of the second color scheme.
  • the image processing unit 12 suppresses the FPN with respect to the captured image when the FPN pixel lines are not continuous in the same color scheme, or when the FPN pixel lines are continuous in the same color scheme and the number of consecutive FPNs is two. Perform the processing to do so. If the FPNs have three or more lines in the same color arrangement, the effect on the image is large, so the CMOS image sensor 11 is replaced as a yield product without being subject to line correction.
  • each pixel of the FPN pixel line 41 is corrected based on the same color pixels of the pixel lines of the same color scheme on both sides of the pixel line. do. That is, as shown by reference numerals 42 and 43, each pixel in the pixel line 41 is divided by 2 by adding the values of the same-colored pixels at the same position in the left and right pixel lines of the same color scheme (that is, the average value). ) To correct.
  • each pixel of the left pixel line 51 of the two pixel lines is set to the left of the pixel line. Correction is performed based on the same color pixels of the adjacent pixel lines of the same color scheme. That is, as shown by reference numerals 53 and 55, each pixel of the pixel line 51 is corrected by the value of each pixel at the same position in the pixel line of the same color arrangement pattern on the left side. Further, each pixel of the right pixel line 52 of the two pixel lines is corrected based on the same color pixel of the pixel line of the same color arrangement pattern to the right of the pixel line. That is, as shown by reference numerals 54 and 56, each pixel of the pixel line 52 is corrected by the value of the same color pixel at the same position in the pixel line of the same color arrangement pattern on the right side.
  • FIG. 6 shows an example of a flowchart regarding preprocessing for correcting FPN.
  • the image pickup device is placed in a light-shielded state (lens closed), the gain is increased to the maximum, and it is confirmed whether FPN is detected (step S11). If FPN is not detected, the process ends. On the other hand, when FPN is detected, the number of FPN lines is automatically or manually detected (step S12).
  • the gain is gradually increased from the gain obtained by the specifications of the CMOS image sensor, and when the gain is increased to what dB, the FPN waveform peak (33) becomes the threshold value (33). It is confirmed whether it exceeds 22) (step S13). Then, the confirmed gain is set as the correction reference value, and when the gain during operation exceeds the correction reference value, the line correction is automatically performed for the FPN of the corresponding line (step S14). ..
  • step S15 it is confirmed whether or not the FPN has 3 or more consecutive lines in the same color arrangement.
  • the FPN is not continuous for 3 or more lines in the same color arrangement, that is, when the FPN stays in 2 consecutive lines in the same color arrangement, the same processing as in the case where the number of FPN lines is 1 is performed. That is, the gain is gradually increased from the gain obtained by the specifications of the CMOS image sensor, and it is confirmed to what dB the gain is raised to allow the FPN waveform peak (33) to exceed the threshold value (22) (step). S13). Then, the confirmed gain is set as the correction reference value, and when the gain during operation exceeds the correction reference value, the line correction is automatically performed for the FPN of the corresponding line (step S14). ..
  • step S15 The above processing is repeated for all the FPNs that are not continuous in the same color arrangement and the FPNs of two consecutive lines in the same color arrangement.
  • the gain at which FPN starts to be generated differs for each FPN, so that the correction reference value is set for each FPN line.
  • step S17 it is confirmed whether the FPN at the maximum gain exceeds the threshold value of the product specification. If the FPN at the maximum gain does not exceed the threshold value of the product specification, it is determined that correction is not necessary, and the process is terminated. On the other hand, if the FPN at the maximum gain exceeds the threshold value of the product specification, it is determined that the product is a yield product, and the above processing is performed again after replacing the SMOS image sensor.
  • the image sensor 1 of this example stores the CMOS image sensor 11 and the gain that exceeds the gain obtained by the CMOS image sensor 11 and starts to generate a line-shaped FPN in the captured image as a correction reference value.
  • an image processing unit 12 that performs line correction for suppressing FPN on a captured image when the gain when providing a gain exceeding the gain obtained by the CMOS image sensor 11 exceeds the correction reference value. It is equipped with.
  • the line correction is automatically applied to the captured image, so that the fixed pattern noise generated by the individual difference of the CMOS image sensor is generated. Can be suppressed. Therefore, the FPN generated at a specific gain or higher can be made inconspicuous. This not only makes it possible to increase the sensitivity of the image sensor using a CMOS image sensor used in general industrial or monitoring applications, but also improves the yield of the CMOS image sensor, making the high-sensitivity image sensor more stable. Will be able to produce.
  • each pixel of the FPN is corrected based on the same color pixel at the same position of the pixel lines on both sides of the FPN, but the same color at different positions (neighboring positions above and below) of the pixel lines on both sides of the FPN. Pixels may also be taken into account for correction. Further, each pixel of FPN may be corrected by taking into account not only the pixel adjacent to one line of FPN but also the pixel adjacent to two lines of FPN.
  • the present invention has been described above based on one embodiment, it goes without saying that the present invention is not limited to the configurations described here, and can be widely applied to devices and systems having other configurations.
  • the present invention also provides, for example, a method including a technical procedure relating to the above processing, a program for executing the above processing by a processor, a storage medium for storing such a program in a computer-readable manner, and the like. Is also possible.
  • the present invention can be used in an image pickup device that uses a CMOS image sensor for photographing.
  • Image pickup device 11: CMOS image sensor, 12: Image processing unit, 13: Memory

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Color Television Image Signal Generators (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Studio Devices (AREA)
PCT/JP2020/036085 2020-09-24 2020-09-24 撮像装置 Ceased WO2022064609A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2022551503A JP7352037B2 (ja) 2020-09-24 2020-09-24 撮像装置
PCT/JP2020/036085 WO2022064609A1 (ja) 2020-09-24 2020-09-24 撮像装置
US18/021,013 US12413871B2 (en) 2020-09-24 2020-09-24 Imaging device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/036085 WO2022064609A1 (ja) 2020-09-24 2020-09-24 撮像装置

Publications (1)

Publication Number Publication Date
WO2022064609A1 true WO2022064609A1 (ja) 2022-03-31

Family

ID=80844643

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/036085 Ceased WO2022064609A1 (ja) 2020-09-24 2020-09-24 撮像装置

Country Status (3)

Country Link
US (1) US12413871B2 (https=)
JP (1) JP7352037B2 (https=)
WO (1) WO2022064609A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025013531A1 (ja) * 2023-07-11 2025-01-16 Hoya株式会社 撮像装置、電子内視鏡システム、データ処理方法、プログラム

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012235442A (ja) * 2011-05-03 2012-11-29 Sharp Corp マシンビジョンシステムのためのノイズ低減技術
JP2015076701A (ja) * 2013-10-08 2015-04-20 株式会社リコー 撮像素子、画像読取装置、画像形成装置及び画像読取方法
JP2017157999A (ja) * 2016-03-01 2017-09-07 キヤノン株式会社 撮像装置、撮像システム、および画像処理方法
JP2017200140A (ja) * 2016-04-28 2017-11-02 キヤノン株式会社 撮像素子、撮像装置およびその制御方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011109580A (ja) 2009-11-20 2011-06-02 Nikon Corp カメラ
US8355064B2 (en) * 2010-09-27 2013-01-15 Sharp Laboratories Of America, Inc. Noise reduction for machine vision systems
US8355063B2 (en) 2010-09-27 2013-01-15 Sharp Laboratories Of America, Inc. Camera noise reduction for machine vision systems

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012235442A (ja) * 2011-05-03 2012-11-29 Sharp Corp マシンビジョンシステムのためのノイズ低減技術
JP2012235443A (ja) * 2011-05-03 2012-11-29 Sharp Corp マシンビジョンシステムのためのノイズ低減技術
JP2015076701A (ja) * 2013-10-08 2015-04-20 株式会社リコー 撮像素子、画像読取装置、画像形成装置及び画像読取方法
JP2017157999A (ja) * 2016-03-01 2017-09-07 キヤノン株式会社 撮像装置、撮像システム、および画像処理方法
JP2017200140A (ja) * 2016-04-28 2017-11-02 キヤノン株式会社 撮像素子、撮像装置およびその制御方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025013531A1 (ja) * 2023-07-11 2025-01-16 Hoya株式会社 撮像装置、電子内視鏡システム、データ処理方法、プログラム

Also Published As

Publication number Publication date
JPWO2022064609A1 (https=) 2022-03-31
US12413871B2 (en) 2025-09-09
JP7352037B2 (ja) 2023-09-27
US20230300482A1 (en) 2023-09-21

Similar Documents

Publication Publication Date Title
JP4479373B2 (ja) イメージセンサ
US8035704B2 (en) Method and apparatus for processing a digital image having defective pixels
CN103404123B (zh) 成像装置和缺陷像素校正方法
US7916191B2 (en) Image processing apparatus, method, program, and recording medium
JP5526014B2 (ja) 撮像装置
US9307168B2 (en) Image capture apparatus and method for controlling image capture apparatus in which defective pixels are indicated
JPWO2016143134A1 (ja) 画像処理装置、画像処理方法およびプログラム
JP4985220B2 (ja) 画像処理装置、カメラ、画像処理プログラムおよび画像処理方法
WO2013183378A1 (ja) 撮像装置、画像処理装置、及び画像処理方法
JP5401191B2 (ja) 撮像装置及び信号処理方法
JP2012253726A (ja) 画像処理装置
KR20090097796A (ko) 색수차 정정 방법
JP7352037B2 (ja) 撮像装置
WO2013001868A1 (ja) 撮像装置、撮像方法、及び撮像プログラム
JP2009105872A (ja) 欠陥画素補正回路及び固体撮像装置
JP2009177436A (ja) 固体撮像装置、信号処理装置、および電子情報機器
US10911700B2 (en) Image capturing apparatus, image processing apparatus, and method
JP7426396B2 (ja) 撮像装置
WO2011064949A1 (ja) 撮像装置及び露光制御方法
JPWO2022064609A5 (https=)
JP5694907B2 (ja) 撮像装置
JP4993275B2 (ja) 画像処理装置
KR20140060847A (ko) 카메라 모듈
JP2009201077A5 (https=)
JP2025035240A (ja) 画像処理装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20955203

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022551503

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20955203

Country of ref document: EP

Kind code of ref document: A1

WWG Wipo information: grant in national office

Ref document number: 18021013

Country of ref document: US