CN109274907B - Signal processing method on CMOS image sensor - Google Patents

Signal processing method on CMOS image sensor Download PDF

Info

Publication number
CN109274907B
CN109274907B CN201811392083.8A CN201811392083A CN109274907B CN 109274907 B CN109274907 B CN 109274907B CN 201811392083 A CN201811392083 A CN 201811392083A CN 109274907 B CN109274907 B CN 109274907B
Authority
CN
China
Prior art keywords
signal
image sensor
processing
value
sensor signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811392083.8A
Other languages
Chinese (zh)
Other versions
CN109274907A (en
Inventor
李明
刘戈扬
刘昌举
李毅强
任思伟
刘航
吕玉冰
周亚军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
CETC 44 Research Institute
Original Assignee
China Electronics Technology Group Corp Chongqing Acoustic Optic Electronic Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Electronics Technology Group Corp Chongqing Acoustic Optic Electronic Co ltd filed Critical China Electronics Technology Group Corp Chongqing Acoustic Optic Electronic Co ltd
Priority to CN201811392083.8A priority Critical patent/CN109274907B/en
Publication of CN109274907A publication Critical patent/CN109274907A/en
Application granted granted Critical
Publication of CN109274907B publication Critical patent/CN109274907B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Solid State Image Pick-Up Elements (AREA)

Abstract

The invention belongs to the field of CMOS image sensor chip design and signal processing thereof, in particular to a signal processing method on a CMOS image sensor, which comprises the steps of carrying out binary inversion operation on an A/D bit signal of each bit of an image sensor signal processed by an A/D converter on the CMOS image sensor; carrying out nonlinear processing on the inverted image sensor signal by adopting a signal processing formula, and then carrying out iterative judgment until an optimal iterative value is determined; under the optimal iteration value, adding an enhancement factor in a signal processing formula to process the dark area of the image sensor signal; carrying out binary inversion operation on the processed image sensor signal; the invention implements the inverse operation on the image signal, which is beneficial to the subsequent processing of the signal, and then carries out the nonlinear and iterative processing on the signal after the inverse operation, thereby achieving the purposes of improving the signal-to-noise ratio and the dynamic state of the signal and realizing the compensation processing of the signal.

Description

Signal processing method on CMOS image sensor
Technical Field
The invention relates to the field of CMOS image sensor chip design and signal processing, in particular to a signal processing method on a CMOS image sensor chip.
Background
The current CMOS image sensor is widely applied to a plurality of fields as a visual perception part, but parameters influencing imaging quality such as signal to noise ratio and dynamic range of the CMOS image sensor cannot be further improved, and especially under a low-light-level environment, the image quality is worse. To obtain a high quality image when using a CMOS image sensor, dynamic expansion and improvement can be made on-chip of the CMOS image sensor without increasing the cost of external system hardware. The traditional image signal processing method has the problem that the image enhancement effect and the real-time property cannot be simultaneously considered.
Disclosure of Invention
The invention aims to solve the problems in the background art and provides an on-chip signal processing method which can be applied to a CMOS image sensor. The invention adopts the technical scheme that aiming at imaging in the environment of low light level and the like, a signal pipeline level processing idea is adopted to perform signal inversion operation and nonlinear processing on the collected photoelectric digital signals so as to realize dynamic expansion and signal enhancement of image signals, and the technical processing effect is better and is beneficial to on-chip integration.
The patent provides an on-chip dynamic expansion and signal processing method of a CMOS image sensor, which not only can better retain image details and greatly improve the visual effect of an image, but also is easy to realize on a chip and realizes the real-time processing of signals with higher efficiency. The method has good application and reference values for improving the quality of image signals in low-light environment such as low-light environment.
The invention relates to a signal processing method on a CMOS image sensor, which comprises the following steps:
s1, performing reverse processing on the image sensor signal processed by the A/D converter on the CMOS image sensor;
s2, carrying out nonlinear processing on the image sensor signal after the reverse processing by adopting a signal processing formula;
s3, carrying out iterative judgment on the image sensor signal after nonlinear processing until an optimal iterative value is determined;
s4, under the optimal iteration value, adding an enhancement factor in the signal processing formula, and processing the dark area of the image sensor signal;
and S5, performing inverse processing on the image sensor signal processed in the step S4.
Wherein, the inversion process means an inversion operation, that is, it means that the low level 0 in the signal is changed to the high level 1, and the high level 1 is changed to the low level 0.
And performing inverse operation, signal nonlinear processing, iterative value judgment and iterative processing and secondary inverse operation on the image data subjected to the on-chip A/D processing of the CMOS image sensor. The processing method has the advantages of good processing effect and contribution to on-chip integration of the image sensor.
In step S1, the quantization bit number of the image sensor signal processed by the a/D converter is defined as n, and the maximum gray scale level is 2nThe pixel value range of the image falls within {0, 2 }nAnd the A/D bit signal of each bit is subjected to binary inversion.
Further, for the image sensor signal under the low light environment condition, the nonlinear processing of the image sensor signal needs to adopt the signal processing formula of the formula step S2, which includes:
F(x)＝[(K(x)-A)/T(x)]+A
where f (x) is the processed image sensor signal, k (x) is the image sensor signal at low light level, t (x) is a nonlinear coefficient, and a is the signal maximum within the window.
Further, in a low light environment, the image output by the image sensor is k (x), and the maximum value of the signal in the image array is a, and in order to obtain the processed result f (x), the formula for calculating the nonlinear coefficient t (x) includes:
T(x)＝1-w×min{I[(x)]}
wherein, (x) represents a window function, I [ (x) ] represents a comparison function of the image signal in the window, and w represents an adjustment factor, and the value range of the adjustment factor is 0-1.
Further, since the maximum value of the image signal is usually caused by noise or pixel over-explosion at a certain position, and a processing error may occur when the maximum value is used as the estimated value of a, the step S3 specifically includes determining the maximum value of the image sensor signal in a window of the window function (x) in which the image sensor signal is associated with the gray level 2n-1Performing comparison judgment when the maximum value of the image sensor signal is equal to or greater than 2n-1When the image sensor signal is detected, the value of A is the maximum value of the image sensor signal, namely the optimal iteration value is the maximum value of the image sensor signal; when the maximum value of the image sensor signal is less than 2n-1When, A takes 2n-1The optimal iteration value is that A takes 2n-1。
Further, since f (x) does not significantly expand and increase in the dark region in the dynamic state of the image signal, an enhancement factor r (x) is added to the denominator in the f (x) formula, in step S4, an enhancement factor is added to the signal processing formula at the optimal iteration value, and the processing of the dark region of the image sensor signal specifically includes adding an enhancement factor r (x) to the denominator in the signal processing formula, that is, f (x) ═ k (x) -a)/t (x) xr (x) > + a; the enhancement factor r (x) has a value of 1 when t (x) is at (0, 0.5) and a value of m times t (x) when t (x) is at (0.5, 1), m ∈ (1, 3).
The invention has the beneficial effects that:
1. aiming at imaging in environments such as low light level and the like, the method adopts a signal pipeline level processing idea to perform inverse processing, nonlinear processing, iterative processing and dynamic compensation processing on the acquired image signal, has good processing effect, is beneficial to on-chip integration and occupies less resources;
2. the invention implements the inverse operation on the image signal, which is beneficial to the subsequent processing of the signal, and then carries out the nonlinear and iterative processing on the signal after the inverse operation, thereby achieving the purposes of improving the signal-to-noise ratio and the dynamic state of the signal and realizing the compensation processing of the signal;
3. the invention aims to improve the signal dynamic of the dark area of the image signal by carrying out enhancement factor processing on the dark area.
Drawings
FIG. 1 is a flow chart of a method of the present invention;
FIG. 2 is a schematic diagram of signal processing according to the present invention;
fig. 3 is a diagram of a window array of signal data in accordance with a preferred embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clearly and completely apparent, the technical solutions in the embodiments of the present invention are described below with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
The signal processing method on the CMOS image sensor of the invention, as shown in figure 1, comprises the following steps:
s1, performing reverse processing on the image sensor signal processed by the A/D converter on the CMOS image sensor;
s2, carrying out nonlinear processing on the image sensor signal after the reverse processing by adopting a signal processing formula;
s3, carrying out iterative judgment on the image sensor signal after nonlinear processing until an optimal iterative value is determined;
s4, under the optimal iteration value, adding an enhancement factor in the signal processing formula, and processing the dark area of the image sensor signal;
and S5, performing inverse processing on the image sensor signal processed in the step S4.
The design idea of the invention is to adopt a signal pipeline stage processing mode, and the signal processing process comprises the following steps: the image data after on-chip a/D processing of the CMOS image sensor is subjected to an inverse process, a signal non-linear process, enhancement factor determination and iteration, and a secondary inverse process, and the process flow is shown in fig. 2.
As a preferred embodiment, the image data output by A/D on CMOS image sensor chip, in this embodiment, defines the quantization bit number as n, where n is 10, and the maximum gray scale is 210The pixel value range of the image falls within {0, 2 }10The inverse process of the signal is to perform binary inverse operation on the a/D bit signal of each bit, i.e. 0 (low level) becomes 1 (high level) and 1 becomes 0 (low level), and the process does not change the quantization bit number of the data, and the maximum gray level is still 210。
For the image sensor signal under the low light environment condition, the formula f (x) [ (k (x) -a)/t (x) ] + a is adopted, wherein k (x) is the image signal under the low light, f (x) is the processed image signal, t (x) is a nonlinear coefficient, and a is the maximum value of the image signal in the selected window.
In a low light environment, the image output by the image sensor is K (x), the size of the window can be set according to needs, and the window array is set to be 6 × 6, such as H12Representing the 1 st row and the second column in the window arrayAs shown in fig. 3, the maximum value of the signal in the image window array is a, and in order to obtain the processed result f (x), the nonlinear coefficient t (x) is represented by the formula t (x) 1-w × min { I [ (x)]Where (x) is a window function, I [ (x)]The method is characterized in that a comparison function of image signals in a window is shown, min is to carry out minimum comparison on the image signals in the window, and when the value of an adjusting factor w is between 0.5 and 0.7, the processing effect is good.
In the window array of 6 × 6, first, the maximum value of the signal in the window of (x) is determined by comparing 36 signal data in the window, and then the sum of the maximum values of the signal is performed by 29(i.e., 512) a comparison is made, a being the maximum value when the signal maximum is equal to or greater than gray level 512 and 512 when the signal maximum is less than gray level 512. And carrying out nonlinear processing on the image signal iteration according to the judged A value.
Since f (x) does not expand and improve significantly in the darker region in the dynamic state of the image signal, an enhancement factor r (x) is added to the denominator in the formula f (x), when t (x) is (0, 0.5), the value of the enhancement factor r (x) is 1, when t (x) is (0.5, 1), the value of the enhancement factor r (x) is m times of t (x), and m ∈ (1, 3).
And performing signal inversion operation on the processed F (x) again to obtain the F (x) meeting the requirements of dynamic and signal-to-noise ratio, and outputting the F (x) to a subsequent component for processing.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, and the storage medium may include: ROM, RAM, magnetic or optical disks, and the like.
The above-mentioned embodiments, which further illustrate the objects, technical solutions and advantages of the present invention, should be understood that the above-mentioned embodiments are only preferred embodiments of the present invention, and should not be construed as limiting the present invention, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (6)

1. A method of signal processing on a CMOS image sensor, the method comprising the steps of:
s1, performing inversion processing on the image sensor signal processed by the A/D converter on the CMOS image sensor by adopting binary inversion operation;
s2, carrying out nonlinear processing on the image sensor signal after the reverse processing by adopting a signal processing formula;
s3, carrying out iterative judgment on the image sensor signal after nonlinear processing until an optimal iterative value is determined;
s4, under the optimal iteration value, adding an enhancement factor in the signal processing formula, and processing the dark area of the image sensor signal;
and S5, performing inverse processing on the image sensor signal processed in the step S4.
2. The signal processing method of claim 1, wherein in step S1, the quantization bit number of the image sensor signal processed by the a/D converter is defined as n, and the maximum gray scale level is 2nThe pixel value range of the image falls within {0, 2 }nAnd the A/D bit signal of each bit is subjected to binary inversion.
3. The signal processing method of claim 1, wherein the signal processing formula of step S2 includes:
F(x)＝[(K(x)-A)/T(x)]+A
where f (x) is the image sensor signal after the non-linear processing, k (x) is the image sensor signal under low light, t (x) is the non-linear coefficient, and a is the signal maximum within the window.
4. The signal processing method of claim 3, wherein the formula for calculating the non-linear coefficient T (x) comprises:
T(x)＝1-w×min{I[(x)]}
wherein, (x) represents a window function, I [ (x) ] represents a comparison function of the image signal in the window, and w represents an adjustment factor, and the value range of the adjustment factor is 0-1.
5. The signal processing method of claim 3, wherein the step S3 specifically comprises determining a maximum value of the image sensor signal in a window of a window function (x) in which the image sensor signal is associated with a gray level of 2n-1Performing comparison judgment when the maximum value of the image sensor signal is equal to or greater than 2n-1When the value of A is the maximum value of the image sensor signal, the value of A is less than the gray level 2n-1When, A takes 2n-1。
6. The method of claim 3, wherein in step S4, under the optimal iteration value, the enhancement factor is added to the signal processing formula, and the processing of the dark region of the image sensor signal specifically includes adding an enhancement factor r (x) to the denominator of the signal processing formula, that is, f (x) ═ k (x) -a/t (x) xr (x) ] + a; the enhancement factor r (x) has a value of 1 when t (x) is at (0, 0.5) and a value of m times t (x) when t (x) is at (0.5, 1), m ∈ (1, 3).
CN201811392083.8A 2018-11-21 2018-11-21 Signal processing method on CMOS image sensor Active CN109274907B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811392083.8A CN109274907B (en) 2018-11-21 2018-11-21 Signal processing method on CMOS image sensor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811392083.8A CN109274907B (en) 2018-11-21 2018-11-21 Signal processing method on CMOS image sensor

Publications (2)

Publication Number Publication Date
CN109274907A CN109274907A (en) 2019-01-25
CN109274907B true CN109274907B (en) 2020-10-16

Family

ID=65190444

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811392083.8A Active CN109274907B (en) 2018-11-21 2018-11-21 Signal processing method on CMOS image sensor

Country Status (1)

Country Link
CN (1) CN109274907B (en)

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003168109A (en) * 2001-12-03 2003-06-13 Canon Inc Image processor, data processor, image processing method and data processing method
TW201136294A (en) * 2009-10-28 2011-10-16 Sony Corp Signal processing apparatus, signal processing method, computer program, image processing apparatus, and image capturing apparatus
CN106204470A (en) * 2016-07-01 2016-12-07 湖南源信光电科技有限公司 Low-light-level imaging method based on fuzzy theory
CN106548111B (en) * 2016-10-26 2019-04-16 凌云光技术集团有限责任公司 A kind of two dimensional code open defect detection method and device

Also Published As

Publication number Publication date
CN109274907A (en) 2019-01-25

Similar Documents

Publication Publication Date Title
EP0794513B1 (en) Method and circuit for image enhancement using quantized mean-separate histogram equalization
US5995656A (en) Image enhancing method using lowpass filtering and histogram equalization and a device therefor
CN110211056B (en) Self-adaptive infrared image de-striping algorithm based on local median histogram
JP2007188493A (en) Method and apparatus for reducing motion blur in motion blur image, and method and apparatus for generating image with reduced motion blur by using a plurality of motion blur images each having its own blur parameter
US9014503B2 (en) Noise-reduction method and apparatus
CN101742084B (en) Contrast ratio enhancement processing method and processing device
CN109816608B (en) Low-illumination image self-adaptive brightness enhancement method based on noise suppression
CN111340732B (en) Low-illumination video image enhancement method and device
EP1394742B1 (en) Method for filtering the noise of a digital image sequence
KR20070078463A (en) Apparatus and method for reducing noise from image sensor
CN1383673A (en) Noise filtering image sequence
CN115409872B (en) Image optimization method for underwater camera
CN109274907B (en) Signal processing method on CMOS image sensor
Sunilkumar et al. Removal of high density impulse noise through modified non-linear filter
CN112488968A (en) Image enhancement method for balanced fusion of degree-based histograms
CN112435178A (en) FPGA-based linear array infrared blind pixel engineering processing method and system
CN110660039A (en) Multi-frame weighted wide dynamic image processing method
CN113822835B (en) Image definition evaluation method easy for hardware realization
CN103871033B (en) A kind of adaptive M TFC Remote Sensing Image Restoration method based on spatial domain
CN112637510A (en) Scene self-adaptive image gain adjusting method, system and device and electronic equipment
CN108681988B (en) Robust image resolution enhancement method based on multiple images
CN113554567A (en) Robust ghost image removing system and method based on wavelet transformation
US7773824B2 (en) Signal processing device and method, recording medium, and program
CN116205810B (en) Video noise reduction method and device and electronic equipment
KR100255636B1 (en) Picture quality improving circuit using histogram adapted to noise

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20230117

Address after: 400060 Chongqing Nanping Nan'an District No. 14 Huayuan Road

Patentee after: CHINA ELECTRONICS TECHNOLOGY GROUP CORPORATION NO.44 Research Institute

Address before: 404100 no.367, Xiyong Road, weidianyuan, Xiyong Town, Shapingba District, Chongqing

Patentee before: CHINA ELECTRONICS TECHNOLOGY GROUP CORPORATION CHONGQING ACOUSTIC-OPTIC-ELECTRONIC CO.,LTD.

TR01 Transfer of patent right