CN113411511A - High frame frequency imaging system image preprocessing method based on histogram analysis - Google Patents

High frame frequency imaging system image preprocessing method based on histogram analysis Download PDF

Info

Publication number
CN113411511A
CN113411511A CN202110728063.9A CN202110728063A CN113411511A CN 113411511 A CN113411511 A CN 113411511A CN 202110728063 A CN202110728063 A CN 202110728063A CN 113411511 A CN113411511 A CN 113411511A
Authority
CN
China
Prior art keywords
image
histogram
pixel value
imaging system
high frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110728063.9A
Other languages
Chinese (zh)
Other versions
CN113411511B (en
Inventor
孙宏海
赵阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changchun Institute of Optics Fine Mechanics and Physics of CAS
Original Assignee
Changchun Institute of Optics Fine Mechanics and Physics of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changchun Institute of Optics Fine Mechanics and Physics of CAS filed Critical Changchun Institute of Optics Fine Mechanics and Physics of CAS
Priority to CN202110728063.9A priority Critical patent/CN113411511B/en
Publication of CN113411511A publication Critical patent/CN113411511A/en
Application granted granted Critical
Publication of CN113411511B publication Critical patent/CN113411511B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/741—Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H04N25/57—Control of the dynamic range
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors

Abstract

The invention discloses an image preprocessing method of a high frame frequency imaging system based on histogram analysis, relates to the field of high-speed imaging, and solves the problems that automatic dimming is difficult to apply in the design of the existing high frame frequency imaging system and the image preprocessing function is lacked; histogram equalization based on shift operations. The histogram is analyzed by improving the statistical method of the histogram information, the floating point operation is replaced by the shift operation, the method is more suitable for the design of a high frame frequency imaging system based on the FPGA, the image preprocessing can be realized while the high frame frequency is ensured, the calculation mode of the histogram equalization is optimized by improving the acquisition method of the histogram information, and the automatic dimming and the histogram equalization can be applied on the basis of ensuring the real-time performance of the direct-transfer high-speed camera so as to enhance the visual effect of imaging.

Description

High frame frequency imaging system image preprocessing method based on histogram analysis
Technical Field
The invention relates to the field of high-speed imaging, in particular to a high-frame-rate imaging system image preprocessing method based on histogram analysis, which can be applied and realized in high-speed camera design.
Background
In the high-speed CMOS imaging process, the photodiode on each pixel converts a light intensity signal sensed in the exposure time into optical charges for storage, and then converts the optical charges into digital pixel signals through A/D conversion. The relationship between the image brightness and the current exposure amount is as follows:
Figure BDA0003138262140000011
where B is the current image brightness, a is a constant, L represents the ambient brightness, G is the camera gain, T represents the exposure time, and F/# is the F-number of the lens. Thus, both parameters G and T affect the brightness of the image. The automatic dimming is to capture the characteristic parameters of the image and compare and analyze the characteristic parameters with the reference values, establish the mapping relation between the image parameters and the dimming parameters, and feed back the adjustment value to the imaging control system to obtain a good image and ensure the imaging quality.
The light regulating method mainly applied in engineering is pixel average value method, one frame image pixel average value
Figure BDA0003138262140000013
Can be calculated from the following formula:
Figure BDA0003138262140000012
m is the number of image rows, N is the number of image columns, and I (x, y) is the pixel value of the x-th row and the y-th column of the image. The average value of the pixels of the image is used as a feedback quantity, the exposure time is adjusted by comparing the feedback quantity with a target value, and the average value of the pixels is controlled within a reasonable range. However, in the actual shooting, if the difference between the background and the target brightness is too large, the pixel average method will cause the subject to be over-exposed or under-exposed.
The existing region weighting algorithm is an improvement of a pixel average value algorithm, the image is integrally divided into a plurality of regions, different regions have different weights, the weight of a shooting main body region is larger, and the weight occupied by a background region is smaller. However, in the implementation of the FPGA, the division of the regions consumes a large amount of computing resources, and is difficult to be directly applied to the design of a high-speed camera. Although the existing dimming algorithm can deal with a plurality of shooting environments, the real-time performance of a high-speed camera is affected due to the complexity of the existing dimming algorithm, and the existing dimming algorithm is difficult to be directly applied to high-frame-rate imaging design.
After the imaging system acquires the image, the appearance of the image can be improved through image processing. Histogram equalization is often used to deal with the problem of indiscernible details due to more concentrated histogram distribution, which can increase the dynamic range and contrast of the image.
The transformation function T (r) of histogram equalization is shown in formula (3), wherein r represents the original pixel value of the normalized image, s represents the pixel value of the image after histogram equalization, and Pr(r) represents the probability density of the pixel value r, and t (r) is the cumulative distribution function of the original image histogram.
In digital image processing, the pixel values are discretized, so the probability is replaced by the frequency of occurrence of the individual pixel values and the discrete form of the transformation function can be represented as equation (4). Where k represents the pixel value before normalization, rkRepresenting the normalized pixel value of k, skRepresenting transformed pixel values, niThe representative pixel value is the number of pixels corresponding to i.
Figure BDA0003138262140000021
Figure BDA0003138262140000022
However, in the design of a high frame rate imaging system, a large amount of logic resources are consumed for realizing histogram equalization in an FPGA, and the real-time performance of the camera is also affected by the fact that the calculation of multiplication and division is realized through a multiplier and a divider IP core. Therefore, there is a need for an image pre-processing method suitable for use in high frame rate imaging systems.
Disclosure of Invention
The invention provides a high frame frequency imaging system image preprocessing method based on histogram analysis, aiming at solving the problems that automatic dimming is difficult to apply in the design of the existing high frame frequency imaging system and the image preprocessing function is lacked.
The invention has the beneficial effects that: the invention analyzes the histogram by improving the statistical method of the histogram information, replaces floating point operation with shift operation, is more suitable for the design of a high frame frequency imaging system based on FPGA, and can realize image preprocessing while ensuring high frame frequency.
Drawings
FIG. 1 is a schematic diagram of an image preprocessing method for a high frame rate imaging system based on histogram analysis according to the present invention;
fig. 2 is a structural diagram of an imaging system of the image preprocessing method of a high frame rate imaging system based on histogram analysis according to the present invention.
Detailed Description
The image preprocessing method of the high frame rate imaging system based on histogram analysis according to the present embodiment is described with reference to fig. 1 and 2, and includes cumulative histogram acquisition, exposure information extraction feedback, and histogram equalization.
Step one, extracting cumulative histogram data of an image, and the specific process is as follows:
first, a cumulative histogram including the number of pixels equal to or smaller than a current pixel value is extracted from an image output from an image sensor, the current pixel value and the corresponding cumulative histogram are comprehensively analyzed, and the number of pixels Z and W around a pixel value of 0 or 255 are detected, and Z and W are calculated as follows:
Figure BDA0003138262140000031
Figure BDA0003138262140000032
in the formula, npAnd nqThe number of pixels when the pixel values are p and q. Z is the pixel value p at [0, delta ]]The number of pixels in the interval W is the pixel value q in the interval [ 255-delta, 255%]The number of pixels in the interval. Using the difference between Z and WThe value determines the image exposure.
Judging the exposure condition through the following formula, and adjusting the exposure time length;
Z-W≥ρ (7)
W-Z≥ρ (8)
setting a threshold rho, and if | Z-W | is less than rho, considering that the image exposure is normal;
if Z is larger than W and Z-W is larger than or equal to rho, the histogram is concentrated near the pixel value 0, the image is underexposed, the camera exposure time length T needs to be increased, and if Z is smaller than W and W-Z is larger than or equal to rho, the image is overexposed, and the camera exposure time length T needs to be reduced.
Step two, acquiring an equalized pixel value according to a histogram equalization algorithm by using the acquired image cumulative histogram; the method specifically comprises the following steps:
firstly: obtaining a calculation result t after the preliminary equalization according to the following formulao；
Figure BDA0003138262140000041
In the formula (I), the compound is shown in the specification,
Figure BDA0003138262140000042
niis a cumulative histogram of pixel values k, D is the gray level of the image, niThe number of pixels with a pixel value of i; m is the number of image rows and N is the number of image columns. D is generally taken as 2d1, D represents the bit depth of the image pixel, and can be approximated by left-shifting the cumulative histogram by D bits when implementing D in FPGA, and by right-shifting the log when implementing divide by M N computation2Calculating (M multiplied by N) bits to obtain a primary calculation result to. The calculation of multiplication and division in the formula (9) is realized through shift operation, and a calculation result t after preliminary equalization is obtainedo。
Secondly, because errors exist between the shift operation and the floating point operation, an error function e (k) can be obtained by fitting the errors, and a equalized final pixel value t is obtained after error compensatione(ii) a To compensate for errors introduced by shift operations, errors are introducedThe difference function e (k) is as follows.
Figure BDA0003138262140000051
In the formula, the scaling factor b is related to the value of M × N, and when the value of M × N is fixed, the scaling factor is also fixed, and the determination method is as follows: and (3) obtaining equalized gray values by taking the value [0, M multiplied by N ] of the cumulative histogram, respectively carrying out floating point operation and shift operation, subtracting the two operation results to obtain an error e, and fitting to determine an error function e (k) and a proportional coefficient b of the cumulative histogram.
And step three, mapping the pixels of the original image and the equalized pixel values one by one to obtain an equalized image. Namely: the final histogram equalized pixel value teComprises the following steps:
te＝T(k)+e(k) (11)
the embodiment is described with reference to fig. 2, in the embodiment, in the process of collecting the cumulative histogram, the FPGA is used to extract cumulative histogram data of the image, the FPGA acquires the cumulative histogram data of the image through a structure composed of a decoder and a counter, the decoder reads a pixel value and outputs a high level pulse at a corresponding gray level output port, and the counter adds 1 after detecting the high level pulse; and after counting the cumulative histogram of one frame, storing the cumulative histogram into the two-port Ram, and taking the gray value as the corresponding address bit.

Claims (3)

1. The image preprocessing method of the high frame frequency imaging system based on histogram analysis is characterized by comprising the following steps: the method is realized by the following steps:
step one, extracting accumulated histogram data of an image; the specific process is as follows: extracting image characteristic values satisfying the following formula from the accumulated histogram data;
Figure FDA0003138262130000011
Figure FDA0003138262130000012
in the formula, npAnd nqRepresenting the number of pixels when the pixel values are p and q; z is the pixel value p at [0, delta ]]The number of pixels in the interval W is the pixel value q in the interval [ 255-delta, 255%]The number of pixels in the interval; judging the exposure condition through the following formula, and adjusting the exposure time length;
Z-W≥ρ
W-Z≥ρ
in the formula, rho is a threshold value; if the | Z-W | is less than rho, the image exposure is normal;
if Z is larger than W and Z-W is larger than or equal to rho, the histogram is concentrated near the pixel value 0, the image is underexposed, and the exposure duration of the camera is increased;
if Z is less than W and W-Z is more than or equal to rho, overexposing the image, and reducing the exposure time of the camera;
step two, acquiring an equalized pixel value according to a histogram equalization algorithm by using the acquired image cumulative histogram; the specific process is as follows:
obtaining a calculation result t after the preliminary equalization according to the following formulaoExpressed by the following formula:
Figure FDA0003138262130000013
in the formula (I), the compound is shown in the specification,
Figure FDA0003138262130000014
a cumulative histogram representing the pixel value k, D being the grey level of the image, niThe number of pixels with a pixel value of i, M is the number of image rows, and N is the number of image columns;
obtaining t in the above calculationoIn the process, errors exist, the errors are fitted simultaneously to obtain an error function e (k), and the equalized final pixel value t is obtained after the error fitting function e (k) is adopted for compensatione；
And step three, mapping the original image and the equalized pixel values one by one to obtain an equalized image.
2. The histogram analysis based image pre-processing method for high frame rate imaging system according to claim 1, wherein: in the second step, an error function e (k) is introduced: is formulated as:
Figure FDA0003138262130000021
in the formula, b is a proportionality coefficient, and the determination method comprises the following steps:
by taking the value of [0, MXN ] to the cumulative histogram]Respectively obtaining the gray value of the equalized image through floating point operation and shift operation, subtracting the two operation results to obtain an error e, and fitting to determine a proportional coefficient b of e (k) and the cumulative histogram; namely: the final histogram equalized pixel value teComprises the following steps:
te＝T(k)+e(k)。
3. the histogram analysis based image pre-processing method for high frame rate imaging system according to claim 1, wherein: in the first step, FPGA is adopted to extract accumulated histogram data of an image, the FPGA acquires the accumulated histogram data of the image through a structure consisting of a decoder and a counter, the decoder reads a pixel value and outputs a high level pulse at a corresponding gray level output port, and the counter adds 1 after detecting the high level pulse; and after counting the cumulative histogram of one frame, storing the cumulative histogram into the two-port Ram, and taking the gray value as the corresponding address bit.
CN202110728063.9A 2021-06-29 2021-06-29 High frame frequency imaging system image preprocessing method based on histogram analysis Active CN113411511B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110728063.9A CN113411511B (en) 2021-06-29 2021-06-29 High frame frequency imaging system image preprocessing method based on histogram analysis

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110728063.9A CN113411511B (en) 2021-06-29 2021-06-29 High frame frequency imaging system image preprocessing method based on histogram analysis

Publications (2)

Publication Number Publication Date
CN113411511A true CN113411511A (en) 2021-09-17
CN113411511B CN113411511B (en) 2022-05-17

Family

ID=77680268

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110728063.9A Active CN113411511B (en) 2021-06-29 2021-06-29 High frame frequency imaging system image preprocessing method based on histogram analysis

Country Status (1)

Country Link
CN (1) CN113411511B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114140694A (en) * 2021-12-07 2022-03-04 盐城工学院 Aesthetic composition method for coupling individual aesthetics with photographic aesthetics

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0566751A (en) * 1991-09-10 1993-03-19 Matsushita Electric Ind Co Ltd Gradation correcting circuit
US5296941A (en) * 1991-12-20 1994-03-22 Matsushita Electric Industrial Co, Ltd. Gradation compensation apparatus of video image
JP2002077616A (en) * 2000-08-31 2002-03-15 Canon Inc Image processing method and apparatus, and recording medium
US20050002565A1 (en) * 2003-07-04 2005-01-06 Ed-Tech Co., Ltd. Apparatus and method for controlling brightness of moving image signals in real time
CN101873429A (en) * 2010-04-16 2010-10-27 杭州海康威视软件有限公司 Processing method and device of image contrast
CN102523386A (en) * 2011-12-16 2012-06-27 中国科学院西安光学精密机械研究所 Automatic exposure method based on histogram equalization
CN106097286A (en) * 2016-06-21 2016-11-09 浙江大华技术股份有限公司 A kind of method and device of image procossing
CN108924525A (en) * 2018-06-06 2018-11-30 平安科技（深圳）有限公司 Brightness of image method of adjustment, device, computer equipment and storage medium
US20190188857A1 (en) * 2017-12-20 2019-06-20 Duelight Llc SYSTEM, METHOD, AND COMPUTER PROGRAM for ADJUSTING IMAGE CONTRAST USING PARAMETERIZED CUMULATIVE DISTRIBUTION FUNCTIONS
US20200195827A1 (en) * 2018-12-12 2020-06-18 Vivotek Inc. Metering compensation method and related monitoring camera apparatus

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0566751A (en) * 1991-09-10 1993-03-19 Matsushita Electric Ind Co Ltd Gradation correcting circuit
US5296941A (en) * 1991-12-20 1994-03-22 Matsushita Electric Industrial Co, Ltd. Gradation compensation apparatus of video image
JP2002077616A (en) * 2000-08-31 2002-03-15 Canon Inc Image processing method and apparatus, and recording medium
US20050002565A1 (en) * 2003-07-04 2005-01-06 Ed-Tech Co., Ltd. Apparatus and method for controlling brightness of moving image signals in real time
CN101873429A (en) * 2010-04-16 2010-10-27 杭州海康威视软件有限公司 Processing method and device of image contrast
CN102523386A (en) * 2011-12-16 2012-06-27 中国科学院西安光学精密机械研究所 Automatic exposure method based on histogram equalization
CN106097286A (en) * 2016-06-21 2016-11-09 浙江大华技术股份有限公司 A kind of method and device of image procossing
US20190188857A1 (en) * 2017-12-20 2019-06-20 Duelight Llc SYSTEM, METHOD, AND COMPUTER PROGRAM for ADJUSTING IMAGE CONTRAST USING PARAMETERIZED CUMULATIVE DISTRIBUTION FUNCTIONS
CN108924525A (en) * 2018-06-06 2018-11-30 平安科技（深圳）有限公司 Brightness of image method of adjustment, device, computer equipment and storage medium
US20200195827A1 (en) * 2018-12-12 2020-06-18 Vivotek Inc. Metering compensation method and related monitoring camera apparatus

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114140694A (en) * 2021-12-07 2022-03-04 盐城工学院 Aesthetic composition method for coupling individual aesthetics with photographic aesthetics

Also Published As

Publication number Publication date
CN113411511B (en) 2022-05-17

Similar Documents

Publication Publication Date Title
US8885093B2 (en) Image pickup apparatus, image pickup method, exposure control method, and program
US8094211B2 (en) Method and apparatus for obtaining image using entropy
CN110211056B (en) Self-adaptive infrared image de-striping algorithm based on local median histogram
CN109671035B (en) Infrared image enhancement method based on histogram
CN115633259B (en) Automatic regulation and control method and system for intelligent camera based on artificial intelligence
US7382941B2 (en) Apparatus and method of compressing dynamic range of image
CN111161163B (en) Infrared image contrast enhancement method for histogram double gamma correction
CN113411511B (en) High frame frequency imaging system image preprocessing method based on histogram analysis
CN104243770A (en) Image signal processing apparatus, imaging apparatus and image processing program
CN111127343B (en) Histogram double-control infrared image contrast enhancement method
US10897578B2 (en) Exposure ratio control
CN110782403A (en) Infrared image non-uniformity correction method
US11640654B2 (en) Image processing method and apparatus
US11816813B2 (en) Image data pre-processing for neural networks
CN111080563B (en) Histogram equalization method based on traversal optimization
CN108551555B (en) Automatic exposure method of industrial camera using CMOS detector
US20230033445A1 (en) Method and system for processing image
US11323666B1 (en) Dynamic depth-color-correction
CN114463208A (en) Dimension-building monitoring video image enhancement method based on contrast-limiting adaptive histogram equalization method
CN112637510A (en) Scene self-adaptive image gain adjusting method, system and device and electronic equipment
CN112991240A (en) Image self-adaptive enhancement algorithm for real-time image enhancement
CN113962844A (en) Image fusion method, storage medium and terminal device
US20120099007A1 (en) Image processing apparatus and control method thereof
CN106657807A (en) Automatic exposure method of color camera
JPH11154235A (en) Method and device for extracting image reference density value and image processor

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant