CN114257741B - A fast-response approach to in-vehicle HDR - Google Patents

Abstract

The invention discloses a quick-response vehicle-mounted HDR method, which comprises the following steps: selecting a first frame of an image to be processed as a sampling frame, selecting an interested area on the sampling frame, and calculating a Target value Target according to pixel values of specific pixel rows in the interested area; comparing the pixel values of all pixels in the sampling frame with Target to obtain a weight matrix, wherein elements in the weight matrix represent weight values of corresponding pixels; recording the next frame of the sampling frame as an operation frame, and adjusting the pixel value of each pixel in the operation frame according to the weight matrix to obtain an adjusted operation frame image; and taking the operation frame before adjustment as a new sampling frame, taking the next frame of the operation frame as a new operation frame, and repeating the steps until the image to be processed is processed. The method and the device can enhance the image processing effect while reducing the chip area and the power consumption, and the generated HDR image effect is better represented in detail.

Description

A fast-response approach to in-vehicle HDR

technical field

The invention belongs to the field of vehicle-mounted high dynamic (HDR) algorithms and fast automatic exposure (AE, auto exposure) methods, and in particular relates to a fast-response vehicle-mounted high dynamic method.

Background technique

The determination of exposure time and gain factor is crucial to the generation of HDR images.

The traditional AE algorithm is to expose the entire image line by line, and perform statistical summation and averaging on all pixels to obtain an average brightness. Then use this brightness average to calculate the exposure time T and gain coefficient Ratio. There are also some AE algorithms that use the weighted gray entropy difference sub-regional exposure algorithm, but this algorithm has a large amount of calculation, which will bring huge waste of area and power consumption, and is not suitable for application on the chip hardware side. In order to quickly estimate the brightness information of the entire image, it is necessary to adjust the AE algorithm to reduce the calculation time as much as possible while ensuring the image effect. Vehicle-mounted sensors are different from mobile phone or camera sensors. Mobile phone or camera sensors can obtain the region of interest by changing the focus position, and perform AE calculations in the region of interest. block regions, since most of the visual information is concentrated in the middle of the image. Therefore, the core of the fast AE algorithm is to focus on the central area of the image. When the central area does not meet the requirements, the pixel information of the upper and lower areas of the image is added as a supplementary calculation to make the results of the AE algorithm more accurate. Existing image processing chips use the AE module to measure light and evaluate the brightness of the current environment. The method used is the frame average method or the global partition exposure method. The exposure time and gain coefficient of the current frame are based on Determined by the overall brightness value of the previous frame.

First, image exposure is performed, then sampling and quantization are performed, and then the pixel values of the entire image are summed and averaged to obtain AP.

Exposure time and gain factor are calculated by AP. This method requires a lot of calculations, and the overall response time is too long, and in some scenarios, it is impossible to give the exposure time and gain factor required by some areas, which is not conducive to the rapid and accurate response of vehicle images.

At the same time, CMOS image processing sensors are given the same exposure time and gain for the same image. The current method of implementing HDR on chip hardware basically uses multiple exposures for image fusion processing. These calculations will bring a lot of problems. Large chip area overhead and power consumption overhead. The exposure and gain of the same image are the same, so it is impossible to fine-tune each pixel. For example, for a tunnel scene, the image scene before entering the tunnel and when exiting the tunnel is very complicated. For pixels in the same row, the pixel values inside and outside the tunnel are very different. The traditional HDR algorithm even if each row Given a specific exposure time and gain factor, it is still impossible to improve the pixel value difference caused by a specific scene, resulting in the generated HDR image effect being poor in some details. If it is used in the judgment of assisted driving and automatic driving, it is easy to misjudgment and cause safety problems.

Contents of the invention

The problem to be solved by the present invention is to propose a fast-response vehicle-mounted HDR method to generate a high-quality HDR image of an area of interest. The content includes the fast AE algorithm, using the results of the AE algorithm to generate the weight matrix and the new HDR algorithm.

Technical scheme of the present invention is:

A fast-response vehicle-mounted HDR method, characterized in that the steps include:

(1) Fast AE algorithm: select the first frame of the image to be processed as the sampling frame, select the region of interest on the sampling frame, and calculate the target value Target through the pixel value of a specific pixel row in the region of interest;

(2) Obtain the weight matrix: compare the pixel values of all pixels in the sampling frame with the Target to obtain the weight matrix, and the elements in the weight matrix represent the weight values of the corresponding pixels;

(3) Pixel-level HDR algorithm: record the next frame of the sampling frame as the operation frame, adjust the pixel value of each pixel in the operation frame according to the weight matrix, and obtain the adjusted operation frame image;

(4) The operation frame before adjustment is used as a new sampling frame, and the next frame of the operation frame is used as a new operation frame, and the above steps (1)-(3) are repeated until the processing of the image to be processed is completed.

Further, in the step (1) fast AE algorithm, the selection method of the region of interest is:

Note that the total number of rows of pixels in the sampling frame is N, and each row is numbered 0, 1, 2,..., N-1;

的行为区域1；Select a number range of

Behavior area 1;

的行为区域2；Select a number range of

Behavior area 2;

的行为区域3；Select a number range of

Behavior area 3;

Areas 1, 2, and 3 together constitute the area of interest; where n ₁ , n ₂ , and n ₃ are the number of rows in area 1, 2, and 3 respectively, and n ₁ =n ₃ , which account for 8%-12 of the total number of rows N %, n ₂ accounts for 25%-35% of the total row number N.

Further, in step (1) fast AE algorithm, the selection method of the specific pixel row is: select all rows in the region of interest, or rows with odd numbers, or rows with even numbers.

Further, in the step (1) fast AE algorithm, the calculation method of the target value Target is:

First calculate the average value of all pixel values in the specific pixel row of area 1 and area 2, which are recorded as AP1 and AP2 respectively;

If Threshold1≤AP2≤Threshold2, then

Target = AP2;

If AP2>Threshold2 or AP2<Threshold1, calculate the average value of all the pixel values of all pixels in a specific pixel row in area 3, denoted as AP3,

Target = (AP1+AP2+AP3)/3;

Wherein, Threshold1 is the lower threshold, Threshold1=128-t; Threshold2 is the upper threshold, Threshold2=128+t, where 4≤t≤32.

Further, in step (2), the weight matrix can be obtained in various ways, which are as follows:

The first way to obtain the weight matrix is to set a number with a bit width of 1 bit to represent the weight value of the pixel:

If P _i,j ≥ Target, then A _i,j =1; if P _i,j <Target, then A _i,j =0;

Among them, P _{i, j} is the pixel value of the pixel point in the i-th row and the j-th column, and A _{i, j} is the weight value of the i-th row and the j-th column pixel point, as shown in the following table:

Bit width = 1bit P_i,j≥Target P_i,j<Target A_i,j 1 0

The weight values of all pixels in the sampled frame form a weight matrix.

The second way to obtain the weight matrix is to set a number with a bit width of 2 bits to represent the weight value of the pixel:

If P _i,j ≥ Target+V, then A _i,j =3;

If Target≤P _i,j <Target+V, then A _i,j =2;

If Target-V≤P _i,j <Target, then A _i,j =1;

If P _i,j <Target-V, then A _i,j =0;

Among them, P _{i, j} is the pixel value of the pixel point in the i-th row and the j-th column, A _{i, j} is the weight value of the i-th row and the j-th column pixel point, and V is the refinement coefficient, usually V≤32, namely as follows surface:

The weight values of all pixels in the sampled frame form a weight matrix.

Under the condition that the chip area allows, a weight value with a higher bit width (such as 3bit, 4bit...) can also be adopted. The larger the bit width, the finer the pixel value division of the image, and the better the effect. When the bit width of the weight matrix increases and more details can be described, the refinement coefficient V can be correspondingly increased.

The chip resources required by the above two methods are relatively large, but considering the actual chip area overhead, the storage content of the weight matrix will often be reduced, and the third method is proposed here.

The third way to obtain the weight matrix is that multiple pixels share one weight:

Since most of the pixel values in the image are not abrupt, the brightness information of adjacent points is a relatively close value. Therefore, n×n adjacent pixels are divided into a whole, and the average pixel value of all pixels in the whole is regarded as the pixel value of any pixel in the whole, where n can select the number of rows and columns of the sampling frame Any common divisor of , in actual operation, can be determined according to specific requirements. The smaller n is, the more accurate the stored information will be, but the chip area will double. Conversely, the less information is stored, but the chip area is doubled. All pixels in the sampling frame are divided accordingly;

If P _i,j ≥ Target, then A _i,j =1; if P _i,j <Target, then A _i,j =0;

Among them, P _i,j is the pixel value of the pixel point in row i and column j, A _i,j is the weight value of pixel point in row i and column j, and the weight values of all pixels in the sampling frame form a weight matrix. Multiple pixels share one weight, which will result in a slightly worse picture effect, but save a lot of storage area.

Further, in step (3), the method for adjusting the pixel value in the operation frame is:

If the corresponding weight value of the pixel in the weight matrix is 1, then subtract the compensation value VP1 from its pixel value; if the weight value of the pixel point is 0, add the compensation value VP1 to its pixel value; the selected VP1 needs Ensure that the pixel values after the operation are all between 8-250; operate on all the pixels in the operation frame to obtain the adjusted operation frame image.

Further, in step (3), the method for adjusting the pixel value in the operation frame is:

If the corresponding weight value of the pixel in the weight matrix is 3, subtract the compensation value VP2 from its pixel value; if the weight value of the pixel is 2, subtract the compensation value VP3 from its pixel value; If the weight value is 1, add the compensation value VP3 to its pixel value; if the weight value of the pixel point is 0, add the compensation value VP2 to its pixel value; where VP2>VP3, the selected VP2 and VP3 need to ensure that after operation The pixel values of all are between 8-250; all pixels in the operation frame are operated to obtain the adjusted operation frame image.

Compared with prior art, the beneficial effect of the present invention is:

1. For the power consumption of each frame of image, once AP2 in area 2 satisfies the corresponding conditions, AP3 in area 3 does not need to be calculated. At the same time, when calculating the target, you can either select all the rows in the region of interest, or you can select only half (odd or even) of the rows, which has little effect on the results in most image scenes. Using the above method, the power consumption and area of the chip, as well as the calculation time can be greatly reduced.

2. When obtaining the weight matrix, you can increase the precision and enhance the image processing effect by increasing the bit width of the weight value, or you can reduce the chip area and speed up the calculation speed by sharing one weight value with multiple pixels, which is convenient Adaptive adjustments are made according to the actual needs and application scenarios of the chip.

3. The HDR algorithm of the present invention operates on a single pixel point, and can distinguish and process differences in pixel values of pixels in the same row, and the generated HDR image effect is better in details.

Description of drawings

FIG. 1 is a schematic diagram of the region of interest selected in Example 1.

FIG. 2 is a matrix of pixel values of a sampled frame in Embodiment 2.

FIG. 3 is a weight matrix of sampled frames in Embodiment 2.

FIG. 4 is the pixel value matrix of the operation frame before correction in the second embodiment.

FIG. 5 is the corrected pixel value matrix of the operation frame in the second embodiment.

Figure 6 is an example where multiple adjacent points take the same weight.

detailed description

Below in conjunction with accompanying drawing, the present invention will be further described:

Example 1

In this embodiment, the operation method of the fast AE algorithm is mainly displayed through an image of 1280*960, as shown in FIG. 1 .

For a 1280*960 image, first calculate the pixel average AP1 of the odd-numbered lines between 192 and 288 lines in area 1, and then calculate the pixel average AP2 of the odd-numbered lines between 336 and 624 lines in area 2. At this time, judge Whether AP2 satisfies 120≤AP2≤136, if yes, then Target=AP2, if not, then continue to calculate the pixel average value AP3 of odd-numbered lines between 672 and 768 lines in area 3, Target=(AP1+AP2+AP3) /3. This can greatly shorten the calculation amount of the AE algorithm, speed up the evaluation of AE and reduce the cost of the chip area.

Example 2

This embodiment mainly takes a 7*7 image as an example to demonstrate the generation of the weight matrix and the operation method of the pixel-level HDR algorithm.

Generation of weight matrix:

The weight value is a level evaluation of the pixel value of an entire image, which is generated by the previous frame image (sampling frame) and has an effect on the next frame image (operation frame). The reason for this operation is that the usual video image refresh rate will not be lower than 60FPS, so it is more than 60 times per second, and the brightness change of the image between two adjacent frames is almost negligible.

In a 7*7 image, a weight value is set for each pixel to evaluate the level of the pixel value. When the weight value is 1bit, a target value Target for evaluating the current brightness value is given by the fast AE module, and each pixel value of the sampling frame image is compared with the target value Target. If it is greater than the target value Target, then this The weight of the pixel value is given as 1, otherwise it is 0. In this way, a weight matrix of 7*7 size can be generated, 1 represents the brighter pixels in the image, and 0 represents the slightly darker pixels in the image.

As shown in Figure 2, it is a sample frame pixel value matrix of a 7*7 image, the pixel values below it are generally low, and the pixel values above are generally high, which can be regarded as a simulation of a tunnel scene. Select the middle 3 rows as the region of interest, count the average value of its pixels as 92, and determine it as the Target value. Pixels with a pixel value lower than 92 are assigned a weight value of 0, and vice versa, a weight value of 1 is assigned. The weight matrix generated after comparison is shown in Figure 3. This weight matrix has only 1 bit information for each pixel, which reflects whether the brightness information of the pixel value at this position in the entire image is darker or brighter.

Pixel-level HDR algorithm:

After having the weight matrix obtained above, operate the frame image, as shown in Figure 4, you can use this weight matrix to adjust to get a better image. Perform the following operations on the operation frame: subtract VP=16 from the pixel value of the pixel marked as 1 in the weight matrix to reduce its brightness; add VP=16 to the pixel value of the pixel marked as 0 to increase its brightness.

As shown in Figure 5, after the adjusted image pixel value matrix, the very dark pixel values in the tunnel have been greatly improved, and can be refined to each pixel point of each row. This can reduce the chance of misjudgment when doing other recognition algorithms in the future.

As shown in Figure 6, it shows an instance where 16 adjacent points take the same weight.

The specific embodiments above are used to explain the present invention, rather than to limit the present invention. Within the spirit of the present invention and the protection scope of the claims, any modification and change made to the present invention will fall into the protection scope of the present invention.

Claims (6)

1. A fast-response vehicle-mounted HDR method is characterized by comprising the following steps:

(1) Fast AE algorithm: selecting a first frame of an image to be processed as a sampling frame, selecting an interested area on the sampling frame, and calculating a Target value Target according to pixel values of specific pixel rows in the interested area;

(2) Acquiring a weight matrix: comparing the pixel values of all pixels in the sampling frame with Target to obtain a weight matrix, wherein elements in the weight matrix represent weight values of corresponding pixels;

(3) Pixel level HDR algorithm: recording the next frame of the sampling frame as an operation frame, and adjusting the pixel value of each pixel in the operation frame according to the weight matrix to obtain an adjusted operation frame image;

(4) Taking the operation frame before adjustment as a new sampling frame, taking the next frame of the operation frame as a new operation frame, and repeating the steps (1) - (3) until the image to be processed is processed;

in the step (1), the method for selecting the region of interest comprises the following steps:

recording the total line number of pixels of a sampling frame as N, wherein the line numbers are respectively 0,1,2, \8230 \ 8230;, N-1;

selecting a number range of

Behavior region 1 of (1);

selecting a number range of

Behavior region 2 of (1);

select the number range of

Behavior region 3;

regions 1,2, 3 together constitute a region of interest; wherein n is ₁ 、n ₂ 、n ₃ Number of rows, n, of regions 1,2, 3, respectively ₁ ＝n ₃ Respectively account for 8% -12% of the total number of lines N, N ₂ Accounting for 25-35% of the total number of rows N;

the selection method of the specific pixel row comprises the following steps: selecting all lines in the region of interest, or the lines with odd numbers, or the lines with even numbers;

the calculation method of the Target value Target comprises the following steps:

firstly, calculating the average values of all pixel values in specific pixel rows of the area 1 and the area 2, and respectively recording the average values as AP1 and AP2;

if Threshold1 is less than or equal to AP2 is less than or equal to Threshold2, then

Target＝AP2；

If AP2> Threshold2 or AP2< Threshold1, the average of all pixel values in a particular pixel row of region 3 is calculated, denoted as AP3,

Target＝(AP1+AP2+AP3)/3；

wherein Threshold1 is a lower Threshold, threshold1=128-t; threshold2 is an upper Threshold, threshold2=128+ t, where t is greater than or equal to 4 and less than or equal to 32.

2. The fast reacting vehicle HDR method as claimed in claim 1, wherein in the step (2), the weight matrix is obtained by:

if P _i,j Equal to or greater than Target, then A _i,j =1; if P _i,j < Target, then A _i,j ＝0；

Wherein, P _i,j Is the pixel value of the ith row and jth column pixel point, A _i,j And for the weighted values of the ith row and jth column of pixel points, the weighted values of all the pixels in the sampling frame form a weighted matrix.

3. The fast reactive vehicle HDR method as claimed in claim 1, wherein in the step (2), the weight matrix is obtained by:

if P _i,j Greater than or equal to Target + V, then A _i,j ＝3；

If Target is less than or equal to P _i,j < Target + V, then A _i,j ＝2；

If Target-V is less than or equal to P _i,j < Target, then A _i,j ＝1；

If P is _i,j < Target-V, then A _i,j ＝0；

Wherein, P _i,j Is the pixel value of the ith row and jth column pixel point, A _i,j And the weight values of the ith row and the jth column of pixel points are the weight values, V is a refinement coefficient and is less than or equal to 32, and the weight values of all pixels in the sampling frame form a weight matrix.

4. The fast reacting vehicle HDR method as claimed in claim 1, wherein in the step (2), the weight matrix is obtained by:

dividing n multiplied by n adjacent pixel points into a whole, taking the average pixel value of all the pixel points in the whole as the pixel value of any pixel point in the whole, wherein n is a common divisor of the row number and the column number of a sampling frame; all pixel points in the sampling frame are correspondingly divided;

if P is _i,j Greater than or equal to Target, then A _i,j =1; if P _i,j < Target, then A _i,j ＝0；

Wherein, P _i,j Is the pixel value of the ith row and jth column pixel point, A _i,j And for the weighted values of the ith row and jth column of pixel points, the weighted values of all the pixels in the sampling frame form a weighted matrix.

5. The fast-response vehicle HDR method as claimed in claim 2 or 4, wherein in step (3), the method for adjusting the pixel values in the operation frame is:

if the weighted value of the pixel point in the weight matrix is 1, subtracting the compensation value VP1 from the pixel value; if the weighted value of the pixel point is 0, adding the compensation value VP1 to the pixel value; the selected VP1 ensures that the pixel values after operation are all between 8 and 250; and operating all pixel points in the operation frame to obtain an adjusted operation frame image.

6. The fast-response vehicle HDR method as claimed in claim 3, wherein in step (3), the method for adjusting the pixel value in the operation frame is:

if the weighted value of the pixel point in the weight matrix is 3, subtracting the compensation value VP2 from the pixel value; if the weighted value of the pixel point is 2, subtracting a compensation value VP3 from the pixel value; if the weighted value of the pixel point is 1, adding the pixel value of the pixel point with a compensation value VP3; if the weighted value of the pixel point is 0, adding the pixel value of the pixel point with a compensation value VP2; wherein VP2 is larger than VP3, and the selected VP2 and VP3 ensure that the pixel values after operation are both between 8 and 250; and operating all pixel points in the operation frame to obtain an adjusted operation frame image.

Images