CN113781317A - Brightness equalization method of panoramic all-around system - Google Patents

Brightness equalization method of panoramic all-around system Download PDF

Info

Publication number
CN113781317A
CN113781317A CN202110880913.7A CN202110880913A CN113781317A CN 113781317 A CN113781317 A CN 113781317A CN 202110880913 A CN202110880913 A CN 202110880913A CN 113781317 A CN113781317 A CN 113781317A
Authority
CN
China
Prior art keywords
optimal
camera
cameras
value
view
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110880913.7A
Other languages
Chinese (zh)
Other versions
CN113781317B (en
Inventor
林耀荣
曾赞云
郑晓雯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN202110880913.7A priority Critical patent/CN113781317B/en
Publication of CN113781317A publication Critical patent/CN113781317A/en
Application granted granted Critical
Publication of CN113781317B publication Critical patent/CN113781317B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a brightness balancing method of a panoramic all-round system, which comprises the following steps: firstly, only processing the brightness Y component of an input view, carrying out block average downsampling on an overlapping area of adjacent camera views, selecting an interior point, and using an obtained interior point sample index set for calculating optimal additive gain; then, brightness equalization pretreatment is carried out by utilizing the optimal additive gain; and finally, reselecting the preprocessed interior point samples, calculating an optimal linear piecewise mapping function, and carrying out brightness equalization post-processing. The method combines the additive gain model and the optimal linear piecewise mapping function, only processes the brightness component of the input view, reduces the computational complexity and improves the image splicing and fusion effect.

Description

一种全景环视系统的亮度均衡方法A Brightness Equalization Method for Panoramic Surround View System

技术领域technical field

本发明涉及数字图像处理技术领域，具体涉及一种全景环视系统的亮度均衡方法。The invention relates to the technical field of digital image processing, in particular to a brightness equalization method of a panoramic surround view system.

背景技术Background technique

近年由于汽车数量的增长，停车场车位紧张，泊车空间狭窄，驾驶员在倒车入库、侧方位停车时，由于汽车左、右后视镜存在视野盲区，给驾驶员造成泊车障碍。全景环视系统可以展示车身四周环境的环视图像，给驾驶员提供车身四周的路面信息，辅助驾驶员进行驾驶判断，提高车辆在泊车时和复杂环境中的安全性能。In recent years, due to the increase in the number of cars, the parking spaces in the parking lot are tight and the parking space is narrow. When the driver reverses the car into the garage and parks on the side, the left and right rearview mirrors of the car have blind spots, causing parking obstacles for the driver. The panoramic surround view system can display the surround view images of the surrounding environment of the vehicle, provide the driver with road information around the vehicle body, assist the driver in making driving judgments, and improve the safety performance of the vehicle when parking and in complex environments.

由于全景环视系统中不同相机的光照条件不同，每个相机自动曝光(AE)和自动白平衡(AWB)参数也不同，因此合成的环视视图在相邻视图之间存在明显的边界，影响视觉效果，更会影响驾驶员对路面状况的判断。Since the lighting conditions of different cameras in the panoramic surround view system are different, and the parameters of automatic exposure (AE) and automatic white balance (AWB) of each camera are also different, the synthetic surround view has obvious boundaries between adjacent views, which affects the visual effect. , it will affect the driver's judgment of the road conditions.

发明内容SUMMARY OF THE INVENTION

本发明的目的是为了解决现有技术中的上述缺陷，提供一种全景环视系统的亮度均衡方法，降低亮度均衡算法的计算复杂度，有利于嵌入式系统实时实现。The purpose of the present invention is to solve the above-mentioned defects in the prior art, and provide a brightness equalization method for a panoramic surround view system, which reduces the computational complexity of the brightness equalization algorithm and is beneficial to the real-time implementation of an embedded system.

本发明的目的可以通过采取如下技术方案达到：The purpose of the present invention can be achieved by adopting the following technical solutions:

一种全景环视系统的亮度均衡方法，所述亮度均衡方法包括以下步骤：A brightness equalization method for a panoramic surround view system, the brightness equalization method comprising the following steps:

S1、图像格式转换：将输入的全景环视系统的P个相机的环视鸟瞰图转换为YUV格式，检查输入的全景环视系统的P个相机的环视鸟瞰图的格式，如果输入环视鸟瞰图不是YUV格式，将P个环视鸟瞰图进行颜色空间转换，转换为YUV格式，得到P个相机的输入视图Im，对应的Y、U、V分量分别为

Figure BDA0003191976740000021
其中m为相机索引，m＝0，1，...，P-1；如果输入环视鸟瞰图是YUV格式，直接得到输入视图Im和对应的Y、U、V分量
Figure BDA0003191976740000022
和
Figure BDA0003191976740000023
S1. Image format conversion: Convert the bird's-eye views of the input P cameras of the panoramic surround view system to YUV format, and check the format of the bird's-eye views of the P cameras of the input panoramic view system. If the input bird's-eye view is not in YUV format , the P look-around bird's-eye views are converted into color space, converted into YUV format, and the input views Im of P cameras are obtained, and the corresponding Y, U, and V components are respectively
Figure BDA0003191976740000021
where m is the camera index, m=0, 1, .
Figure BDA0003191976740000022
and
Figure BDA0003191976740000023

S2、块平均降采样：只考虑亮度Y分量，相机m的输入视图

Figure BDA0003191976740000024
与相邻相机n的输入视图
Figure BDA0003191976740000025
的重叠区域图像记为Ωmn，其中，n为按顺时针方向的下一个相机索引，n≡(m+1)modP，符号“mod”表示求模运算，符号“≡”表示求模运算中的恒等于；相机n的输入视图
Figure BDA0003191976740000026
与相邻相机m的输入视图
Figure BDA0003191976740000027
的重叠区域图像记为Ωnm，对每组相邻相机的重叠区域图像Ωmn和Ωnm分别分块，每个子块大小为N×N个像素，取每个子块的平均值作为对应子块的输出，得到由子块的平均值组成的降采样图像Φmn和Φnm；S2, block average downsampling: only consider the luminance Y component, the input view of camera m
Figure BDA0003191976740000024
Input view with neighboring camera n
Figure BDA0003191976740000025
The image of the overlapping area of is denoted as Ω mn , where n is the next camera index in the clockwise direction, n≡(m+1)modP, the symbol "mod" represents the modulo operation, and the symbol "≡" represents the modulo operation. is equal to; the input view of camera n
Figure BDA0003191976740000026
Input view with neighboring camera m
Figure BDA0003191976740000027
The overlapping area image of , is denoted as Ω nm , and the overlapping area images Ω mn and Ω nm of each group of adjacent cameras are divided into blocks, each sub-block size is N×N pixels, and the average value of each sub-block is taken as the corresponding sub-block The output of , obtains down-sampled images Φmn and Φnm consisting of the average value of the sub-blocks;

S3、样本选择，选择差异较小的内点，获得样本索引集合；基于每组相邻相机重叠区域的降采样图像Φmn和Φnm的对应像素灰度值的差异，选择差异较小的内点的坐标组成样本索引集合Smn：S3. Sample selection, select inliers with small differences to obtain a sample index set; based on the difference between the gray values of the corresponding pixels of the down-sampled images Φmn and Φnm in the overlapping area of each group of adjacent cameras, select the inliers with small differences. The coordinates of the points make up the sample index set S mn :

Smn＝{[i，j]|(Φmn[i，j]-Φnm[i，j])2＜Th，[i，j]∈[1，Nmn]×[1，Mmn]}S mn = {[i, j]|(Φ mn [i, j]-Φ nm [i, j]) 2 <Th, [i, j]∈[1, N mn ]×[1, M mn ] }

其中，i、j分别表示图像的横坐标和纵坐标，[i，j]表示图像的坐标，Φmn[i，j]表示降采样图像Φmn对应坐标[i，j]的像素灰度值，Φnm[i，j]表示降采样图像Φnm对应坐标[i，j]的像素灰度值，Th表示内点阈值，Nmn和Mmn分别表示降采样图像Φmn和Φnm的列数和行数；Among them, i and j represent the abscissa and ordinate of the image respectively, [i, j] represent the coordinates of the image, Φ mn [i, j] represents the pixel gray value of the down-sampled image Φ mn corresponding to the coordinates [i, j] , Φ nm [i, j] represents the pixel gray value of the down-sampled image Φ nm corresponding to coordinates [i, j], Th represents the interior point threshold, N mn and M mn represent the columns of the down-sampled image Φ mn and Φ nm , respectively number and number of lines;

S4、计算P个相机的最优加性增益

Figure BDA0003191976740000028
使得调整后的相邻相机重叠区域的降采样图像Φmn和Φnm的对应内点像素灰度值的均方误差最小，满足以下公式：S4. Calculate the optimal additive gain of P cameras
Figure BDA0003191976740000028
The mean square error of the corresponding inner pixel gray values of the adjusted down-sampled images Φ mn and Φ nm in the overlapping area of adjacent cameras is minimized, and the following formula is satisfied:

Figure BDA0003191976740000031
Figure BDA0003191976740000031

其中，gm为第m个相机的加性增益，

Figure BDA0003191976740000032
表示第m个相机的最优加性增益；where g m is the additive gain of the mth camera,
Figure BDA0003191976740000032
represents the optimal additive gain of the mth camera;

S5、加性增益模型亮度均衡预处理，每个相机的输入视图的Y分量加上最优加性增益进行亮度均衡预处理，公式如下：S5. Additive gain model brightness equalization preprocessing. The Y component of the input view of each camera is added with the optimal additive gain to perform brightness equalization preprocessing. The formula is as follows:

Figure BDA0003191976740000033
Figure BDA0003191976740000033

其中，

Figure BDA0003191976740000034
表示第m个输入视图的Y分量经最优加性增益预处理后的输出视图；in,
Figure BDA0003191976740000034
represents the output view of the Y component of the mth input view preprocessed by the optimal additive gain;

S6、修正降采样图像，每个降采样图像Φmn和Φnm分别加上对应的最优加性增益，公式如下：S6. Correct the down-sampling image, add the corresponding optimal additive gain to each down-sampling image Φ mn and Φ nm respectively, the formula is as follows:

Figure BDA0003191976740000035
Figure BDA0003191976740000035

Figure BDA0003191976740000036
Figure BDA0003191976740000036

其中，

Figure BDA0003191976740000037
和
Figure BDA0003191976740000038
分别表示降采样图像Φmn和Φnm经最优加性增益修正后的降采样图像；in,
Figure BDA0003191976740000037
and
Figure BDA0003191976740000038
represent the down-sampled images Φ mn and Φ nm corrected by the optimal additive gain, respectively;

S7、更新样本索引集合，使用步骤S3中方法重新获得相邻相机重叠区域的修正降采样图像

Figure BDA0003191976740000039
和
Figure BDA00031919767400000310
的内点的样本索引集合
Figure BDA00031919767400000311
满足以下公式：S7, update the sample index set, and use the method in step S3 to re-obtain the corrected down-sampling image of the overlapping area of the adjacent cameras
Figure BDA0003191976740000039
and
Figure BDA00031919767400000310
A collection of sample indices for interior points of
Figure BDA00031919767400000311
The following formulas are satisfied:

Figure BDA00031919767400000312
Figure BDA00031919767400000312

其中，

Figure BDA00031919767400000313
表示修正降采样图像
Figure BDA00031919767400000314
对应坐标[i，j]的像素灰度值，
Figure BDA00031919767400000315
表示修正降采样图像
Figure BDA00031919767400000316
对应坐标[i，j]的像素灰度值；in,
Figure BDA00031919767400000313
Represents a corrected downsampled image
Figure BDA00031919767400000314
the pixel gray value corresponding to the coordinates [i, j],
Figure BDA00031919767400000315
Represents a corrected downsampled image
Figure BDA00031919767400000316
The pixel gray value corresponding to the coordinates [i, j];

S8、计算最优映射曲线函数，第m个相机的映射曲线函数y＝Tm(x)将[0，255]之间的输入像素值x映射到[0，255]之间的输出值y，P个相机的最优映射曲线函数Tm()通过最优化下面的公式得到：S8. Calculate the optimal mapping curve function. The mapping curve function y=T m (x) of the mth camera maps the input pixel value x between [0, 255] to the output value y between [0, 255] , the optimal mapping curve function T m () of the P cameras is obtained by optimizing the following formula:

Figure BDA0003191976740000041
Figure BDA0003191976740000041

其中，Tm()表示第m个相机的映射曲线函数，

Figure BDA0003191976740000042
表示对应的最优映射曲线函数，β为惩罚因子，约束映射曲线函数的输入值和输出值的偏离程度；Among them, T m ( ) represents the mapping curve function of the mth camera,
Figure BDA0003191976740000042
Represents the corresponding optimal mapping curve function, β is the penalty factor, which constrains the degree of deviation between the input value and the output value of the mapping curve function;

S9、最优映射曲线函数亮度均衡后处理，对P个相机的最优加性增益预处理视图

Figure BDA0003191976740000043
通过最优映射曲线函数进行亮度均衡后处理，得到输出环视鸟瞰图的Y分量
Figure BDA0003191976740000044
公式如下：S9, the optimal mapping curve function brightness equalization post-processing, the optimal additive gain preprocessing view for the P cameras
Figure BDA0003191976740000043
The brightness equalization post-processing is carried out through the optimal mapping curve function, and the Y component of the output look-around bird's-eye view is obtained.
Figure BDA0003191976740000044
The formula is as follows:

Figure BDA0003191976740000045
Figure BDA0003191976740000045

最后得到P个相机经亮度均衡处理后的输出环视鸟瞰图的Y、U、V分量分别为

Figure BDA0003191976740000046
Finally, the Y, U, and V components of the output look-around bird's-eye view of the P cameras after brightness equalization processing are respectively:
Figure BDA0003191976740000046

进一步地，所述亮度均衡方法在经过步骤S9中最优映射曲线函数亮度均衡后处理得到输出环视鸟瞰图的Y分量

Figure BDA0003191976740000047
之后，还包括以下步骤：Further, the brightness equalization method obtains the Y component of the output look-around bird's-eye view after the brightness equalization of the optimal mapping curve function in step S9.
Figure BDA0003191976740000047
After that, the following steps are also included:

S10、图像格式还原，如果输入的环视鸟瞰图不是YUV格式，将亮度均衡处理后的输出环视鸟瞰图进行颜色空间转换，转换为原来输入的格式。S10, restore the image format, if the input look-around bird's-eye view is not in YUV format, convert the output look-around bird's-eye view after brightness equalization processing to the color space, and convert it to the original input format.

进一步地，所述步骤S8中最优映射曲线函数计算过程如下：Further, the calculation process of the optimal mapping curve function in the step S8 is as follows:

S801、第m个相机的映射曲线函数使用线性分段映射函数表示，线性分段映射函数使用一组锚点定义，锚点共d+1个，第k个锚点的坐标为

Figure BDA0003191976740000048
k为锚点索引，k＝0，1，...，d，第m个相机的映射曲线函数Tm()可以表示为：S801. The mapping curve function of the mth camera is represented by a linear piecewise mapping function, the linear piecewise mapping function is defined by a set of anchor points, there are d+1 anchor points in total, and the coordinate of the kth anchor point is
Figure BDA0003191976740000048
k is the anchor point index, k=0, 1, . . . , d, the mapping curve function T m ( ) of the m-th camera can be expressed as:

Figure BDA0003191976740000051
Figure BDA0003191976740000051

其中，x表示输入像素值，Tm(x)表示映射曲线函数的输出映射像素值；where x represents the input pixel value, and T m (x) represents the output mapped pixel value of the mapping curve function;

S802、第m个相机的最优线性分段映射函数的第0个锚点固定为

Figure BDA0003191976740000052
第d个锚点固定为
Figure BDA0003191976740000053
根据修正降采样图像
Figure BDA0003191976740000054
和
Figure BDA0003191976740000055
的内点的像素灰度值的分布范围，确定图像像素灰度值的最小值和最大值，得到其他锚点的横坐标范围为
Figure BDA0003191976740000056
其中
Figure BDA0003191976740000057
和
Figure BDA0003191976740000058
通过下式确定：S802, the 0th anchor point of the optimal linear piecewise mapping function of the mth camera is fixed as
Figure BDA0003191976740000052
The d-th anchor point is fixed as
Figure BDA0003191976740000053
Downsample the image according to the correction
Figure BDA0003191976740000054
and
Figure BDA0003191976740000055
The distribution range of the pixel gray value of the inner point, determine the minimum and maximum value of the pixel gray value of the image, and obtain the abscissa range of other anchor points as
Figure BDA0003191976740000056
in
Figure BDA0003191976740000057
and
Figure BDA0003191976740000058
Determined by:

Figure BDA0003191976740000059
Figure BDA0003191976740000059

其中，l为按顺时针方向的前一个相机索引，l≡(m-1)mod P，

Figure BDA00031919767400000510
和
Figure BDA00031919767400000511
分别对应第m个相机按顺时针方向的两个重叠区域的修正降采样图像，
Figure BDA00031919767400000512
和
Figure BDA00031919767400000513
分别为
Figure BDA00031919767400000514
和
Figure BDA00031919767400000515
的内点的样本索引集合，max()表示取最大值运算，min()表示取最小值运算；Among them, l is the previous camera index in the clockwise direction, l≡(m-1)mod P,
Figure BDA00031919767400000510
and
Figure BDA00031919767400000511
respectively correspond to the corrected down-sampled images of the two overlapping regions of the mth camera in the clockwise direction,
Figure BDA00031919767400000512
and
Figure BDA00031919767400000513
respectively
Figure BDA00031919767400000514
and
Figure BDA00031919767400000515
The sample index set of the interior points of , max() represents the operation of taking the maximum value, and min() represents the operation of taking the minimum value;

其他锚点

Figure BDA00031919767400000516
均匀地分布在横坐标范围
Figure BDA00031919767400000517
内，通过下式确定：other anchors
Figure BDA00031919767400000516
Evenly distributed in the abscissa range
Figure BDA00031919767400000517
is determined by the following formula:

Figure BDA00031919767400000518
Figure BDA00031919767400000518

其中，

Figure BDA00031919767400000519
为第m个相机的第k个锚点的横坐标；in,
Figure BDA00031919767400000519
is the abscissa of the k-th anchor point of the m-th camera;

S803、计算样本索引集合

Figure BDA00031919767400000520
和
Figure BDA00031919767400000521
第m个相机的第k个锚点纵坐标
Figure BDA00031919767400000522
的优化，只涉及像素灰度值在
Figure BDA00031919767400000523
区间的修正降采样图像
Figure BDA00031919767400000524
和
Figure BDA00031919767400000525
的内点；分别选择像素灰度值在
Figure BDA00031919767400000526
区间的修正降采样图像
Figure BDA00031919767400000527
和
Figure BDA00031919767400000528
的内点的坐标组成样本索引集合
Figure BDA00031919767400000529
和
Figure BDA00031919767400000530
S803. Calculate the sample index set
Figure BDA00031919767400000520
and
Figure BDA00031919767400000521
The ordinate of the kth anchor point of the mth camera
Figure BDA00031919767400000522
The optimization involves only the pixel gray value in the
Figure BDA00031919767400000523
Corrected downsampled image of the interval
Figure BDA00031919767400000524
and
Figure BDA00031919767400000525
The interior points of ; respectively select the pixel gray value in
Figure BDA00031919767400000526
Corrected downsampled image of the interval
Figure BDA00031919767400000527
and
Figure BDA00031919767400000528
The coordinates of the interior points make up the sample index set
Figure BDA00031919767400000529
and
Figure BDA00031919767400000530

Figure BDA00031919767400000531
Figure BDA00031919767400000531

Figure BDA00031919767400000532
Figure BDA00031919767400000532

其中，

Figure BDA0003191976740000061
表示修正降采样图像
Figure BDA0003191976740000062
对应坐标[i，j]的像素灰度值，
Figure BDA0003191976740000063
和
Figure BDA0003191976740000064
分别为
Figure BDA0003191976740000065
和
Figure BDA0003191976740000066
的内点的样本索引集合，
Figure BDA0003191976740000067
表示第m个相机的第k-1个锚点的横坐标，
Figure BDA0003191976740000068
表示第k+1个锚点的横坐标；in,
Figure BDA0003191976740000061
Represents a corrected downsampled image
Figure BDA0003191976740000062
the pixel gray value corresponding to the coordinates [i, j],
Figure BDA0003191976740000063
and
Figure BDA0003191976740000064
respectively
Figure BDA0003191976740000065
and
Figure BDA0003191976740000066
the set of sample indices of the interior points of ,
Figure BDA0003191976740000067
represents the abscissa of the k-1th anchor point of the mth camera,
Figure BDA0003191976740000068
Represents the abscissa of the k+1th anchor point;

S804、按顺序依次迭代求解第m个相机的最优线性分段映射函数的d-1个锚点纵坐标

Figure BDA0003191976740000069
得到第m个相机的最优线性分段映射函数；S804, iteratively solve the ordinates of the d-1 anchor points of the optimal linear piecewise mapping function of the mth camera in sequence
Figure BDA0003191976740000069
Obtain the optimal linear piecewise mapping function of the mth camera;

S805、重复执行步骤S804，直到所有相机的所有锚点纵坐标的前后两次迭代变化量的绝对值小于设定阈值Tanchor，停止迭代，输出由锚点确定的最优线性分段映射函数作为最优映射曲线函数。S805. Repeat step S804 until the absolute value of the change amount of the two iterations before and after the ordinates of all anchor points of all cameras is less than the set threshold T anchor , stop the iteration, and output the optimal linear piecewise mapping function determined by the anchor points as Optimal mapping curve function.

进一步地，所述步骤S804中按顺序依次迭代求解第m个相机的最优线性分段映射函数的d-1个锚点纵坐标

Figure BDA00031919767400000610
得到第m个相机的最优线性分段映射函数，过程如下：Further, in the step S804, iteratively solves the ordinates of the d-1 anchor points of the optimal linear piecewise mapping function of the mth camera in sequence.
Figure BDA00031919767400000610
To obtain the optimal linear piecewise mapping function of the mth camera, the process is as follows:

S80401、固定其他锚点的纵坐标，更新第m个相机的最优线性分段映射函数的奇数组的锚点的纵坐标

Figure BDA00031919767400000611
S80401. Fix the ordinate of other anchor points, and update the ordinate of the anchor point of the odd group of the optimal linear piecewise mapping function of the mth camera
Figure BDA00031919767400000611

第k个锚点的纵坐标

Figure BDA00031919767400000612
的更新计算公式如下：The ordinate of the kth anchor point
Figure BDA00031919767400000612
The update calculation formula of is as follows:

Figure BDA00031919767400000613
Figure BDA00031919767400000613

其中，A、B、C均为中间计算变量，定义如下：Among them, A, B, and C are all intermediate calculation variables, which are defined as follows:

Figure BDA00031919767400000614
Figure BDA00031919767400000614

Figure BDA0003191976740000071
Figure BDA0003191976740000071

其中，

Figure BDA0003191976740000072
和
Figure BDA0003191976740000073
为相机m和n重叠区域的修正降采样图像，
Figure BDA0003191976740000074
和
Figure BDA0003191976740000075
为相机l和m重叠区域的修正降采样图像，Tn()和Tl()分别为第n，l个相机的映射曲线函数，函数Fmk()和Y’mk()为分段函数，定义如下：in,
Figure BDA0003191976740000072
and
Figure BDA0003191976740000073
is the corrected downsampled image for the overlapping area of cameras m and n,
Figure BDA0003191976740000074
and
Figure BDA0003191976740000075
is the corrected down-sampling image of the overlapping area of cameras l and m, T n () and T l () are the mapping curve functions of the nth and lth cameras, respectively, and the functions F mk () and Y' mk () are piecewise functions , defined as follows:

Figure BDA0003191976740000076
Figure BDA0003191976740000076

Figure BDA0003191976740000077
Figure BDA0003191976740000077

S80402、固定其他锚点的纵坐标，更新第m个相机的最优线性分段映射函数的偶数组的锚点的纵坐标

Figure BDA0003191976740000078
S80402. Fix the ordinate of other anchor points, and update the ordinate of the anchor point of the even group of the optimal linear piecewise mapping function of the mth camera
Figure BDA0003191976740000078

本发明相对于现有技术具有如下的优点及效果：Compared with the prior art, the present invention has the following advantages and effects:

1、现有技术需要处理全景环视系统环视鸟瞰图的所有颜色分量通道，本发明只对亮度Y分量进行亮度均衡，计算量是现有技术的1/3，降低了算法的计算复杂度。1. The prior art needs to deal with all the color component channels of the panoramic bird's-eye view of the panoramic view system. The present invention only performs luminance equalization on the luminance Y component, and the calculation amount is 1/3 of the prior art, which reduces the computational complexity of the algorithm.

2、现有技术采用加性增益模型，具有显式数学解，本发明结合加性增益模型和最优线性分段映射函数，利用加性增益模型进行预处理，利用最优线性分段映射函数进行后处理，进一步改善亮度均衡效果，降低了全景环视系统不同相机视图重叠区域的亮度差异。2. The prior art adopts an additive gain model and has an explicit mathematical solution. The present invention combines the additive gain model and the optimal linear piecewise mapping function, utilizes the additive gain model for preprocessing, and utilizes the optimal linear piecewise mapping function. Post-processing is performed to further improve the brightness balance effect and reduce the brightness difference in the overlapping areas of different camera views of the panoramic surround view system.

附图说明Description of drawings

图1是本发明公开的全景环视系统的亮度均衡方法的步骤流程图；Fig. 1 is the step flow chart of the brightness equalization method of the panoramic surround view system disclosed by the present invention;

图2是本发明公开的块平均降采样示意图；2 is a schematic diagram of block average downsampling disclosed in the present invention;

图3是本发明公开的线性分段映射函数的示意图。FIG. 3 is a schematic diagram of the linear piecewise mapping function disclosed in the present invention.

具体实施方式Detailed ways

为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

实施例Example

本实施例公开了一种全景环视系统的亮度均衡方法，如图1所示，包括以下步骤：This embodiment discloses a brightness equalization method for a panoramic surround view system, as shown in FIG. 1 , including the following steps:

S1、图像格式转换：检查输入的全景环视系统的P个相机的环视鸟瞰图的格式，如果输入环视鸟瞰图不是YUV格式，将P个环视鸟瞰图进行颜色空间转换，转换为YUV格式，得到P个相机的输入视图Im，对应的Y、U、V分量分别为

Figure BDA0003191976740000081
其中m为相机索引，m＝0，1，...，P-1；如果输入环视鸟瞰图是YUV格式，直接得到输入视图Im和对应的Y、U、V分量
Figure BDA0003191976740000082
和
Figure BDA0003191976740000083
在本实施例中，P取4。S1. Image format conversion: Check the format of the surround-view bird's-eye view of the P cameras of the input panoramic surround-view system. If the input surround-view bird's-eye view is not in YUV format, perform color space conversion on the P surround-view bird's-eye views, convert them to YUV format, and get P The input view Im of each camera, the corresponding Y, U, V components are respectively
Figure BDA0003191976740000081
where m is the camera index, m=0, 1, .
Figure BDA0003191976740000082
and
Figure BDA0003191976740000083
In this embodiment, P is 4.

S2、块平均降采样：如图2所示，只考虑亮度Y分量，相机m的输入视图

Figure BDA0003191976740000084
与相邻相机n的输入视图
Figure BDA0003191976740000085
的重叠区域图像记为Ωmn，其中，m为相机索引，n为按顺时针方向的下一个相机索引，n≡(m+1)mod P，符号“mod”表示求模运算，符号“≡”表示求模运算中的恒等于；相机n的输入视图
Figure BDA0003191976740000086
与相邻相机m的输入视图
Figure BDA0003191976740000087
的重叠区域图像记为Ωnm，对每组相邻相机的重叠区域图像Ωmn和Ωnm分别分块，每个子块大小为N×N个像素，取每个子块的平均值作为对应子块的输出，得到由子块的平均值组成的降采样图像Φmn和Φnm；在本实施例中，N取4。S2, block average downsampling: As shown in Figure 2, only the luminance Y component is considered, the input view of camera m
Figure BDA0003191976740000084
Input view with neighboring camera n
Figure BDA0003191976740000085
The image of the overlapping area of is denoted as Ω mn , where m is the camera index, n is the next camera index in the clockwise direction, n≡(m+1)mod P, the symbol "mod" represents the modulo operation, and the symbol "≡ ” means constant in the modulo operation; the input view of camera n
Figure BDA0003191976740000086
Input view with neighboring camera m
Figure BDA0003191976740000087
The overlapping area image of , is denoted as Ω nm , and the overlapping area images Ω mn and Ω nm of each group of adjacent cameras are divided into blocks, each sub-block size is N×N pixels, and the average value of each sub-block is taken as the corresponding sub-block , the down-sampled images Φ mn and Φ nm composed of the average value of the sub-blocks are obtained; in this embodiment, N is 4.

S3、样本选择，选择差异较小的内点，获得样本索引集合；基于每组相邻相机重叠区域的降采样图像Φmn和Φnm的对应像素灰度值的差异，选择差异较小的内点的坐标组成样本索引集合Smn：S3. Sample selection, select inliers with small differences to obtain a sample index set; based on the difference between the gray values of the corresponding pixels of the down-sampled images Φmn and Φnm in the overlapping area of each group of adjacent cameras, select the inliers with small differences. The coordinates of the points make up the sample index set S mn :

Smn＝{[i，j]|(Φmn[i，j]-Φmm[i，j])2＜Th，[i，j]∈[1，Nmn]×[1，Mmn]}S mn = {[i, j]|(Φ mn [i, j]-Φ mm [i, j]) 2 <Th, [i, j]∈[1, N mn ]×[1, M mn ] }

其中，i、j分别表示图像的横坐标和纵坐标，[i，j]表示图像的坐标，Φmn[i，j]表示降采样图像Φmn对应坐标[i，j]的像素灰度值，Φnm[i，j]表示降采样图像Φnm对应坐标[i，j]的像素灰度值，Th表示内点阈值，Nmn和Mmn分别表示降采样图像Φmn和Φnm的列数和行数；在本实施例中，Th取500。Among them, i and j represent the abscissa and ordinate of the image respectively, [i, j] represent the coordinates of the image, Φ mn [i, j] represents the pixel gray value of the down-sampled image Φ mn corresponding to the coordinates [i, j] , Φ nm [i, j] represents the pixel gray value of the down-sampled image Φ nm corresponding to coordinates [i, j], Th represents the interior point threshold, N mn and M mn represent the columns of the down-sampled image Φ mn and Φ nm , respectively number and number of rows; in this embodiment, Th is taken as 500.

S4、计算P个相机的最优加性增益

Figure BDA0003191976740000091
使得调整后的相邻相机重叠区域的降采样图像Φmn和Φnm的对应内点像素灰度值的均方误差最小，满足以下公式：S4. Calculate the optimal additive gain of P cameras
Figure BDA0003191976740000091
The mean square error of the corresponding inner pixel gray values of the adjusted down-sampled images Φ mn and Φ nm in the overlapping area of adjacent cameras is minimized, and the following formula is satisfied:

Figure BDA0003191976740000092
Figure BDA0003191976740000092

其中，gm为第m个相机的加性增益，

Figure BDA0003191976740000093
表示对应的最优加性增益，Smn表示相邻相机重叠区域的降采样图像Φmn和Φnm的内点的样本索引集合；where g m is the additive gain of the mth camera,
Figure BDA0003191976740000093
represents the corresponding optimal additive gain, and S mn represents the sample index set of the interior points of the down-sampled images Φ mn and Φ nm of the overlapping area of adjacent cameras;

P个相机的最优加性增益

Figure BDA0003191976740000094
的计算流程如下：Optimal additive gain for P cameras
Figure BDA0003191976740000094
The calculation process is as follows:

S401、选择具有中等亮度的相机输入视图作为参考视图，参考视图的索引ref的计算方法如下：S401. Select a camera input view with medium brightness as a reference view, and the calculation method of the index ref of the reference view is as follows:

S40101、计算降采样图像Φmn和Φnm的内点的样本均值Amn和Anm：S40101. Calculate the sample mean values A mn and A nm of the interior points of the down-sampled images Φ mn and Φ nm :

Figure BDA0003191976740000095
Figure BDA0003191976740000095

Figure BDA0003191976740000101
Figure BDA0003191976740000101

其中，Φmn[i，j]表示降采样图像Φmn对应坐标[i，j]的像素灰度值，Φnm[i，j]表示降采样图像Φnm对应坐标[i，j]的像素灰度值，ρmn表示样本索引集合Smn的索引数量；Among them, Φ mn [i, j] represents the pixel gray value of the down-sampled image Φ mn corresponding to coordinates [i, j], Φ nm [i, j] represents the pixel of the down-sampled image Φ nm corresponding to coordinates [i, j] Gray value, ρ mn represents the index number of the sample index set S mn ;

S40102、计算第m个相机的输入视图Y分量

Figure BDA0003191976740000102
的重叠区域亮度均值Am：S40102. Calculate the Y component of the input view of the mth camera
Figure BDA0003191976740000102
The average brightness of the overlapping area Am :

Am＝Amn+Aml A m =A mn +A ml

其中，n≡(m+1)mod P，l为按顺时针方向的前一个相机索引，l≡(m-1)modP，Amn和Aml分别对应第m个相机按顺时针方向的两个重叠区域的降采样图像Φmn和Φml的内点样本均值；Among them, n≡(m+1)mod P, l is the index of the previous camera in the clockwise direction, l≡(m-1)modP, A mn and A ml respectively correspond to the two clockwise directions of the mth camera. The mean of the inlier samples of the down-sampled images Φmn and Φml of the overlapping regions;

S40103、计算P个相机的亮度均值A：

Figure BDA0003191976740000103
S40103. Calculate the brightness mean value A of the P cameras:
Figure BDA0003191976740000103

S40104、计算参考视图的相机索引ref：

Figure BDA0003191976740000104
将第ref个视图作为参考视图；S40104. Calculate the camera index ref of the reference view:
Figure BDA0003191976740000104
Use the ref-th view as the reference view;

S402、计算降采样图像Φmn和Φnn的内点的样本均值差dm：dm＝Amn-Anm；S402, calculate the sample mean difference d m of the inner points of the down-sampled images Φ mn and Φ nn : d m =A mn −A nm ;

S403、计算所有相机的样本均值差的平均值d：

Figure BDA0003191976740000105
S403. Calculate the average value d of the sample mean difference of all cameras:
Figure BDA0003191976740000105

S404、最优加性增益

Figure BDA0003191976740000106
通过以下公式计算：S404. Optimal additive gain
Figure BDA0003191976740000106
Calculated by the following formula:

Figure BDA0003191976740000107
Figure BDA0003191976740000107

其中，ref为参考视图的相机索引，dm为第m个样本均值差；Among them, ref is the camera index of the reference view, and d m is the mean difference of the mth sample;

S5、加性增益模型亮度均衡预处理，每个相机的输入视图的Y分量加上最优加性增益进行亮度均衡预处理，公式如下：S5. Additive gain model brightness equalization preprocessing. The Y component of the input view of each camera is added with the optimal additive gain to perform brightness equalization preprocessing. The formula is as follows:

Figure BDA0003191976740000108
Figure BDA0003191976740000108

其中，

Figure BDA0003191976740000111
表示第m个输入视图的Y分量经最优加性增益预处理后的输出视图，
Figure BDA0003191976740000112
表示第m个相机的最优加性增益；in,
Figure BDA0003191976740000111
represents the output view of the Y component of the mth input view preprocessed by the optimal additive gain,
Figure BDA0003191976740000112
represents the optimal additive gain of the mth camera;

S6、修正降采样图像，每个降采样图像Φmn和Φnm分别加上对应的最优加性增益，公式如下：S6. Correct the down-sampling image, add the corresponding optimal additive gain to each down-sampling image Φ mn and Φ nm respectively, the formula is as follows:

Figure BDA0003191976740000113
Figure BDA0003191976740000113

Figure BDA0003191976740000114
Figure BDA0003191976740000114

其中，

Figure BDA0003191976740000115
和
Figure BDA0003191976740000116
分别表示降采样图像Φmn和Φnm经最优加性增益修正后的降采样图像；in,
Figure BDA0003191976740000115
and
Figure BDA0003191976740000116
represent the down-sampled images Φ mn and Φ nm corrected by the optimal additive gain, respectively;

S7、更新样本索引集合；使用步骤S3中所描述的方法重新获得修正降采样图像

Figure BDA0003191976740000117
和
Figure BDA0003191976740000118
的内点的样本索引集合
Figure BDA0003191976740000119
满足以下公式：S7, update the sample index set; use the method described in step S3 to re-obtain the corrected down-sampling image
Figure BDA0003191976740000117
and
Figure BDA0003191976740000118
A collection of sample indices for interior points of
Figure BDA0003191976740000119
The following formulas are satisfied:

Figure BDA00031919767400001110
Figure BDA00031919767400001110

其中，

Figure BDA00031919767400001111
表示修正降采样图像
Figure BDA00031919767400001112
对应坐标[i，j]的像素灰度值，
Figure BDA00031919767400001113
表示修正降采样图像
Figure BDA00031919767400001114
对应坐标[i，j]的像素灰度值；in,
Figure BDA00031919767400001111
Represents a corrected downsampled image
Figure BDA00031919767400001112
the pixel gray value corresponding to the coordinates [i, j],
Figure BDA00031919767400001113
Represents a corrected downsampled image
Figure BDA00031919767400001114
The pixel gray value corresponding to the coordinates [i, j];

S8、计算最优映射曲线函数，第m个相机的映射曲线函数y＝Tm(x)将[0，255]之间的输入像素值x映射到[0，255]之间的输出值y，P个相机的最优映射曲线函数Tm()通过最优化下面的公式得到：S8. Calculate the optimal mapping curve function. The mapping curve function y=T m (x) of the mth camera maps the input pixel value x between [0, 255] to the output value y between [0, 255] , the optimal mapping curve function T m () of the P cameras is obtained by optimizing the following formula:

Figure BDA00031919767400001115
Figure BDA00031919767400001115

其中，Tm()表示第m个相机的映射曲线函数，

Figure BDA00031919767400001116
表示对应的最优映射曲线函数，n为按顺时针方向的下一个相机索引，β为惩罚因子，约束映射曲线函数的输入值和输出值的偏离程度，
Figure BDA00031919767400001117
和
Figure BDA00031919767400001118
为相邻相机重叠区域的修正降采样图像；Among them, T m ( ) represents the mapping curve function of the mth camera,
Figure BDA00031919767400001116
represents the corresponding optimal mapping curve function, n is the next camera index in the clockwise direction, β is the penalty factor, which constrains the degree of deviation between the input value and the output value of the mapping curve function,
Figure BDA00031919767400001117
and
Figure BDA00031919767400001118
A corrected downsampled image for the overlapping area of adjacent cameras;

P个相机的最优映射曲线函数y＝Tm(x)的计算流程如下：The calculation process of the optimal mapping curve function y=T m (x) of the P cameras is as follows:

S801、第m个相机的映射曲线函数使用线性分段映射函数表示，如图3所示，线性分段映射函数使用一组锚点定义，锚点共d+1个，在本实施例中，d取5，第k个锚点的坐标为

Figure BDA0003191976740000121
k为锚点索引，k＝0，1，...，d，第m个相机的映射曲线函数Tm()可以表示为：S801. The mapping curve function of the mth camera is represented by a linear piecewise mapping function. As shown in FIG. 3, the linear piecewise mapping function is defined by a set of anchor points, and there are d+1 anchor points in total. In this embodiment, d is set to 5, and the coordinates of the kth anchor point are
Figure BDA0003191976740000121
k is the anchor point index, k=0, 1, . . . , d, the mapping curve function T m ( ) of the m-th camera can be expressed as:

Figure BDA0003191976740000122
当
Figure BDA0003191976740000123
Figure BDA0003191976740000122
when
Figure BDA0003191976740000123

其中，m表示相机索引，x表示输入像素值，Tm(x)表示映射曲线函数的输出映射像素值；where m represents the camera index, x represents the input pixel value, and T m (x) represents the output mapped pixel value of the mapping curve function;

S802、第m个相机的最优线性分段映射函数的第0个锚点固定为

Figure BDA0003191976740000124
第d个锚点固定为
Figure BDA0003191976740000125
根据修正降采样图像
Figure BDA0003191976740000126
和
Figure BDA0003191976740000127
的内点的像素灰度值的分布范围，确定图像像素灰度值的最小值和最大值，可以得到其他锚点的横坐标范围为
Figure BDA0003191976740000128
其中
Figure BDA0003191976740000129
和
Figure BDA00031919767400001210
通过下式确定：S802, the 0th anchor point of the optimal linear piecewise mapping function of the mth camera is fixed as
Figure BDA0003191976740000124
The d-th anchor point is fixed as
Figure BDA0003191976740000125
Downsample the image according to the correction
Figure BDA0003191976740000126
and
Figure BDA0003191976740000127
The distribution range of the pixel gray value of the inner point, determine the minimum and maximum value of the pixel gray value of the image, and the abscissa range of other anchor points can be obtained as
Figure BDA0003191976740000128
in
Figure BDA0003191976740000129
and
Figure BDA00031919767400001210
Determined by:

Figure BDA00031919767400001211
Figure BDA00031919767400001211

其中，m为相机索引，n为按顺时针方向的下一个相机索引，n≡(m+1)modP，l为按顺时针方向的前一个相机索引，l≡(m-1)mod P，

Figure BDA00031919767400001212
和
Figure BDA00031919767400001213
分别对应第m个相机按顺时针方向的两个重叠区域的修正降采样图像，
Figure BDA00031919767400001214
和
Figure BDA00031919767400001215
分别为
Figure BDA00031919767400001216
和
Figure BDA00031919767400001217
的内点的样本索引集合，max()表示取最大值运算，min()表示取最小值运算；Among them, m is the camera index, n is the next camera index in the clockwise direction, n≡(m+1)modP, l is the previous camera index in the clockwise direction, l≡(m-1)mod P,
Figure BDA00031919767400001212
and
Figure BDA00031919767400001213
respectively correspond to the corrected down-sampled images of the two overlapping regions of the mth camera in the clockwise direction,
Figure BDA00031919767400001214
and
Figure BDA00031919767400001215
respectively
Figure BDA00031919767400001216
and
Figure BDA00031919767400001217
The sample index set of the interior points of , max() represents the operation of taking the maximum value, and min() represents the operation of taking the minimum value;

其他锚点

Figure BDA00031919767400001218
均匀地分布在横坐标范围
Figure BDA00031919767400001219
内，通过下式确定：other anchors
Figure BDA00031919767400001218
Evenly distributed in the abscissa range
Figure BDA00031919767400001219
is determined by the following formula:

Figure BDA00031919767400001220
Figure BDA00031919767400001220

其中，

Figure BDA00031919767400001221
为第m个相机的第k个锚点的横坐标；in,
Figure BDA00031919767400001221
is the abscissa of the k-th anchor point of the m-th camera;

S803、计算样本索引集合

Figure BDA0003191976740000131
和
Figure BDA0003191976740000132
第m个相机的第k个锚点纵坐标
Figure BDA0003191976740000133
的优化，只涉及像素灰度值在
Figure BDA0003191976740000134
区间的修正降采样图像
Figure BDA0003191976740000135
和
Figure BDA0003191976740000136
的内点；分别选择像素灰度值在
Figure BDA0003191976740000137
区间的修正降采样图像
Figure BDA0003191976740000138
和
Figure BDA0003191976740000139
的内点的坐标组成样本索引集合
Figure BDA00031919767400001310
和
Figure BDA00031919767400001311
S803. Calculate the sample index set
Figure BDA0003191976740000131
and
Figure BDA0003191976740000132
The ordinate of the kth anchor point of the mth camera
Figure BDA0003191976740000133
The optimization involves only the pixel gray value in the
Figure BDA0003191976740000134
Corrected downsampled image of the interval
Figure BDA0003191976740000135
and
Figure BDA0003191976740000136
The interior points of ; respectively select the pixel gray value in
Figure BDA0003191976740000137
Corrected downsampled image of the interval
Figure BDA0003191976740000138
and
Figure BDA0003191976740000139
The coordinates of the interior points make up the sample index set
Figure BDA00031919767400001310
and
Figure BDA00031919767400001311

Figure BDA00031919767400001312
Figure BDA00031919767400001312

Figure BDA00031919767400001313
Figure BDA00031919767400001313

其中，k为锚点索引，k＝1，2，...，d-1，

Figure BDA00031919767400001314
表示修正降采样图像
Figure BDA00031919767400001315
对应坐标[i，j]的像素灰度值，
Figure BDA00031919767400001316
表示修正降采样图像
Figure BDA00031919767400001317
对应坐标[i，j]的像素灰度值，
Figure BDA00031919767400001318
和
Figure BDA00031919767400001319
分别为
Figure BDA00031919767400001320
和
Figure BDA00031919767400001321
的内点的样本索引集合，
Figure BDA00031919767400001322
表示第m个相机的第k-1个锚点的横坐标，
Figure BDA00031919767400001323
表示第k+1个锚点的横坐标；Among them, k is the anchor index, k=1, 2, ..., d-1,
Figure BDA00031919767400001314
Represents a corrected downsampled image
Figure BDA00031919767400001315
the pixel gray value corresponding to the coordinates [i, j],
Figure BDA00031919767400001316
Represents a corrected downsampled image
Figure BDA00031919767400001317
the pixel gray value corresponding to the coordinates [i, j],
Figure BDA00031919767400001318
and
Figure BDA00031919767400001319
respectively
Figure BDA00031919767400001320
and
Figure BDA00031919767400001321
the set of sample indices of the interior points of ,
Figure BDA00031919767400001322
represents the abscissa of the k-1th anchor point of the mth camera,
Figure BDA00031919767400001323
Represents the abscissa of the k+1th anchor point;

S804、按顺序依次迭代求解第m个相机的最优线性分段映射函数，m＝0，1，...，P-1；S804, iteratively solve the optimal linear piecewise mapping function of the mth camera in sequence, m=0, 1, . . . , P-1;

迭代求解第m个相机的最优线性分段映射函数的d-1个锚点纵坐标

Figure BDA00031919767400001324
过程如下：Iteratively solve the d-1 ordinates of the d-1 anchor points of the optimal linear piecewise mapping function of the mth camera
Figure BDA00031919767400001324
The process is as follows:

S80401、固定其他锚点的纵坐标，更新第m个相机的最优线性分段映射函数的奇数组的锚点的纵坐标

Figure BDA00031919767400001325
S80401. Fix the ordinate of other anchor points, and update the ordinate of the anchor point of the odd group of the optimal linear piecewise mapping function of the mth camera
Figure BDA00031919767400001325

第k个锚点的纵坐标

Figure BDA00031919767400001326
的更新计算公式如下：The ordinate of the kth anchor point
Figure BDA00031919767400001326
The update calculation formula of is as follows:

Figure BDA00031919767400001327
Figure BDA00031919767400001327

其中，A、B、C均为中间计算变量，定义如下：Among them, A, B, and C are all intermediate calculation variables, which are defined as follows:

Figure BDA00031919767400001328
Figure BDA00031919767400001328

Figure BDA0003191976740000141
Figure BDA0003191976740000141

其中，

Figure BDA0003191976740000142
和
Figure BDA0003191976740000143
为相机m和n重叠区域的修正降采样图像，
Figure BDA0003191976740000144
和
Figure BDA0003191976740000145
为相机l和m重叠区域的修正降采样图像，Tn()和Tl()分别为第n、l个相机的映射曲线函数，函数Fmk()和Y’mk()为分段函数，定义如下：in,
Figure BDA0003191976740000142
and
Figure BDA0003191976740000143
is the corrected downsampled image for the overlapping area of cameras m and n,
Figure BDA0003191976740000144
and
Figure BDA0003191976740000145
is the modified down-sampled image of the overlapping area of cameras l and m, T n () and T l () are the mapping curve functions of the nth and lth cameras, respectively, and the functions F mk () and Y' mk () are piecewise functions , defined as follows:

Figure BDA0003191976740000146
Figure BDA0003191976740000146

Figure BDA0003191976740000147
Figure BDA0003191976740000147

S80402、固定其他锚点的纵坐标，更新第m个相机的最优线性分段映射函数的偶数组的锚点的纵坐标

Figure BDA0003191976740000148
S80402. Fix the ordinate of other anchor points, and update the ordinate of the anchor point of the even group of the optimal linear piecewise mapping function of the mth camera
Figure BDA0003191976740000148

S805、重复执行步骤S804，直到所有相机的所有锚点纵坐标的前后两次迭代变化量的绝对值小于设定阈值Tanchor，停止迭代，输出由锚点确定的最优线性分段映射函数作为最优映射曲线函数；在本实施例中，Tanchor取1.0e-6；S805. Repeat step S804 until the absolute value of the change amount of the two iterations before and after the ordinates of all anchor points of all cameras is less than the set threshold T anchor , stop the iteration, and output the optimal linear piecewise mapping function determined by the anchor points as Optimal mapping curve function; In this embodiment, T anchor takes 1.0e-6;

S9、最优映射曲线函数亮度均衡后处理；对P个相机的最优加性增益预处理视图

Figure BDA0003191976740000149
通过最优映射曲线函数进行亮度均衡后处理，得到输出环视鸟瞰图的Y分量
Figure BDA00031919767400001410
公式如下：S9, optimal mapping curve function brightness equalization post-processing; optimal additive gain preprocessing view for P cameras
Figure BDA0003191976740000149
The brightness equalization post-processing is carried out through the optimal mapping curve function, and the Y component of the output look-around bird's-eye view is obtained.
Figure BDA00031919767400001410
The formula is as follows:

Figure BDA00031919767400001411
Figure BDA00031919767400001411

S10、图像格式还原；P个相机经亮度均衡处理后的输出环视鸟瞰图的Y、U、V分量分别为

Figure BDA00031919767400001412
如果输入的环视鸟瞰图不是YUV格式，将亮度均衡处理后的输出环视鸟瞰图进行颜色空间转换，转换为原来输入的格式。S10. Image format restoration; the Y, U, and V components of the output look-around bird's-eye view of the P cameras after brightness equalization processing are respectively:
Figure BDA00031919767400001412
If the input look-around bird's-eye view is not in YUV format, perform color space conversion on the output look-around bird's-eye image after brightness equalization to convert it to the original input format.

综上所述，由于全景环视系统中不同相机的光照条件不同，每个相机自动曝光(AE)和自动白平衡(AWB)参数也不同，合成的环视视图在相邻视图之间存在明显的边界，影响视觉效果，针对上述现有技术问题，本实施例公开了一种全景环视系统的亮度均衡方法，该方法只处理输入视图的亮度Y分量，对相邻相机视图的重叠区域进行块平均降采样，选择内点，得到的内点样本索引集合用于计算最优加性增益；然后利用最优加性增益进行亮度均衡预处理；最后重新选择预处理后的内点样本，计算最优线性分段映射函数，进行亮度均衡后处理。现有技术在所有颜色分量通道采用加性增益模型，具有显式数学解，本发明结合加性增益模型和最优线性分段映射函数，利用加性增益模型进行预处理，利用最优线性分段映射函数进行后处理，进一步改善亮度均衡效果，改善图像拼接融合效果；本发明只对输入视图的亮度分量进行处理，计算量是现有技术的1/3，降低了计算复杂度。To sum up, due to the different lighting conditions of different cameras in the panoramic surround view system, and the automatic exposure (AE) and automatic white balance (AWB) parameters of each camera are also different, the synthetic surround view has obvious boundaries between adjacent views. , affecting the visual effect. In view of the above problems in the prior art, this embodiment discloses a brightness equalization method for a panoramic surround view system. The method only processes the brightness Y component of the input view, and performs block average reduction on the overlapping area of adjacent camera views. Sampling, select interior points, and the obtained interior point sample index set is used to calculate the optimal additive gain; then use the optimal additive gain to perform brightness equalization preprocessing; finally re-select the preprocessed interior point samples to calculate the optimal linearity Segment mapping function for post-processing of luminance equalization. The prior art adopts the additive gain model in all color component channels, and has an explicit mathematical solution. The present invention combines the additive gain model and the optimal linear piecewise mapping function, uses the additive gain model for preprocessing, and uses the optimal linear The segment mapping function is post-processed to further improve the brightness equalization effect and the image splicing and fusion effect; the present invention only processes the brightness component of the input view, and the calculation amount is 1/3 of the prior art, which reduces the calculation complexity.

上述实施例为本发明较佳的实施方式，但本发明的实施方式并不受上述实施例的限制，其他的任何未背离本发明的精神实质与原理下所作的改变、修饰、替代、组合、简化，均应为等效的置换方式，都包含在本发明的保护范围之内。The above-mentioned embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited by the above-mentioned embodiments, and any other changes, modifications, substitutions, combinations, The simplification should be equivalent replacement manners, which are all included in the protection scope of the present invention.

Claims (4)

1. A brightness equalization method for a panoramic surround view system, said brightness equalization method comprising the steps of:
s1, image format conversion: converting the input panoramic all-round looking aerial view of the P cameras of the panoramic all-round looking system into a YUV format to obtain an input view I of the P camerasmThe corresponding Y, U, V components are respectively
Figure FDA0003191976730000011
Wherein m is a camera index, m is 0,1, …, P-1;
s2, block average down-sampling: input view of camera m, taking into account only the luminance Y component
Figure FDA0003191976730000012
Input views with neighboring cameras n
Figure FDA0003191976730000013
Is recorded as omegamnWherein n is the next camera index in the clockwise direction, n ≡ (m +1) mod P, the symbol "mod" represents the modulo operation, and the symbol "≡" represents identity in the modulo operation; input views for camera n
Figure FDA0003191976730000014
Input views with neighboring cameras m
Figure FDA0003191976730000015
Is recorded as omeganmFor each set of overlapping area images omega of adjacent camerasmn and ΩnmRespectively blocking, each subblock is N multiplied by N pixels, taking the average value of each subblock as the output of the corresponding subblock, and obtaining the downsampled image phi consisting of the average values of the subblocksmn and Фnm；
S3, selecting a sample, selecting an inner point with smaller difference, and obtaining a sample index set; downsampled image phi based on overlapping area of each group of adjacent camerasmn and ФnmSelecting the coordinates of the inner points with smaller difference to form a sample index set Smn：
Smn＝{[i,j]|(Φmn[i,j]-Φnm[i,j])2＜Th,[i,j]∈[1,Nmn]X[1,Mmn]}
Wherein i, j represent the abscissa and ordinate of the image, respectively, [ i, j]Coordinates, phi, representing imagesmn[i,j]Representing a downsampled image phimnCorresponding coordinate [ i, j]Pixel gray value of (1), phinm[i,j]Representing a downsampled image phinmCorresponding coordinate [ i, j]Th represents an inner point threshold, Nmn and MmnRespectively representing down-sampled images phimn and ФnmThe number of columns and rows;
s4, calculating the optimal additive gains of the P cameras
Figure FDA0003191976730000016
Down-sampled image phi enabling adjusted overlap area of adjacent camerasmn and ФnmThe mean square error of the corresponding interior point pixel gray value is the minimum, and the following formula is satisfied:
Figure FDA0003191976730000021
wherein ,gmFor the additive gain of the m-th camera,
Figure FDA0003191976730000022
represents the optimal additive gain for the mth camera;
s5, performing brightness equalization preprocessing by an additive gain model, and performing brightness equalization preprocessing by adding the optimal additive gain to the Y component of the input view of each camera, wherein the formula is as follows:
Figure FDA0003191976730000023
wherein ,
Figure FDA0003191976730000024
an output view which represents the Y component of the mth input view after the optimal additive gain preprocessing;
s6, correcting the down-sampled images, each down-sampled image phimn and ФnmRespectively adding the corresponding optimal additive gains, wherein the formula is as follows:
Figure FDA0003191976730000025
Figure FDA0003191976730000026
wherein ,
Figure FDA0003191976730000027
and
Figure FDA0003191976730000028
respectively representing down-sampled images phimn and ФnmThe down-sampled image is corrected by the optimal additive gain;
s7, updating the sample index set, and obtaining the modified down-sampled image of the overlapping area of the adjacent cameras again by using the method in the step S3
Figure FDA0003191976730000029
And
Figure FDA00031919767300000210
sample index set of inliers of
Figure FDA00031919767300000211
The following formula is satisfied:
Figure FDA00031919767300000212
wherein ,
Figure FDA00031919767300000213
representing modified downsampled images
Figure FDA00031919767300000214
Corresponding coordinate [ i, j]The gray value of the pixel of (a),
Figure FDA00031919767300000215
representing modified downsampled images
Figure FDA00031919767300000216
Corresponding coordinate [ i, j]Pixel gray scale value of (a);
s8, calculating the optimal mapping curve function, wherein the mapping curve function y of the mth camera is Tm(x) Will [0,255 ]]The input pixel value x in between maps to [0,255%]Output value y between, optimal mapping curve function T of P camerasm() Obtained by optimizing the following formula:
Figure FDA0003191976730000031
wherein ,Tm() A mapping curve function representing the mth camera,
Figure FDA0003191976730000032
representing a corresponding optimal mapping curve function, wherein beta is a penalty factor and restricts the deviation degree of an input value and an output value of the mapping curve function;
s9, brightness equalization post-processing of the optimal mapping curve function, and view preprocessing of the optimal additive gain of the P cameras
Figure FDA0003191976730000033
Performing brightness equalization post-processing through the optimal mapping curve function to obtain a Y component of the output all-round aerial view
Figure FDA0003191976730000034
The formula is as follows:
Figure FDA0003191976730000035
finally, Y, U, V components of the output circular bird's-eye view image of the P cameras after brightness equalization processing are respectively obtained
Figure FDA0003191976730000036
2. The brightness equalization method of the panoramic all-around view system according to claim 1, wherein the brightness equalization method obtains the Y component of the output all-around view through the brightness equalization post-processing of the optimal mapping curve function in step S9
Figure FDA0003191976730000037
Then, the method also comprises the following steps:
and S10, restoring the image format, and if the input circular view is not in the YUV format, performing color space conversion on the output circular view after the brightness equalization processing to convert the output circular view into the original input format.
3. The brightness equalizing method of a panoramic looking-around system according to claim 1, wherein the optimal mapping curve function in step S8 is calculated as follows:
s801, the mapping curve function of the mth camera is expressed by a linear piecewise mapping function, the linear piecewise mapping function is defined by a group of anchor points, the number of the anchor points is d +1, and the coordinate of the kth anchor point is
Figure FDA0003191976730000041
k is anchor point index, k is 0,1, …, d, mapping curve function T of mth cameram() Can be expressed as:
Figure FDA0003191976730000042
when in use
Figure FDA0003191976730000043
Where x represents the input pixel value, Tm(x) An output mapped pixel value representing a mapping curve function;
s802, fixing the 0 th anchor point of the optimal linear piecewise mapping function of the mth camera
Figure FDA0003191976730000044
The d anchor point is fixed as
Figure FDA0003191976730000045
Down-sampling images based on correction
Figure FDA0003191976730000046
And
Figure FDA0003191976730000047
determining the distribution range of the pixel gray value of the inner pointThe minimum value and the maximum value of the pixel gray value are obtained, and the abscissa ranges of other anchor points are obtained
Figure FDA0003191976730000048
wherein
Figure FDA0003191976730000049
And
Figure FDA00031919767300000410
is determined by the following formula:
Figure FDA00031919767300000411
where l is the previous camera index in the clockwise direction, l ≡ (m-1) mod P,
Figure FDA00031919767300000412
and
Figure FDA00031919767300000413
modified down-sampled images corresponding to two overlapping regions of the mth camera in the clockwise direction respectively,
Figure FDA00031919767300000414
and
Figure FDA00031919767300000415
are respectively as
Figure FDA00031919767300000416
And
Figure FDA00031919767300000417
max () represents the maximum value taking operation, min () represents the minimum value taking operation;
other anchor points
Figure FDA00031919767300000418
Uniformly distributed in the range of abscissa
Figure FDA00031919767300000419
And (d) is determined by the following formula:
Figure FDA00031919767300000420
wherein ,
Figure FDA00031919767300000421
is the abscissa of the kth anchor point of the mth camera;
s803, calculating a sample index set
Figure FDA00031919767300000422
And
Figure FDA00031919767300000423
kth anchor point ordinate of mth camera
Figure FDA00031919767300000424
Only with respect to the grey value of the pixel
Figure FDA00031919767300000425
Modified downsampled image of interval
Figure FDA00031919767300000426
And
Figure FDA00031919767300000427
the inner point of (a); selecting the gray value of the pixel at
Figure FDA00031919767300000428
Modified downsampled image of interval
Figure FDA00031919767300000429
And
Figure FDA00031919767300000430
the coordinates of the inliers of (a) constitute a sample index set
Figure FDA00031919767300000431
And
Figure FDA00031919767300000432
Figure FDA00031919767300000433
Figure FDA0003191976730000051
wherein ,
Figure FDA0003191976730000052
representing modified downsampled images
Figure FDA0003191976730000053
Corresponding coordinate [ i, j]The gray value of the pixel of (a),
Figure FDA0003191976730000054
and
Figure FDA0003191976730000055
are respectively as
Figure FDA0003191976730000056
And
Figure FDA0003191976730000057
is determined by the sample index set of the inliers,
Figure FDA0003191976730000058
the abscissa representing the anchor point of the (k-1) th camera,
Figure FDA0003191976730000059
represents the abscissa of the (k +1) th anchor point;
s804, sequentially and iteratively solving d-1 anchor point vertical coordinates of the optimal linear piecewise mapping function of the mth camera in sequence
Figure FDA00031919767300000510
Obtaining an optimal linear piecewise mapping function of the mth camera;
s805, the step S804 is repeatedly executed until the absolute value of the iteration variation of the front and back two times of all the anchor point vertical coordinates of all the cameras is smaller than the set threshold value TanchorAnd stopping iteration, and outputting the optimal linear piecewise mapping function determined by the anchor point as an optimal mapping curve function.
4. The brightness equalizing method of claim 3, wherein the d-1 vertical coordinates of anchor points of the optimal linear piecewise mapping function of the mth camera are sequentially solved by iteration in step S804
Figure FDA00031919767300000511
Obtaining the optimal linear piecewise mapping function of the mth camera, wherein the process is as follows:
s80401, fixing the vertical coordinates of other anchor points, and updating the vertical coordinates of the anchor points of the odd array of the optimal linear piecewise mapping function of the mth camera
Figure FDA00031919767300000512
Ordinate of the kth anchor point
Figure FDA00031919767300000513
The updated calculation formula of (2) is as follows:
Figure FDA00031919767300000514
wherein A, B, C are intermediate calculation variables, and are defined as follows:
Figure FDA00031919767300000515
Figure FDA0003191976730000061
Figure FDA0003191976730000062
wherein ,
Figure FDA0003191976730000063
and
Figure FDA0003191976730000064
for a modified down-sampled image of the overlapping region of cameras m and n,
Figure FDA0003191976730000065
and
Figure FDA0003191976730000066
for modified downsampled images of the overlapping region of cameras l and m, Tn() and Tl() Mapping curve function, function F, for the n, l cameras respectivelymk() and Y’mk() For the piecewise function, the following is defined:
Figure FDA0003191976730000067
Figure FDA0003191976730000068
s80402, fixing the vertical coordinates of other anchor points, and updating the vertical coordinates of the anchor points of the even group of the optimal linear piecewise mapping function of the mth camera
Figure FDA0003191976730000069
CN202110880913.7A 2021-08-02 2021-08-02 A Brightness Equalization Method for Panoramic Surround View System Active CN113781317B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110880913.7A CN113781317B (en) 2021-08-02 2021-08-02 A Brightness Equalization Method for Panoramic Surround View System

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110880913.7A CN113781317B (en) 2021-08-02 2021-08-02 A Brightness Equalization Method for Panoramic Surround View System

Publications (2)

Publication Number Publication Date
CN113781317A true CN113781317A (en) 2021-12-10
CN113781317B CN113781317B (en) 2023-08-18

Family

ID=78836472

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110880913.7A Active CN113781317B (en) 2021-08-02 2021-08-02 A Brightness Equalization Method for Panoramic Surround View System

Country Status (1)

Country Link
CN (1) CN113781317B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117237237A (en) * 2023-11-13 2023-12-15 深圳元戎启行科技有限公司 Luminosity balancing method and device for vehicle-mounted 360-degree panoramic image
CN118918198A (en) * 2024-10-11 2024-11-08 深圳市云希谷科技有限公司 Photo preview method, electronic device and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060177150A1 (en) * 2005-02-01 2006-08-10 Microsoft Corporation Method and system for combining multiple exposure images having scene and camera motion
US20090231447A1 (en) * 2008-03-12 2009-09-17 Chung-Ang University Industry-Academic Cooperation Foundation Apparatus and method for generating panorama images and apparatus and method for object-tracking using the same
CN109166076A (en) * 2018-08-10 2019-01-08 深圳岚锋创视网络科技有限公司 Luminance regulating method, device and the portable terminal of polyphaser splicing

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060177150A1 (en) * 2005-02-01 2006-08-10 Microsoft Corporation Method and system for combining multiple exposure images having scene and camera motion
US20090231447A1 (en) * 2008-03-12 2009-09-17 Chung-Ang University Industry-Academic Cooperation Foundation Apparatus and method for generating panorama images and apparatus and method for object-tracking using the same
CN109166076A (en) * 2018-08-10 2019-01-08 深圳岚锋创视网络科技有限公司 Luminance regulating method, device and the portable terminal of polyphaser splicing

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
范翔;夏顺仁;: "基于特征的显微图像全自动拼接", 浙江大学学报(工学版) *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117237237A (en) * 2023-11-13 2023-12-15 深圳元戎启行科技有限公司 Luminosity balancing method and device for vehicle-mounted 360-degree panoramic image
CN118918198A (en) * 2024-10-11 2024-11-08 深圳市云希谷科技有限公司 Photo preview method, electronic device and storage medium
CN118918198B (en) * 2024-10-11 2024-12-06 深圳市云希谷科技有限公司 Photo preview method, electronic device and storage medium

Also Published As

Publication number Publication date
CN113781317B (en) 2023-08-18

Similar Documents

Publication Publication Date Title
CN102750674B (en) Video image defogging method based on self-adapting allowance
CN110148095A (en) A kind of underwater picture Enhancement Method and enhancement device
US9509909B2 (en) Method and apparatus for a surround view camera system photometric alignment
CN113781317A (en) Brightness equalization method of panoramic all-around system
KR20120136813A (en) Apparatus and method for image processing
WO2018168539A1 (en) Learning method and program
CN104252700A (en) Histogram equalization method for infrared image
WO2023206343A1 (en) Image super-resolution method based on image pre-training strategy
CN112767243B (en) Method and system for realizing super-resolution of hyperspectral image
CN115115516B (en) Real world video super-resolution construction method based on Raw domain
CN109886906B (en) Detail-sensitive real-time low-light video enhancement method and system
CN112104847A (en) SONY-RGBW array color reconstruction method based on residual error and high-frequency replacement
CN112529813B (en) Image defogging processing method and device and computer storage medium
CN104933687B (en) A kind of multiple dimensioned emergence algorithm of jointing line for considering region of variation
CN111932594A (en) Billion pixel video alignment method and device based on optical flow and medium
CN115731132A (en) Image restoration method, device, equipment and medium
CN113191959B (en) A method to improve the limit image quality of digital imaging system based on degradation calibration
CN109767407A (en) A Quadratic Estimation Method for Atmospheric Transmittance Image in Dehazing Process
CN114155173A (en) Image defogging method and device and nonvolatile storage medium
CN117372307B (en) A distributed image enhancement method for collaborative detection by multiple UAVs
CN118195960B (en) Multi-layer subnet multi-task image restoration method combining spatial aggregation attention
CN105160627B (en) Super-resolution image acquisition method and system based on classification self-learning
CN109934342B (en) Neural network model training method, depth image restoration method and system
Huang et al. An end-to-end dehazing network with transitional convolution layer
KR101230909B1 (en) Apparatus and method for processing wide angle image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant