CN103824297B

CN103824297B - In complicated high dynamic environment, background and the method for prospect is quickly updated based on multithreading

Info

Publication number: CN103824297B
Application number: CN201410081798.7A
Authority: CN
Inventors: 杨路; 程洪; 苏建安
Original assignee: University of Electronic Science and Technology of China
Current assignee: University of Electronic Science and Technology of China
Priority date: 2014-03-07
Filing date: 2014-03-07
Publication date: 2016-08-24
Anticipated expiration: 2034-03-07
Also published as: CN103824297A

Abstract

The present invention relates to image processing field, be specifically related to a kind of in complicated high dynamic environment, quickly update background and the method for prospect based on multithreading, comprise the following steps: pixel and vicinity points thereof are carried out Haar like feature extraction；According to the complexity of scene, the characteristic vector of front 10～35 two field pictures is directly incorporated into the end of background model matrix；Obtain a certain pixel in image, calculate this pixel distance to its background model, differentiate that the most whether this pixel is as foreground point with this；If showing that current pixel point is background dot by step 3, then update eigenvalue minimum with current P2M distance in background model matrix；If showing that current pixel point is background dot by step 3, from the pixel that current pixel point is neighbouring, randomly selecting a point, updating feature maximum with current pixel characteristic distance in its background model.The present invention can update background in real time, exactly, adapts to various complex environment, is effectively improved accuracy and the adaptability of foreground detection.

Description

Method for fast updating background and foreground in complex and highly dynamic environment based on multithreading

技术领域technical field

本发明涉及图像处理领域，具体涉及一种基于多线程在复杂高动态环境中快速更新背景和前景的方法。The invention relates to the field of image processing, in particular to a method for quickly updating background and foreground in a complex and highly dynamic environment based on multithreading.

背景技术Background technique

随着视频监控、人机交互、图像编码和检索等新兴领域的迅速发展和技术需求的快速增多，计算机视觉的相关技术也取得了巨大的突破，图像处理成为行业的基础和领域的技术核心，而背景建模作为其中的常见处理手段，也取得了长足的进步。With the rapid development of emerging fields such as video surveillance, human-computer interaction, image coding and retrieval, and the rapid increase in technical requirements, computer vision related technologies have also made great breakthroughs, and image processing has become the foundation of the industry and the technical core of the field. As a common processing method, background modeling has also made great progress.

最流行、使用最广泛的背景建模方法当属混合高斯模型。它将图像中的每一个像素点进行建模，定义每个像素点的分布模型为由多个单高斯模型组成的集合，根据每一个新的像素值更新模型参数，按照一定的准则判断哪些像素点为背景点、哪些为前景点；当光照发生大规模的迅速变化时，混合高斯模型将为其新建一个高斯体，但仍以以前的像素值作为背景(因为新的高斯体的“力量”还不到能够取代原来主高斯体的地步)，直到一定帧数后，新的高斯体取代原来的背景。但是对于城市里高动态、光照变化复杂的环境，前景目标数量很大而移动缓慢，背景中的光照、阴影、树叶摇动等小变化层出不穷，常常出现背景还没来得及更新完毕环境又发生变化的情况，混合高斯模型就出现了不断建新的高斯体、在各种变化间疲于计算的结果，达不到实时、准确检测前景的目的。The most popular and widely used background modeling method is the Gaussian mixture model. It models each pixel in the image, defines the distribution model of each pixel as a set of multiple single Gaussian models, updates the model parameters according to each new pixel value, and judges which pixels according to certain criteria Points are background points and which are foreground points; when the illumination changes rapidly on a large scale, the mixed Gaussian model will create a new Gaussian volume for it, but still use the previous pixel values as the background (because the "power" of the new Gaussian volume It is not enough to replace the original main Gaussian body), until after a certain number of frames, the new Gaussian body replaces the original background. However, for the environment with high dynamics and complex lighting changes in the city, the number of foreground objects is large and the movement is slow, and small changes such as lighting, shadows, and leaf shaking in the background emerge in endlessly, and the environment often changes before the background is updated. , the mixed Gaussian model has the result of constantly building new Gaussian bodies and exhausting calculations between various changes, which cannot achieve the purpose of real-time and accurate detection of prospects.

贝叶斯方法作为混合高斯建模的替代品，使用了核密度估计的方法，递归地使用贝叶斯学习来近似每个像素的概率密度分布，代替了混合高斯模型的精确参数估计方法。但是贝叶斯方法仍然无法解决前景移动缓慢的判别问题(即将移动缓慢的前景误认为背景)；接着，码本算法将像素值量化编码，使得邻域的信息被加入模型，解决了缓慢前景的问题。但是码本算法需要花费大量的时间在离线训练上，难以满足复杂高动态环境多变的要求。As an alternative to Gaussian mixture modeling, the Bayesian method uses kernel density estimation, which recursively uses Bayesian learning to approximate the probability density distribution of each pixel, instead of the exact parameter estimation method of the Gaussian mixture model. However, the Bayesian method still cannot solve the problem of slow-moving foreground discrimination (that is, the slow-moving foreground is mistaken for the background); then, the codebook algorithm quantizes and encodes the pixel values, so that the information of the neighborhood is added to the model, which solves the problem of slow foreground. question. However, the codebook algorithm needs to spend a lot of time on offline training, and it is difficult to meet the changing requirements of complex and highly dynamic environments.

将背景建模问题看作一个信号重建的问题是最近一种较为流行的做法。当环境中的前景只占很小一部分时，使用压缩感知理论来对前景进行检测是一种行之有效的方法，即是将前景看作是信号重建中的噪声，这样背景建模就成了一个主信号量恢复的问题。同时，还有一种做法是将前景在环境中看作一个暂时出现的量，使用稀疏表达，将前景看作一个稀疏量，从而从过去的一些图像里恢复出当前的背景。然而，在复杂高动态环境下，任何时间或者空间上的稀疏假设都是不存在的，前景有可能在在时空中占有很大的比例。因此，便有了一种将混合高斯建模和信号恢复串联起来的方法，使用主成分分析，将背景从主特征中恢复出来。但是，这种方法显然需要花费大量的资源在训练和参数估计上，难以达到实时的目的。It is a popular approach recently to view the background modeling problem as a signal reconstruction problem. When the foreground in the environment only accounts for a small part, it is an effective method to use the compressed sensing theory to detect the foreground, that is, the foreground is regarded as the noise in the signal reconstruction, so that the background modeling becomes A master semaphore recovery problem. At the same time, there is another way to regard the foreground as a temporary quantity in the environment, and use sparse expression to treat the foreground as a sparse quantity, so as to restore the current background from some past images. However, in a complex and highly dynamic environment, any temporal or spatial sparsity assumption does not exist, and the foreground may occupy a large proportion in space and time. Therefore, there is a method of combining Gaussian mixture modeling and signal recovery, using principal component analysis, to recover the background from the main features. However, this method obviously needs to spend a lot of resources on training and parameter estimation, and it is difficult to achieve real-time purposes.

发明内容Contents of the invention

本发明的目的在于提供基于多线程在复杂高动态环境中快速更新背景和前景的方法，解决现有的建模方法无法应对复杂高动态环境，无法实时、准确地更新背景模型，并且前景检测的准确性和适应性不高的问题。The purpose of the present invention is to provide a method for quickly updating the background and foreground based on multi-threading in a complex and highly dynamic environment, so as to solve the problem that the existing modeling method cannot cope with the complex and highly dynamic environment, cannot update the background model in real time and accurately, and the foreground detection Problems with low accuracy and adaptability.

为解决上述的技术问题，本发明采用以下技术方案：In order to solve the above-mentioned technical problems, the present invention adopts the following technical solutions:

一种基于多线程在复杂高动态环境中快速更新背景和前景的方法，包括以下步骤：A method for rapidly updating background and foreground in a complex and highly dynamic environment based on multithreading, comprising the following steps:

步骤一，对像素点及其邻近像素点进行Haar-like特征提取；Step 1, perform Haar-like feature extraction on the pixel and its adjacent pixels;

步骤二，根据场景的复杂程度，将前10～35帧图像的特征向量直接并入背景模型矩阵的末尾；Step 2, according to the complexity of the scene, the feature vectors of the first 10-35 frames of images are directly incorporated into the end of the background model matrix;

步骤三，获取图像中的某一像素点，计算该像素点到其背景模型的P2M距离，以此来判别该像素点当前是否为前景点；Step 3, obtain a certain pixel in the image, and calculate the P2M distance from the pixel to its background model, so as to judge whether the pixel is currently a foreground point;

步骤四，如果通过步骤三得出当前像素点为背景点，则更新背景模型矩阵中与当前P2M距离最小的特征值；Step 4, if the current pixel point is obtained as a background point through step 3, then update the eigenvalue with the smallest distance from the current P2M in the background model matrix;

步骤五，如果通过步骤三得出当前像素点为背景点，从当前像素点邻近的像素点中随机选取一个点，更新其背景模型中与当前像素特征P2M距离最大的特征。Step 5, if the current pixel point is determined to be the background point through step 3, randomly select a point from the pixels adjacent to the current pixel point, and update the feature in its background model with the largest P2M distance from the current pixel feature.

更进一步的技术方案是，所述步骤一中Haar-like特征提取具体方法是：A further technical solution is that the specific method of Haar-like feature extraction in the step 1 is:

获取当前帧图像中第k个像素点和其邻近点构成的像素块向量P_k，利用积分图，将图像从起点开始到各个点所形成的矩形区域像素之和作为一个数组的元素保存起来，当要计算某个区域的像素和时直接索引数组中对应点的值，通过加减算法得到乘数，将压缩感知矩阵与像素块向量乘法的问题转变成积分图得到的乘数和权值相乘再求和的问题，从而获取到压缩感知矩阵A，进而得到第k个像素点的特征向量v_k＝AP_k。Obtain the pixel block vector P _k formed by the kth pixel point and its adjacent points in the current frame image, and use the integral map to save the sum of the pixels in the rectangular area formed by the image from the starting point to each point as an array element, When it is necessary to calculate the pixel sum of a certain area, directly index the value of the corresponding point in the array, and obtain the multiplier through the addition and subtraction algorithm, and convert the problem of multiplication of the compressed sensing matrix and the pixel block vector into an integral image to obtain the multiplier and weight. The problem of multiplication and summation is obtained to obtain the compressed sensing matrix A, and then to obtain the feature vector v _k =AP _k of the kth pixel.

更进一步的技术方案是，所述压缩感知矩阵(n＞m)，其中n为目标向量的维度，即是以当前像素为中心的图像块的长度，m为经过压缩感知后的特征维度，即是背景模型矩阵的行数。A further technical solution is that the compressed sensing matrix (n>m), where n is the dimension of the target vector, that is, the length of the image block centered on the current pixel, and m is the feature dimension after compressed sensing, that is, the number of rows of the background model matrix.

更进一步的技术方案是，所述步骤二中，判断当前帧图像如果属于前N帧，则第k个像素点的背景模型矩阵表示为M_k＝{v_k,1,v_k,2,…,v_k,N}，其中代表第k个像素点第i帧的特征向量。A further technical solution is that in said step 2, if it is judged that the current frame image belongs to the previous N frames, then the background model matrix of the kth pixel point Expressed as M _k ={v _k,1 ,v _k,2 ,…,v _k,N }, where Represents the feature vector of the i-th frame at the k-th pixel.

更进一步的技术方案是，所述步骤三中，判别该像素点当前是否为前景点的方法为：A further technical solution is that in the step 3, the method for judging whether the pixel is currently a foreground point is:

用I_k代表当前帧图像中第k个像素点的像素值，用代表其背景模型，用v_k代表该像素点的特征向量，用v_k,l代表该像素点特征向量第l维的值(1≤l≤m)，用v_k,i,l代表该像素点背景模型中第i个的特征向量第l维的值(1≤i≤N，1≤l≤m)。那么最小P2M距离定义为Use I _k to represent the pixel value of the kth pixel in the current frame image, use represents its background model, use v _k to represent the feature vector of the pixel, use v _k,l to represent the value of the l-th dimension of the pixel feature vector (1≤l≤m), and use v _k,i,l to represent the pixel The value of the l-th dimension of the i-th feature vector in the point background model (1≤i≤N, 1≤l≤m). Then the minimum P2M distance is defined as

$M m i i n no__P P 22 M m (({I I}_{k k},, {M m}_{k k})) = = {Σ Σ}_{l l = = 11}^{m m} \underset{i i &Element; &Element; {{11,, 22,, ... ...,, N N}}}{min min} {(({v v}_{k k,, l l} - - {v v}_{k k,, i i,, l l}))}^{22}$

如果满足if satisfied

Min_P2M(I_k，M_k)＞ThresholdMin_P2M(I _k , M _k )＞Threshold

那么，认为该像素点为前景点，其中Threshold为手动指定的常量，通过国际通用的背景模型效果的F-Measure来评估：Then, the pixel is considered as the foreground point, where Threshold is a manually specified constant, and is evaluated by the F-Measure of the international background model effect:

$F f = = 22 ((\frac{p p r r e e c c i i s the s i i o o n no \cdot &Center Dot; r r e e c c a a l l l l}{p p r r e e c c i i s the s i i o o n no + + r r e e c c a a l l l l})),, 00 \leq \leq F f \leq \leq 11$

其中，precision为前景检测的准确率，recall为前景检测的捕获率，F-Measure越大，前景检测效果越好。Among them, precision is the accuracy rate of foreground detection, and recall is the capture rate of foreground detection. The larger the F-Measure, the better the foreground detection effect.

更进一步的技术方案是，所述当前像素点为背景点时，对其更新分为像素点背景更新和邻域像素点背景更新，A further technical solution is that when the current pixel is a background point, its update is divided into pixel background update and neighborhood pixel background update,

其中所述像素点背景更新方法是：用v_k,i,l表示该第k个像素点背景模型中第i个的特征向量第l维的值(1≤i≤N，1≤l≤m)，用v_k,l表示该像素点特征向量第l维的值(1≤l≤m)，利用公式Wherein said pixel point background update method is: use v _{k, i, l} to represent the value of the l-th dimension of the i-th feature vector in the k-th pixel point background model (1≤i≤N, 1≤l≤m ), use v _k,l to represent the value of the l-th dimension of the feature vector of the pixel (1≤l≤m), using the formula

${v v}_{k k,, i i,, l l}^{n no e e w w} = = {v v}_{k k,, l l},, ((i i = = \underset{i i &Element; &Element; {{11,, 22,, ... ...,, N N}}}{arg arg min min} {(({v v}_{k k,, l l} - - {v v}_{k k,, i i,, l l}))}^{22}))$

得出该像素和其邻近的块新的特征向量v_k,i,l ^new；Obtain the new feature vector v _k,i,l ^new of this pixel and its adjacent blocks;

其中所述邻域像素点背景更新方法是：我们从第k个像素点的8邻域中随机选取一个像素点，设其为图像中第j个像素点，用I_j代表该像素点的像素值，用代表其背景模型，用v_j代表该像素点的特征向量，用v_j,l代表该像素点特征向量第l维的值(1≤l≤m)，用v_j,i,l代表该像素点背景模型中第i个的特征向量第l维的值(1≤i≤N，1≤l≤m)，最大P2M距离为Wherein said neighborhood pixel point background updating method is: we randomly select a pixel point from the 8 neighbors of the k pixel point, set it as the j pixel point in the image, represent the pixel of this pixel point with I _j value, with represents its background model, use v _j to represent the feature vector of the pixel, use v _j,l to represent the value of the l-th dimension of the pixel feature vector (1≤l≤m), and use v _j,i,l to represent the pixel The value of the l-th dimension of the i-th feature vector in the point background model (1≤i≤N, 1≤l≤m), the maximum P2M distance is

$M m a a x x__P P 22 M m (({I I}_{j j},, {M m}_{j j})) = = {Σ Σ}_{l l = = 11}^{m m} \underset{i i &Element; &Element; {{11,, 22,, ... ...,, N N}}}{max max} {(({v v}_{j j,, l l} - - {v v}_{j j,, i i,, l l}))}^{22}$

邻域随机点的背景更新，利用公式，The background update of random points in the neighborhood, using the formula,

${v v}_{j j,, i i,, l l}^{n no e e w w} = = {v v}_{j j,, l l},, ((i i = = \underset{i i &Element; &Element; {{11,, 22,, ... ...,, N N}}}{arg arg min min} {(({v v}_{j j,, l l} - - {v v}_{j j,, i i,, l l}))}^{22}))$

得到新的像素特这向量v_k,i,l ^new，其中v_j,i,l为该第j个像素点背景模型中第i个的特征向量第l维的值(1≤i≤N，1≤l≤m)，v_j,l为该像素点特征向量第l维的值(1≤l≤m)。Get a new pixel feature vector v _k,i,l ^new , where v _j,i,l is the value of the l-th dimension of the i-th feature vector in the j-th pixel background model (1≤i≤N, 1≤l≤m), v _j,l is the value of the l-th dimension of the pixel feature vector (1≤l≤m).

与现有技术相比，本发明的有益效果是：Compared with prior art, the beneficial effect of the present invention is:

采用像素到模型的思想，将单个像素点用一系列基于压缩感知的局部描述器特征表示，并用点到类的距离来衡量像素是否为前景点；同时，在进行背景模型更新的时候，也使用了点到类的距离来对局部描述器构成的模型平滑而有效地进行更新，从而使得背景建模和前景检测无论在室内还是室外的复杂环境，都有着快速而高效的性能；能实时、准确地更新背景，适应各种复杂环境，有效提高前景检测的准确性和适应性。Using the idea of pixel to model, a single pixel is represented by a series of local descriptor features based on compressed sensing, and the distance from point to class is used to measure whether the pixel is a foreground point; at the same time, when updating the background model, it is also used The distance between points and classes is used to update the model composed of local descriptors smoothly and effectively, so that background modeling and foreground detection have fast and efficient performance in indoor or outdoor complex environments; real-time and accurate It can update the background accurately, adapt to various complex environments, and effectively improve the accuracy and adaptability of foreground detection.

附图说明Description of drawings

图1为本发明的流程示意图。Fig. 1 is a schematic flow chart of the present invention.

图2为Haar-like特征提取的示意图。Figure 2 is a schematic diagram of Haar-like feature extraction.

图3为最小点到类距离更新背景模型的示意图。Figure 3 is a schematic diagram of the minimum point-to-class distance update background model.

图4为最大点到类距离更新背景模型的示意图。Figure 4 is a schematic diagram of the maximum point-to-class distance update background model.

具体实施方式detailed description

为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

本发明可以在Windows和Linux平台上实现，编程语言也是可以选择的，可以采用C++实现的，使用了多线程的方法。The present invention can be realized on Windows and Linux platforms, and the programming language can also be selected, and can be realized by using C++, using a multithreading method.

图1示出了本发明一种基于多线程在复杂高动态环境中快速更新背景和前景的方法的一个实施例：一种基于多线程在复杂高动态环境中快速更新背景和前景的方法，包括以下步骤：Fig. 1 shows an embodiment of a method for rapidly updating background and foreground based on multithreading in a complex and highly dynamic environment of the present invention: a method for rapidly updating background and foreground in a complex and highly dynamic environment based on multithreading, including The following steps:

图2示出了本发明一种基于多线程在复杂高动态环境中快速更新背景和前景的方法的一个优选实施例，所述步骤一中Haar-like特征提取具体方法是：Fig. 2 has shown a kind of preferred embodiment of the method for rapidly updating background and foreground based on multi-threading in the complex high dynamic environment of the present invention, the specific method of Haar-like feature extraction in the described step 1 is:

获取当前帧图像中第k个像素点和其邻近点构成的像素块向量P_k，利用积分图(积分图是用来计算压缩感知矩阵A的)，将图像从起点开始到各个点所形成的矩形区域像素之和作为一个数组的元素保存起来，当要计算某个区域的像素和时直接索引数组中对应点的值，通过加减算法得到乘数，将压缩感知矩阵与像素块向量乘法的问题转变成积分图得到的乘数和权值相乘再求和的问题，从而获取到压缩感知矩阵A，进而得到第k个像素点的特征向量v_k＝AP_k。Obtain the pixel block vector P _k formed by the kth pixel point and its neighbors in the current frame image, and use the integral map (the integral map is used to calculate the compressed sensing matrix A) to convert the image from the starting point to each point. The sum of pixels in a rectangular area is stored as an array element. When calculating the sum of pixels in a certain area, directly index the value of the corresponding point in the array, and obtain the multiplier through the addition and subtraction algorithm. Multiply the compressed sensing matrix and the pixel block vector The problem is transformed into the problem of multiplying the multiplier obtained from the integral image and the weight and then summing, so as to obtain the compressed sensing matrix A, and then obtain the feature vector v _k =AP _k of the kth pixel.

根据本发明基于多线程在复杂高动态环境中快速更新背景和前景的方法的另一个优选实施例，所述压缩感知矩阵(n＞m)，其中n为目标向量的维度，即是以当前像素为中心的图像块的长度，m为经过压缩感知后的特征维度，即是背景模型矩阵的行数；根据实验，对于室外场景，m不宜大于5，n不宜大于32，m设为3、n设为25时效果较好。According to another preferred embodiment of the method for rapidly updating background and foreground in a complex and highly dynamic environment based on multithreading in the present invention, the compressed sensing matrix (n>m), where n is the dimension of the target vector, that is, the length of the image block centered on the current pixel, and m is the feature dimension after compressed sensing, that is, the number of rows of the background model matrix; according to the experiment, for In outdoor scenes, m should not be greater than 5, and n should not be greater than 32. When m is set to 3 and n is set to 25, the effect is better.

根据本发明基于多线程在复杂高动态环境中快速更新背景和前景的方法的另一个优选实施例，所述步骤二中，判断当前帧图像如果属于前N帧，则第k个像素点的背景模型矩阵表示为M_k＝{v_k,1,v_k,2,…,v_k,N}，其中代表第k个像素点第i帧的特征向量；根据目前已经存在的例如CodeBook和PBAS等背景建模方法，及现场的实验，背景模型需要进行10～35帧的训练，对于普通的室外场景和交通场景，训练20帧已经足够。According to another preferred embodiment of the method for quickly updating the background and the foreground based on multithreading in a complex and highly dynamic environment of the present invention, in the second step, if the current frame image is judged to belong to the previous N frames, then the background of the kth pixel is model matrix Expressed as M _k ={v _k,1 ,v _k,2 ,…,v _k,N }, where Represents the feature vector of the i-th frame of the k-th pixel; according to existing background modeling methods such as CodeBook and PBAS, as well as on-site experiments, the background model needs to be trained for 10 to 35 frames. For ordinary outdoor scenes and For traffic scenes, 20 frames of training is enough.

根据本发明一种基于多线程在复杂高动态环境中快速更新背景和前景的方法的另一个优选实施例，所述步骤三中，判别该像素点当前是否为前景点的方法为：According to another preferred embodiment of the method for quickly updating the background and foreground based on multithreading in a complex and highly dynamic environment of the present invention, in the third step, the method for judging whether the pixel is currently a foreground point is:

如果满足if satisfied

Min_P2M(I_k，M_k)＞ThresholdMin_P2M(I _k , M _k )＞Threshold

其中，precision为前景检测的准确率，recall为前景检测的捕获率，F-Measure越大，前景检测效果越好，根据实验，当Threshold＝3000时，可以取得比较好的效果。Among them, precision is the accuracy rate of foreground detection, and recall is the capture rate of foreground detection. The larger the F-Measure, the better the foreground detection effect. According to experiments, when Threshold=3000, better results can be achieved.

根据本发明基于多线程在复杂高动态环境中快速更新背景和前景的方法的另一个优选实施例，所述当前像素点为背景点时，对其更新分为像素点背景更新和邻域像素点背景更新，According to another preferred embodiment of the method for rapidly updating background and foreground based on multithreading in a complex and highly dynamic environment of the present invention, when the current pixel is a background point, its update is divided into pixel background update and neighborhood pixel background update,

其中所述像素点背景更新方法是：如图3所示，用v_k,i,l表示该第k个像素点背景模型中第i个的特征向量第l维的值(1≤i≤N，1≤l≤m)，用v_k,l表示该像素点特征向量第l维的值(1≤l≤m)，利用公式Wherein said pixel point background updating method is: as shown in Figure 3, use v _{k, i, l} to represent the value of the l-th dimension of the i-th feature vector in the k-th pixel background model (1≤i≤N , 1≤l≤m), use v _k,l to represent the value of the l-th dimension of the feature vector of the pixel (1≤l≤m), use the formula

其中所述邻域像素点背景更新方法是：如图4所示，我们从第k个像素点的8邻域中随机选取一个像素点，设其为图像中第j个像素点，用I_j代表该像素点的像素值，用代表其背景模型，用v_j代表该像素点的特征向量，用v_j,l代表该像素点特征向量第l维的值(1≤l≤m)，用v_j,i,l代表该像素点背景模型中第i个的特征向量第l维的值(1≤i≤N，1≤l≤m)，最大P2M距离为Wherein said neighborhood pixel point background updating method is: as shown in Figure 4, we randomly select a pixel point from the 8 neighbors of the k pixel point, set it as the j pixel point in the image, use I _j Represents the pixel value of the pixel point, using represents its background model, use v _j to represent the feature vector of the pixel, use v _j,l to represent the value of the l-th dimension of the pixel feature vector (1≤l≤m), and use v _j,i,l to represent the pixel The value of the l-th dimension of the i-th feature vector in the point background model (1≤i≤N, 1≤l≤m), the maximum P2M distance is

Claims

1. A method for rapidly updating background and foreground in a complex and highly dynamic environment based on multithreading, characterized in that: comprise the following steps:

Step 1, perform Haar-like feature extraction on the pixel and its adjacent pixels;

Step 2, according to the complexity of the scene, the feature vectors of the first 10-35 frames of images are directly incorporated into the end of the background model matrix;

Step 3, obtain a certain pixel in the image, and calculate the P2M distance from the pixel to its background model, so as to judge whether the pixel is currently a foreground point;

Step 4, if the current pixel point is obtained as a background point through step 3, then update the eigenvalue with the smallest distance from the current P2M in the background model matrix;

Step 5, if the current pixel point is obtained as a background point through step 3, randomly select a point from the pixels adjacent to the current pixel point, and update the feature with the largest distance from the current pixel feature P2M in its background model;

The specific method of Haar-like feature extraction in the step 1 is:

Obtain the pixel block vector P _k formed by the kth pixel point and its adjacent points in the current frame image, and use the integral map to save the sum of the pixels in the rectangular area formed by the image from the starting point to each point as an array element, When it is necessary to calculate the pixel sum of a certain area, directly index the value of the corresponding point in the array, and obtain the multiplier through the addition and subtraction algorithm, and convert the problem of multiplication of the compressed sensing matrix and the pixel block vector into an integral image to obtain the multiplier and weight. multiplication and summation, so as to obtain the compressed sensing matrix A, and then obtain the feature vector v _k = AP _k of the kth pixel;

In the step 3, the method for judging whether the pixel is currently a foreground point is as follows:

Use I _k to represent the pixel value of the kth pixel in the current frame image, use represents its background model, use v _k to represent the feature vector of the pixel, use v _k,l to represent the value of the l-th dimension of the pixel feature vector (1≤l≤m), and use v _k,i,l to represent the pixel The value of the l-th dimension of the i-th feature vector in the point background model (1≤i≤N, 1≤l≤m); then the minimum P2M distance is defined as

m i no_P 2 m (I_{k}, m_{k}) = Σ_{l = 1}^{m} \min_{i &Element; {1, 2, ..., N}} {(v_{k, l} - v_{k, i, l})}^{2}

if satisfied

Min_P2M(I _k , M _k )＞Threshold

Then, the pixel is considered to be the foreground point, where Threshold is a manually specified constant, which is evaluated by the F-Measure of the international background model effect:

\begin{matrix} F f = = 22 ((\frac{p p r r e e c c i i s the s i i o o n no \cdot &Center Dot; r r e e c c a a l l l l}{p p r r e e c c i i s the s i i o o n no + + r r e e c c a a l l l l})) & 00 \leq \leq F f \leq \leq 11 \end{matrix}

Among them, precision is the accuracy rate of foreground detection, and recall is the capture rate of foreground detection. The larger the F-Measure, the better the foreground detection effect.

2. The method for rapidly updating background and foreground based on multithreading in a complex and highly dynamic environment according to claim 1, characterized in that: the compressed sensing matrix (n>m), where n is the dimension of the target vector, that is, the length of the image block centered on the current pixel, and m is the feature dimension after compressed sensing, that is, the number of rows of the background model matrix.

3. The method for rapidly updating background and foreground based on multithreading in a complex and highly dynamic environment according to claim 1, characterized in that: in said step 2, if it is judged that the current frame image belongs to the previous N frames, then the kth Pixel's background model matrix Expressed as M _k ={v _k,1 ,v _k,2 ,…,v _k,N }, where Represents the feature vector of the i-th frame at the k-th pixel.

4. the method for quickly updating background and foreground based on multi-threading in a complex and highly dynamic environment according to claim 1, characterized in that: when the current pixel is a background point, its update is divided into pixel point background update and Neighborhood pixel background update,

Wherein said pixel point background update method is: use v _{k, i, l} to represent the value of the l-th dimension of the i-th feature vector in the k-th pixel point background model (1≤i≤N, 1≤l≤m ), use v _k,l to represent the value of the l-th dimension of the feature vector of the pixel (1≤l≤m), using the formula

\begin{matrix} {v v}_{k k,, i i,, l l}^{n no e e w w} = = {v v}_{k k,, l l} & ((i i = = \underset{i i &Element; &Element; {{11,, 22,, ... ...,, N N}}}{arg arg min min} {(({v v}_{k k,, l l} - - {v v}_{k k,, i i,, l l}))}^{22})) \end{matrix}

Obtain the new feature vector v _k,i,l ^new of this pixel and its adjacent blocks;

Wherein said neighborhood pixel point background updating method is: we randomly select a pixel point from the 8 neighbors of the k pixel point, set it as the j pixel point in the image, represent the pixel of this pixel point with I _j value, with represents its background model, use v _j to represent the feature vector of the pixel, use v _j,l to represent the value of the l-th dimension of the pixel feature vector (1≤l≤m), and use v _j,i,l to represent the pixel The value of the l-th dimension of the i-th feature vector in the point background model (1≤i≤N, 1≤l≤m), the maximum P2M distance is

M m a a x x__P P 22 M m (({I I}_{j j},, {M m}_{j j})) = = {Σ Σ}_{l l = = 11}^{m m} \underset{i i &Element; &Element; {{11,, 22,, ... ...,, N N}}}{max max} {(({v v}_{j j,, l l} - - {v v}_{j j,, i i,, l l}))}^{22}

The background update of random points in the neighborhood, using the formula,

\begin{matrix} {v v}_{j j,, i i,, l l}^{n no e e w w} = = {v v}_{j j,, l l} & ((i i = = \underset{i i &Element; &Element; {{11,, 22,, ... ...,, N N}}}{arg arg min min} {(({v v}_{j j,, l l} - - {v v}_{j j,, i i,, l l}))}^{22})) \end{matrix}

Get a new pixel feature vector v _k,i,l ^new , where v _j,i,l is the value of the l-th dimension of the i-th feature vector in the j-th pixel background model (1≤i≤N, 1 ≤l≤m), v _j,l is the value of the l-th dimension of the pixel feature vector (1≤l≤m).