CN104143077B - Pedestrian target search method and system based on image - Google Patents
Pedestrian target search method and system based on image Download PDFInfo
- Publication number
- CN104143077B CN104143077B CN201310169245.2A CN201310169245A CN104143077B CN 104143077 B CN104143077 B CN 104143077B CN 201310169245 A CN201310169245 A CN 201310169245A CN 104143077 B CN104143077 B CN 104143077B
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- pedestrian target
- sequence
- histogram
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000001514 detection method Methods 0.000 claims abstract description 40
- 230000011218 segmentation Effects 0.000 claims abstract description 33
- 238000004364 calculation method Methods 0.000 claims description 47
- 238000000605 extraction Methods 0.000 claims description 13
- 238000005516 engineering process Methods 0.000 claims description 7
- 238000009825 accumulation Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 7
- 239000000284 extract Substances 0.000 description 7
- 239000003086 colorant Substances 0.000 description 4
- 239000000203 mixture Substances 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
一种基于图像的行人目标检索方法,包括:从原始视频图像中获取行人目标序列和相应的前景序列;计算行人目标序列i的第j帧Ri,j的权值,将行人目标序列i的所有帧对应的前景序列的像素点按照权值进行累加,获得绝对区域直方图H1与模糊区域直方图H2,计算行人目标直方图Hf,并使用测地距离计算行人目标直方图Hf之间的距离,按照测地距离的大小对行人目标序列进行排序。从而避免了分割服饰的难题,在视频中提取出检测跟踪出来的一系列运动目标的前景。通过累计每幅图像的前景可以有效地提取出其服饰信息。因此,基于背景分割检测提高了视频中行人检测的鲁棒性,避免服饰分割有效地提高了行人检索的准确性。此外,还提供一种基于图像的行人目标检索系统。
An image-based pedestrian target retrieval method, including: obtaining the pedestrian target sequence and the corresponding foreground sequence from the original video image; calculating the weight of the jth frame R i, j of the pedestrian target sequence i, and combining The pixels of the foreground sequence corresponding to all frames are accumulated according to the weights to obtain the absolute area histogram H 1 and the blurred area histogram H 2 , calculate the pedestrian target histogram H f , and use the geodesic distance to calculate the pedestrian target histogram H f The pedestrian target sequence is sorted according to the size of the geodesic distance. Thereby avoiding the difficult problem of clothing segmentation, and extracting the foreground of a series of moving targets detected and tracked in the video. Clothing information can be effectively extracted by accumulating the foreground of each image. Therefore, detection based on background segmentation improves the robustness of pedestrian detection in videos, and avoiding clothing segmentation effectively improves the accuracy of pedestrian retrieval. In addition, an image-based pedestrian target retrieval system is also provided.
Description
技术领域technical field
本发明涉及图像处理技术,特别是涉及一种基于图像的行人目标检索方法和系统。The invention relates to image processing technology, in particular to an image-based pedestrian target retrieval method and system.
背景技术Background technique
随着互联网时代的到来,图像检索技术得到广泛的发展与应用。特别是随着智能交通的发展与应用,图像检索技术也随之应用到智能交通分析中。现代城市中广泛分布的摄像头已使得传统的交通分析及行人车辆追踪变得更为简单方便。但是由于监控视频数量过于庞大,面对突发的案件或交通事故往往很难依靠人力来跟踪分析大量的监控视频。With the advent of the Internet age, image retrieval technology has been widely developed and applied. Especially with the development and application of intelligent transportation, image retrieval technology is also applied to intelligent transportation analysis. The widely distributed cameras in modern cities have made traditional traffic analysis and pedestrian and vehicle tracking easier and more convenient. However, due to the large number of surveillance videos, it is often difficult to rely on manpower to track and analyze a large number of surveillance videos in the face of sudden cases or traffic accidents.
当前的行人检索方法一般利用单张检测到的行人目标图像,对行人目标进行分割,然后利用分割出的行人服饰作为特征来检索。静态图像中的行人检测技术这些年有些突破,例如,能够应对一般的正常行人场景,但在实际视频中不能具有很好鲁棒性。服饰分割的准确性对检索有很大的影响。在其他方法中对行人的服饰分割进行了研究,但是由于行人姿态的多样性导致服饰的分割无法太准确,或者太准确的分割的时间代价太大。也有使用视频中的运动目标边沿信息来检测与分割行人,但其没有使用行人模型来区分行人与汽车等其他运动目标,只是利用条纹密度等方法来判断,会有较大的误判率。Current pedestrian retrieval methods generally use a single detected pedestrian target image to segment the pedestrian target, and then use the segmented pedestrian clothing as features for retrieval. Pedestrian detection technology in static images has made some breakthroughs in recent years. For example, it can deal with general normal pedestrian scenes, but it cannot be very robust in actual video. The accuracy of clothing segmentation has a great impact on retrieval. In other methods, the clothing segmentation of pedestrians has been studied, but due to the diversity of pedestrian poses, the segmentation of clothing cannot be too accurate, or the time cost of too accurate segmentation is too high. There is also the use of moving target edge information in the video to detect and segment pedestrians, but it does not use the pedestrian model to distinguish pedestrians from other moving targets such as cars, but only uses stripe density and other methods to judge, which will have a large misjudgment rate.
发明内容Contents of the invention
基于此,提供一种高检出、低误检的基于图像的行人目标检索方法。Based on this, an image-based pedestrian target retrieval method with high detection and low false detection is provided.
一种基于图像的行人目标检索方法,包括以下步骤:An image-based pedestrian target retrieval method, comprising the following steps:
从原始视频图像中获取行人目标序列和相应的前景序列;Obtain pedestrian target sequences and corresponding foreground sequences from raw video images;
将行人目标序列i的所有帧对应的前景序列的像素点按照所述权值进行累加,获得绝对区域直方图H1与模糊区域直方图H2,Accumulate the pixels of the foreground sequence corresponding to all the frames of the pedestrian target sequence i according to the weight to obtain the absolute area histogram H 1 and the blurred area histogram H 2 ,
计算行人目标直方图Hf,并使用测地距离计算行人目标直方图Hf之间的距离,按照测地距离的大小对行人目标序列进行排序;其中,x∈{0,1},x=0为灰度优先,x=1表示彩色优先;颜色分量均匀分为HBINS个间隔,灰度值V小于阈值Tg为黑色区域,灰度区域[Tg,1]均匀分为VBINS个间隔。Calculate the pedestrian target histogram H f , and use the geodesic distance to calculate the distance between the pedestrian target histograms H f , and sort the pedestrian target sequence according to the size of the geodesic distance; where, x∈{0,1}, x= 0 means grayscale priority, x=1 means color priority; the color components are evenly divided into HBINS intervals, the grayscale value V is less than the threshold Tg is a black area, and the grayscale area [Tg, 1] is evenly divided into VBINS intervals.
在其中一个实施例中,从原始视频图像中获取行人目标序列和相应的前景序列的步骤包括:In one of the embodiments, the step of obtaining pedestrian target sequence and corresponding foreground sequence from the original video image includes:
采用混合高斯模型分割行人目标的前景和背景;Segment the foreground and background of pedestrian targets using a mixture of Gaussian models;
统计该帧对应的前景内包含可能存在行人目标的窗口W={wi},其中,窗口内包含的前景面积占窗口面积一半以上,且窗口的高大于Hmin=60像素,宽大于Wmin=30像素;Statistically, the foreground corresponding to the frame contains a window W={w i } that may contain pedestrian targets, where the foreground area contained in the window accounts for more than half of the window area, and the height of the window is greater than Hmin=60 pixels, and the width is greater than Wmin=30 pixel;
将窗口的重合部分进行合并;Merge the overlapping parts of the windows;
采用基于梯度方向直方图在合并后的窗口检测行人目标;Use the gradient orientation histogram to detect pedestrian targets in the merged window;
采用基于学习的目标跟踪方法对检测到的行人目标进行跟踪获得行人目标序列;Use the learning-based target tracking method to track the detected pedestrian targets to obtain pedestrian target sequences;
利用获取到的行人目标序列,从原始视频图片与其对应的背景分割图中获取前景序列。Using the obtained pedestrian target sequence, the foreground sequence is obtained from the original video image and its corresponding background segmentation map.
在其中一个实施例中,所述将窗口的重合部分进行合并的步骤包括:In one of the embodiments, the step of merging overlapping parts of the windows includes:
将任何两个有重合的窗口标记为同一个集合Si;Mark any two overlapping windows as the same set S i ;
判断集合Si,如果并且有i≠j∧wi1∩wi2≠φ,那么合并Si与Sj;重复本步骤直至所有包含有重合部分窗口的集合都被合并;To judge the set S i , if And there is i≠j∧w i1 ∩w i2 ≠φ, then merge S i and S j ; repeat this step until all sets containing overlapping windows are merged;
用一个面积最小的包含集合Si所有窗口的矩形Ti来代表集合Si中所有的窗口,重新形成窗口集合T={Ti}。Use a rectangle T i containing all the windows of the set S i with the smallest area to represent all the windows in the set S i , and re-form the window set T={T i }.
在其中一个实施例中,所述绝对区域直方图H1与模糊区域直方图H2的计算步骤包括:In one of the embodiments, the calculation steps of the absolute region histogram H1 and the blurred region histogram H2 include:
将行人目标图像转换到HSV颜色空间;Convert pedestrian target image to HSV color space;
如果行人目标图像像素的灰度值V<Tg,则H1[1]=H1[1]+1;If the gray value of the pedestrian target image pixel V<Tg, then H 1 [1]=H 1 [1]+1;
如果像素灰度值V≥Tg并且饱和度值Sg<S<Sc,则计算模糊区域直方图H2;其中,模糊区域直方图计算包括颜色部分与灰度部分:If the pixel gray value V≥Tg and the saturation value S g < S < S c , then calculate the fuzzy area histogram H 2 ; wherein, the fuzzy area histogram calculation includes the color part and the gray part:
在其中一个实施例中,所述将行人目标序列i的所有帧对应的前景序列的像素点按照所述权值进行累加,获得绝对区域直方图H1与模糊区域直方图H2的步骤包括:In one of the embodiments, the steps of accumulating the pixels of the foreground sequence corresponding to all the frames of the pedestrian target sequence i according to the weight, and obtaining the absolute region histogram H1 and the blurred region histogram H2 include:
将绝对区域直方图H1与模糊区域直方图H2置零;Set the absolute area histogram H 1 and the fuzzy area histogram H 2 to zero;
计算行人目标序列i的第j帧前景Fi,j的内切椭圆,将内切椭圆与矩形之间部分置为背景;Calculate the inscribed ellipse of the j-th frame foreground F i,j of the pedestrian target sequence i, and set the part between the inscribed ellipse and the rectangle as the background;
将行人目标序列i的第j帧Ri,j对应的前景部分像素按照所述权值进行累加至第j帧Ri,j的绝对区域直方图H1与模糊区域直方图H2,累积值为weight(j),H1[index]=H1[index]+weight(j)或H2[index]=H2[index]+weight(j),直至行人目标序列中所有帧都计算完成;Accumulate the foreground pixels corresponding to the jth frame R i,j of the pedestrian target sequence i to the absolute area histogram H 1 and the blurred area histogram H 2 of the jth frame R i,j according to the weight value, and the accumulated value is weight(j), H 1 [index]=H 1 [index]+weight(j) or H 2 [index]=H 2 [index]+weight(j), until all frames in the pedestrian target sequence are calculated ;
保存绝对区域直方图H1与模糊区域直方图H2为序列i的颜色特征直方图。Save the absolute area histogram H 1 and the fuzzy area histogram H 2 as the color feature histogram of sequence i.
上述基于图像的行人目标检索方法通过基于学习的目标跟踪及前景分割,得到具有鲁棒性的行人检测结果,进而获取行人目标序列及前景序列。再通过计算获取行人目标序列的颜色特征直方图,即对行人目标的服饰颜色进行特征提取及匹配,最后根据公式计算行人目标直方图Hf,并使用测地距离获取行人目标直方图Hf之间的距离,按照测地距离的大小对行人目标序列进行排序。从而获取具有相关性排列的检索结果。上述基于图像的行人目标检索方法避免了分割服饰这一难题,而是在视频中提取出检测跟踪出来的一系列运动目标的前景。通过累计每幅图像的前景可以有效地提取出其服饰信息。因此,基于背景分割的行人检测提高了视频中行人检测的鲁棒性,避免服饰分割有效地提高了行人检索的准确性。The above image-based pedestrian target retrieval method obtains robust pedestrian detection results through learning-based target tracking and foreground segmentation, and then obtains pedestrian target sequences and foreground sequences. Then obtain the color feature histogram of the pedestrian target sequence through calculation, that is, perform feature extraction and matching on the clothing color of the pedestrian target, and finally calculate the pedestrian target histogram H f according to the formula, and use the geodesic distance to obtain the pedestrian target histogram H f . The pedestrian target sequence is sorted according to the size of the geodesic distance. In this way, retrieval results with correlation rankings can be obtained. The above-mentioned image-based pedestrian target retrieval method avoids the difficult problem of segmenting clothing, but extracts the foreground of a series of moving targets detected and tracked in the video. Clothing information can be effectively extracted by accumulating the foreground of each image. Therefore, background segmentation-based pedestrian detection improves the robustness of pedestrian detection in videos, and avoiding clothing segmentation effectively improves the accuracy of pedestrian retrieval.
此外,还提供一种高检出、低误检的基于图像的行人目标检索系统。In addition, an image-based pedestrian object retrieval system with high detection and low false detection is provided.
一种基于图像的行人目标检索系统,其特征在于,包括图像获取模块、行人序列多直方图特征计算模块、行人序列特征计算模块及测地距离排序模块;所述行人序列多直方图特征计算模块分别与所述图像获取模块及所述行人序列特征计算模块连接,所述行人序列特征计算模块还与所述测地距离排序模块连接;An image-based pedestrian target retrieval system, characterized in that it includes an image acquisition module, a pedestrian sequence multi-histogram feature calculation module, a pedestrian sequence feature calculation module and a geodesic distance sorting module; the pedestrian sequence multi-histogram feature calculation module Respectively connected to the image acquisition module and the pedestrian sequence feature calculation module, the pedestrian sequence feature calculation module is also connected to the geodesic distance sorting module;
所述图像获取模块用于从原始视频图像中获取行人目标序列和相应的前景序列;The image acquisition module is used to acquire pedestrian target sequences and corresponding foreground sequences from original video images;
所述行人序列多直方图特征计算模块用于采用公式The pedestrian sequence multi-histogram feature calculation module is used to adopt the formula
所述行人序列多直方图特征计算模块还用于将行人目标序列i的所有帧对应的前景序列的像素点按照所述权值进行累加,获得绝对区域直方图H1与模糊区域直方图H2,The pedestrian sequence multi-histogram feature calculation module is also used to accumulate the pixels of the foreground sequence corresponding to all the frames of the pedestrian target sequence i according to the weights to obtain the absolute area histogram H1 and the blurred area histogram H2 ,
所述行人序列特征计算模块用于根据公式The pedestrian sequence feature calculation module is used to calculate according to the formula
计算行人目标直方图Hf;其中,x∈{0,1},x=0为灰度优先,x=1表示彩色优先;颜色分量均匀分为HBINS个间隔,灰度值V小于阈值Tg为黑色区域,灰度区域[Tg,1]均匀分为VBINS个间隔;Calculate pedestrian target histogram H f ; among them, x∈{0,1}, x=0 means gray priority, x=1 means color priority; the color components are evenly divided into HBINS intervals, and the gray value V is less than the threshold Tg is The black area, the gray area [Tg,1] is evenly divided into VBINS intervals;
所述测地距离排序模块用于使用测地距离计算行人目标直方图Hf之间的距离,按照测地距离的大小对行人目标序列进行排序。The geodesic distance sorting module is used to calculate the distance between the pedestrian target histograms H f using the geodesic distance, and sort the pedestrian target sequence according to the size of the geodesic distance.
在其中一个实施例中,所述图像获取模块包括前景分割模块、行人目标检测模块及行人目标序列提取模块;In one of the embodiments, the image acquisition module includes a foreground segmentation module, a pedestrian target detection module and a pedestrian target sequence extraction module;
所述前景分割模块分别与所述行人目标检测模块及所述行人目标序列提取模块连接;The foreground segmentation module is respectively connected with the pedestrian target detection module and the pedestrian target sequence extraction module;
所述前景分割模块用于采用混合高斯模型分割每帧行人目标的前景和背景;The foreground segmentation module is used to segment the foreground and background of each frame pedestrian target by using a Gaussian mixture model;
所述行人目标检测模块包括行人位置初步估计模块及基于HOG的行人检测模块;The pedestrian target detection module includes a pedestrian position preliminary estimation module and a pedestrian detection module based on HOG;
所述行人位置初步估计模块用于统计该帧对应的前景模板窗口内可能包含存在行人目标的窗口W={wi},其中,窗口内包含的前景面积占窗口面积一半以上,且窗口的高大于Hmin=60像素,宽大于Wmin=30像素,并将存在重合的窗口按照如下步骤进行合并:The pedestrian position preliminary estimation module is used to count the window W={w i } that may contain pedestrian targets in the foreground template window corresponding to the frame, wherein the foreground area contained in the window accounts for more than half of the window area, and the height of the window is Greater than Hmin=60 pixels, width greater than Wmin=30 pixels, and overlapped windows are merged according to the following steps:
将任何两个有重合的窗口标记为同一个集合Si;Mark any two overlapping windows as the same set S i ;
判断集合Si,如果并且有i≠j∧wi1∩wi2≠φ,那么合并Si与Sj;重复判断直至所有包含有重合部分窗口的集合都被合并;To judge the set S i , if And there is i≠j∧w i1 ∩w i2 ≠φ, then merge S i and S j ; repeat the judgment until all sets containing overlapping windows are merged;
将一个面积最小的包含集合Si所有窗口的矩形Ti来代表集合Si中所有的窗口,重新形成窗口集合T={Ti}。A rectangle T i containing all the windows of the set S i with the smallest area is used to represent all the windows in the set S i , and the window set T={T i } is re-formed.
所述基于HOG的行人检测模块是采用HOG特征在初步估计得到的行人位置处进行行人检测。The HOG-based pedestrian detection module uses HOG features to detect pedestrians at the initially estimated pedestrian positions.
在其中一个实施例中,所述行人目标序列提取模块包括行人跟踪模块及前景序列获取模块;In one of the embodiments, the pedestrian target sequence extraction module includes a pedestrian tracking module and a foreground sequence acquisition module;
所述行人跟踪模块用于使用基于学习方法的行人跟踪技术(TLD)对检测到的行人目标进行跟踪,获取行人目标序列;The pedestrian tracking module is used to track the detected pedestrian target by using the pedestrian tracking technology (TLD) based on the learning method, and obtain the pedestrian target sequence;
所述前景序列获取模块用于在行人目标序列中提取对应的前景序列;The foreground sequence acquisition module is used to extract the corresponding foreground sequence in the pedestrian target sequence;
在其中一个实施例中,所述行人序列多直方图特征计算模块包括颜色空间转化模块和单帧图像颜色直方图计算模块;In one of the embodiments, the pedestrian sequence multi-histogram feature calculation module includes a color space conversion module and a single-frame image color histogram calculation module;
所述颜色空间转化模块用于将行人目标图像转换到HSV颜色空间;The color space conversion module is used to convert the pedestrian target image to the HSV color space;
所述单帧图像颜色直方图计算模块包括以下计算步骤:The single frame image color histogram calculation module includes the following calculation steps:
如果行人目标图像像素的灰度值V<Tg,则H1[1]=H1[1]+1;If the gray value of the pedestrian target image pixel V<Tg, then H 1 [1]=H 1 [1]+1;
如果像素灰度值V≥Tg并且饱和度值Sg<S<Sc,则计算模糊区域直方图H2;其中,模糊区域直方图计算包括颜色部分与灰度部分:If the pixel gray value V≥Tg and the saturation value S g < S < S c , then calculate the fuzzy area histogram H 2 ; wherein, the fuzzy area histogram calculation includes the color part and the gray part:
在其中一个实施例中,所述行人序列多直方图特征计算模块还包括行人序列颜色直方图累积计算模块,所述行人序列颜色直方图累积计算模块包括如下计算步骤:In one of the embodiments, the pedestrian sequence multi-histogram feature calculation module also includes a pedestrian sequence color histogram accumulation calculation module, and the pedestrian sequence color histogram accumulation calculation module includes the following calculation steps:
将绝对区域直方图H1与模糊区域直方图H2置零;Set the absolute area histogram H 1 and the fuzzy area histogram H 2 to zero;
计算行人目标序列i的第j帧前景Fi,j的内切椭圆,将内切椭圆与矩形之间部分置为背景;Calculate the inscribed ellipse of the j-th frame foreground F i,j of the pedestrian target sequence i, and set the part between the inscribed ellipse and the rectangle as the background;
将行人目标序列i的第j帧Ri,j对应的前景部分像素按照所述权值进行累加至第j帧Ri,j的绝对区域直方图H1与模糊区域直方图H2,累积值为weight(j),H1[index]=H1[index]+weight(j)或H2[index]=H2[index]+weight(j),直至行人目标序列中所有帧都计算完成;Accumulate the foreground pixels corresponding to the jth frame R i,j of the pedestrian target sequence i to the absolute area histogram H 1 and the blurred area histogram H 2 of the jth frame R i,j according to the weight value, and the accumulated value is weight(j), H 1 [index]=H 1 [index]+weight(j) or H 2 [index]=H 2 [index]+weight(j), until all frames in the pedestrian target sequence are calculated ;
保存绝对区域直方图H1与模糊区域直方图H2为序列i的颜色特征直方图。Save the absolute area histogram H 1 and the fuzzy area histogram H 2 as the color feature histogram of sequence i.
上述基于图像的行人目标检索系统通过基于学习的目标跟踪及前景分割,得到具有鲁棒性的行人检测结果,进而获取行人目标序列及前景序列。再通过计算获取行人目标序列的颜色特征直方图,即对行人目标的服饰颜色进行特征提取及匹配,最后根据公式计算行人目标直方图Hf,并使用测地距离获取行人目标直方图Hf之间的距离,按照测地距离的大小对行人目标序列进行排序。从而获取具有相关性排列的检索结果。上述基于图像的行人目标检索系统避免了分割服饰这一难题,而是在视频中提取出检测跟踪出来的一系列运动目标的前景。通过累计每幅图像的前景可以有效地提取出其服饰信息。因此,基于背景分割的行人检测提高了视频中行人检测的鲁棒性,避免服饰分割有效地提高了行人检索的准确性。The above-mentioned image-based pedestrian target retrieval system obtains robust pedestrian detection results through learning-based target tracking and foreground segmentation, and then obtains pedestrian target sequences and foreground sequences. Then obtain the color feature histogram of the pedestrian target sequence through calculation, that is, perform feature extraction and matching on the clothing color of the pedestrian target, and finally calculate the pedestrian target histogram H f according to the formula, and use the geodesic distance to obtain the pedestrian target histogram H f . The pedestrian target sequence is sorted according to the size of the geodesic distance. In this way, retrieval results with correlation rankings can be obtained. The above-mentioned image-based pedestrian target retrieval system avoids the difficult problem of segmenting clothing, but extracts the foreground of a series of moving targets detected and tracked in the video. Clothing information can be effectively extracted by accumulating the foreground of each image. Therefore, background segmentation-based pedestrian detection improves the robustness of pedestrian detection in videos, and avoiding clothing segmentation effectively improves the accuracy of pedestrian retrieval.
附图说明Description of drawings
图1为基于图像的行人目标检索方法的流程图;Fig. 1 is the flowchart of image-based pedestrian target retrieval method;
图2(a)为获取的行人目标序列示意图;Figure 2(a) is a schematic diagram of the acquired pedestrian target sequence;
图2(b)为行人目标序列对应的前景序列示意图;Figure 2(b) is a schematic diagram of the foreground sequence corresponding to the pedestrian target sequence;
图2(c)为内切椭圆去干扰的示意图;Figure 2(c) is a schematic diagram of inscribed ellipse de-interference;
图3彩色颜色直方图的分布示意图;The distribution schematic diagram of Fig. 3 color color histogram;
图4为权值函数示意图;Fig. 4 is a schematic diagram of weight function;
图5为基于图像的行人目标检索系统的示意图。Fig. 5 is a schematic diagram of an image-based pedestrian target retrieval system.
具体实施方式Detailed ways
如图1所示,为基于图像的行人目标检索方法的流程图。As shown in Figure 1, it is a flowchart of an image-based pedestrian target retrieval method.
一种基于图像的行人目标检索方法,包括以下步骤:An image-based pedestrian target retrieval method, comprising the following steps:
步骤S110,从原始视频图像中获取行人目标序列和相应的前景序列。具体地,检测视频中的行人目标,对检测得到的行人目标进行跟踪获取行人序列,,并根据所述行人目标序列位置从原始视频图像与其对应的背景分割图中获取前景序列。如图2(a)所示,为获取的行人目标序列,图2(b)为行人目标序列对应的前景序列。Step S110, acquiring pedestrian target sequences and corresponding foreground sequences from the original video images. Specifically, pedestrian targets in the video are detected, the detected pedestrian targets are tracked to obtain pedestrian sequences, and the foreground sequences are obtained from the original video image and its corresponding background segmentation map according to the position of the pedestrian target sequence. As shown in Figure 2(a), it is the acquired pedestrian target sequence, and Figure 2(b) is the foreground sequence corresponding to the pedestrian target sequence.
步骤S110具体包括:Step S110 specifically includes:
①采用混合高斯模型分割行人目标的前景和背景。①Using a mixed Gaussian model to segment the foreground and background of pedestrian targets.
混合高斯模型使用K(基本为3到5个)个高斯模型来表征图像中各个像素点的特征,在新一帧图像获得后更新混合高斯模型,用当前图像中的每个像素点与混合高斯模型匹配,如果成功则判定该点为背景点,否则为前景点。The mixed Gaussian model uses K (basically 3 to 5) Gaussian models to characterize the characteristics of each pixel in the image, and updates the mixed Gaussian model after a new frame of image is obtained, using each pixel in the current image and the mixed Gaussian Model matching, if successful, it is determined that the point is a background point, otherwise it is a foreground point.
②初步估计可能的行人位置,具体地,包括以下步骤:② Preliminary estimation of possible pedestrian positions, specifically, includes the following steps:
⑴统计该帧对应前景中包含可能存在行人目标的窗口W={wi},其中,窗口内包含的前景面积占窗口面积一半以上,且窗口的高大于Hmin=60像素,宽大于Wmin=30像素。进行行人目标检测的单帧图像一般是从多帧图像抽取出来,优选地,是从15帧图像抽取的。⑴Statistically, the frame corresponds to the window W={w i } that may contain pedestrian targets in the foreground, where the foreground area contained in the window accounts for more than half of the window area, and the height of the window is greater than Hmin=60 pixels, and the width is greater than Wmin=30 pixels. The single-frame image for pedestrian target detection is generally extracted from multiple frames of images, preferably, extracted from 15 frames of images.
⑵将任何两个有重合的窗口标记为同一个集合Si。(2) Mark any two overlapping windows as the same set S i .
⑶如果并且有i≠j∧wi1∩wi2≠φ,那么合并Si与Sj;重复本步骤直至所有包含有重合部分窗口的集合都被合并。(3) if And there is i≠j∧w i1 ∩w i2 ≠φ, then merge S i and S j ; repeat this step until all sets containing overlapping partial windows are merged.
(4)用一个面积最小的包含集合Si所有窗口的矩形Ti来代表集合Si中所有的窗口,重新形成窗口集合T={Ti}。(4) Use a rectangle T i containing all the windows of the set S i with the smallest area to represent all the windows in the set S i , and re-form the window set T={T i }.
③采用基于梯度方向直方图在合并后的窗口检测行人目标。③ Use the gradient orientation histogram to detect pedestrian targets in the merged window.
图像梯度方向直方图是一种解决人体目标检测的图像描述子,该方法使用梯度方向直方图(Histogram of Oriented Gradients,简称HOG)特征来表达人体,提取人体的外形信息和运动信息,形成丰富的特征集。在本实施例中,采用梯度方向直方图检测合并后的窗口中所包含的行人目标。所采用的分类器阈值为-0.5。The histogram of image gradient orientation is an image descriptor for human target detection. This method uses the Histogram of Oriented Gradients (HOG) feature to express the human body, extracts the shape information and motion information of the human body, and forms a rich feature set. In this embodiment, the gradient direction histogram is used to detect pedestrian targets included in the combined window. The classifier threshold used was -0.5.
④采用基于学习的目标跟踪方法对检测到的行人目标进行跟踪获得行人目标序列。④A learning-based target tracking method is used to track the detected pedestrian targets to obtain pedestrian target sequences.
利用跟踪学习检测(Tracking-Learning-Detection,TLD)算法能够对目标进行长期的持续跟踪,对动态图像序列中的目标进行跟踪。TLD能够对目标进行持续跟踪,即使在可见光下跟踪失效也能够通过红外图像的弥补,从而使跟踪效果更为精确。Using the Tracking-Learning-Detection (TLD) algorithm, the target can be tracked continuously for a long time, and the target in the dynamic image sequence can be tracked. TLD can continuously track the target, even if the tracking fails under visible light, it can be compensated by the infrared image, so that the tracking effect is more accurate.
利用获取到的行人目标序列,从原始视频图片与其对应的背景分割图中,分别获取行人目标序列与前景序列。Using the acquired pedestrian target sequence, the pedestrian target sequence and the foreground sequence are respectively obtained from the original video image and its corresponding background segmentation image.
步骤S130,将行人目标序列i的所有帧对应的前景序列的像素点按照所述权值进行累加,获得绝对区域直方图H1与模糊区域直方图H2,将累积后的绝对区域直方图H1与模糊区域直方图H2记为行人目标序列i的颜色特征直方图。Step S130: Accumulate the pixels of the foreground sequence corresponding to all frames of the pedestrian target sequence i according to the weights to obtain the absolute area histogram H 1 and the blurred area histogram H 2 , and the accumulated absolute area histogram H 1 and the fuzzy area histogram H 2 is recorded as the color feature histogram of the pedestrian target sequence i.
将彩色图像从RGB空间转换为HSV空间,根据人眼的视觉特性,根据颜色饱和度值将彩色颜色分为3个区域,饱和度大于Sc的彩色区域,饱和度小于Sg的灰度区域及饱和度处于之间的模糊区域,彩色区域与灰度区域统称为准确颜色区域。对于彩色区域仅考虑其颜色分量H值,对于灰度区域仅考虑其灰度分量V值,而模糊区域则同时考虑其颜色分量与灰度分量值;另外,对于灰度值太小的灰度分量值都作为灰度中的黑色。Convert the color image from the RGB space to the HSV space. According to the visual characteristics of the human eye, the color color is divided into 3 areas according to the color saturation value, the color area with saturation greater than S c , and the gray area with saturation less than S g and saturation at The fuzzy area between, the color area and the grayscale area are collectively referred to as the accurate color area. For the color area, only the color component H value is considered, for the grayscale area, only the grayscale component V value is considered, and for the blurred area, both the color component and the grayscale component value are considered; in addition, for the grayscale whose grayscale value is too small The component values are all treated as black in grayscale.
RGB色彩模式是工业界的一种颜色标准,是通过对红(R)、绿(G)、蓝(B)三个颜色通道的变化以及它们相互之间的叠加来得到各式各样的颜色的。The RGB color mode is a color standard in the industry. It obtains a variety of colors by changing the three color channels of red (R), green (G), and blue (B) and superimposing them with each other. of.
HSV(也叫HSB)是对RGB色彩空间中点的两种有关系的表示,描述上比RGB更准确的感知颜色联系,并计算简单。H指hue(色相)、S指saturation(饱和度)、L指lightness(亮度)、V指value(色调)、B指brightness(明度)。HSV (also called HSB) is two related representations of points in the RGB color space. It describes more accurate perceptual color connections than RGB, and is simple to calculate. H refers to hue (hue), S refers to saturation (saturation), L refers to lightness (brightness), V refers to value (hue), and B refers to brightness (brightness).
色相(H)是色彩的基本属性,就是平常所说的颜色名称,如红色、黄色等。Hue (H) is the basic attribute of color, which is commonly referred to as the color name, such as red, yellow, etc.
饱和度(S)是指色彩的纯度,越高色彩越纯,低则逐渐变灰,取0-100%的数值。色调(V),亮度(L)取0-100%。Saturation (S) refers to the purity of the color, the higher the color, the purer the color, and the lower it will gradually become gray, taking the value of 0-100%. Hue (V), brightness (L) range from 0-100%.
HSV把颜色描述在圆柱坐标系内的点,这个圆柱的中心轴取值为自底部的黑色到顶部的白色而在它们中间是的灰色,绕这个轴的角度对应于“色相”,到这个轴的距离对应于“饱和度”,而沿着这个轴的高度对应于“亮度”,“色调”或“明度”。HSV describes colors as points in a cylindrical coordinate system. The central axis of this cylinder takes values from black at the bottom to white at the top and gray in between. The angle around this axis corresponds to the "hue", to this axis The distance corresponds to "saturation", while the height along this axis corresponds to "brightness", "hue" or "lightness".
HSV(色相,饱和度,色调)在概念上可以被认为是颜色的倒圆锥体(黑点在下顶点,白色在上底面圆心)。因为HSV是设备依赖的RGB的简单变换,(h,s,l)或(h,s,v)三元组定义的颜色依赖于所使用的特定红色、绿色和蓝色“加法原色”。每个独特的RGB设备都伴随着一个独特的HSV空间。但是(h,s,l)或(h,s,v)三元组在被约束于特定RGB空间。比如,sRGB的时候就变成明确的了。HSV (Hue, Saturation, Hue) can be conceptually thought of as an inverted cone of color (black point at the lower apex, white at the center of the upper base). Because HSV is a simple device-dependent transformation of RGB, the colors defined by (h,s,l) or (h,s,v) triplets depend on the particular red, green, and blue "additive primaries" being used. Each unique RGB device is accompanied by a unique HSV space. But (h, s, l) or (h, s, v) triplets are constrained to a specific RGB space. For example, when sRGB becomes clear.
HSV模型是三原色光模式的一种非线性变换。The HSV model is a nonlinear transformation of the three primary color light modes.
将HSV空间中的颜色分量均匀分为HBINS个间隔,优选地,将彩色空间0-360度平均分为HBINS=100个间隔。灰度值V小于Tg=0.05为黑色区域,灰度区域[Tg,1]均匀分为VBINS个间隔,优选地,VBINS=100。图像的临时彩色直方图包括2个部分:准确颜色区域对应的绝对区域直方图H1,其长度为1+VBINS+HBINS;及模糊颜色对应的区域的模糊区域直方图H2,其长度为VBINS+HBINS,如图3所示。准确颜色区域的绝对区域直方图H1中的灰度区域为1+VBINS,颜色区域HBINS为彩色的,颜色排列依次为红色、黄色、绿色、蓝色和红色,在颜色交叉部分,按各颜色所占比例形成新的颜色。模糊颜色区域的模糊区域直方图H2的灰度区域为VBINS,颜色区域HBINS的颜色排列一致。准确颜色区域的绝对区域直方图H1与最终的颜色直方图Hf是绝对区域直方图H1与模糊区域直方图H2之和,其颜色分布与准确颜色区域的直方图分布一致,长度为1+VBINS+HBINS。最终的颜色直方图定义为:The color components in the HSV space are evenly divided into HBINS intervals, preferably, the color space 0-360 degrees is evenly divided into HBINS=100 intervals. A gray value V less than Tg=0.05 is a black area, and the gray area [Tg, 1] is evenly divided into VBINS intervals, preferably, VBINS=100. The temporary color histogram of the image includes two parts: the absolute region histogram H 1 corresponding to the accurate color region, whose length is 1+VBINS+HBINS; and the fuzzy region histogram H 2 of the region corresponding to the blurred color, whose length is VBINS +HBINS, as shown in Figure 3. The grayscale area in the absolute area histogram H1 of the accurate color area is 1+VBINS, the color area HBINS is colored, and the colors are arranged in the order of red, yellow, green, blue and red. In the color intersection part, according to each color The proportion forms the new color. The gray area of the fuzzy area histogram H2 of the fuzzy color area is VBINS, and the color arrangement of the color area HBINS is consistent. The absolute area histogram H 1 of the accurate color area and the final color histogram H f are the sum of the absolute area histogram H 1 and the fuzzy area histogram H 2 , and its color distribution is consistent with the histogram distribution of the accurate color area, and the length is 1+VBINS+HBINS. The final color histogram is defined as:
其中x∈{0,1},x=0为灰度优先,x=1表示彩色优先。Among them, x∈{0,1}, x=0 means grayscale priority, and x=1 means color priority.
在颜色直方图定义完成后,则计算绝对区域直方图H1与模糊区域直方图H2,具体计算步骤为:After the color histogram is defined, the absolute area histogram H 1 and the fuzzy area histogram H 2 are calculated. The specific calculation steps are:
①将行人目标图像转换到HSV颜色空间。① Convert the pedestrian target image to HSV color space.
②如果行人目标图像像素的灰度值V<Tg,则H1[1]=H1[1]+1。② If the gray value V<Tg of the pedestrian target image pixel, then H 1 [1]=H 1 [1]+1.
⑤如果像素灰度值V≥Tg并且饱和度值Sg<S<Sc,则计算模糊区域直方图H2;其中,模糊区域直方图计算包括颜色部分与灰度部分。⑤ If the pixel gray value V≥Tg and the saturation value S g < S < S c , calculate the fuzzy area histogram H 2 ; wherein, the calculation of the fuzzy area histogram includes the color part and the gray scale part.
优选地,颜色区域的阈值Sc=0.15,Sg=0.05Preferably, the thresholds of color regions S c =0.15, S g =0.05
基于绝对区域直方图H1与模糊区域直方图H2,则颜色特征直方图Hf的计算步骤为:Based on the absolute area histogram H 1 and the fuzzy area histogram H 2 , the calculation steps of the color feature histogram H f are:
①将绝对区域直方图H1与模糊区域直方图H2置零。① Set the absolute area histogram H 1 and the fuzzy area histogram H 2 to zero.
②取第i个行人目标序列及对应的前景序列其中Ni为第i个行人目标序列的序列长度;使用以下公式计算行人目标序列每帧图像对应的权值f,并对行人目标序列i进行加权,取T=Ni*2/3。如图4所示,为T与累积值weight(j)函数示意图。② Take the i-th pedestrian target sequence and the corresponding foreground sequence Where N i is the sequence length of the i-th pedestrian target sequence; use the following formula to calculate the weight f corresponding to each frame image of the pedestrian target sequence, and weight the pedestrian target sequence i, taking T=N i *2/3. As shown in Figure 4, it is a schematic diagram of the function of T and the cumulative value weight(j).
计算行人目标序列的第j帧前景Fi,j的内切椭圆,将内切椭圆与矩形之间部分置为背景。采用内切椭圆能够去除图像中的干扰,如图2(c)所示。Calculate the inscribed ellipse of the foreground F i,j of the jth frame of the pedestrian target sequence, and set the part between the inscribed ellipse and the rectangle as the background. The interference in the image can be removed by using an inscribed ellipse, as shown in Figure 2(c).
③将行人目标序列i的第j帧Ri,j对应的前景部分像素按照所述权值进行累积至第j帧Ri,j的绝对区域直方图H1与模糊区域直方图H2,累积值为weight(j),H1[index]=H1[index]+weight(j)或H2[index]=H2[index]+weight(j),直至行人目标序列中所有帧都计算完成。③Accumulate the foreground pixels corresponding to the jth frame R i,j of the pedestrian target sequence i to the absolute area histogram H 1 and the blurred area histogram H 2 of the jth frame R i,j according to the weight, and accumulate The value is weight(j), H 1 [index]=H 1 [index]+weight(j) or H 2 [index]=H 2 [index]+weight(j), until all frames in the pedestrian target sequence are calculated Finish.
④保存绝对区域直方图H1与模糊区域直方图H2为序列i的颜色特征直方图。④ Save the absolute area histogram H 1 and the fuzzy area histogram H 2 as the color feature histogram of sequence i.
步骤S140,根据公式Step S140, according to the formula
计算行人目标直方图Hf,并使用测地距离计算行人目标直方图Hf之间的距离,按照测地距离的大小对行人目标序列进行排序;其中,x∈{0,1},x=0为灰度优先,x=1表示彩色优先;颜色分量均匀分为HBINS=100个间隔,灰度值V小于阈值Tg=0.05为黑色区域,灰度区域[Tg,1]均匀分为VBINS=10个间隔。Calculate the pedestrian target histogram H f , and use the geodesic distance to calculate the distance between the pedestrian target histograms H f , and sort the pedestrian target sequence according to the size of the geodesic distance; where, x∈{0,1}, x= 0 means grayscale priority, x=1 means color priority; the color components are evenly divided into HBINS=100 intervals, the grayscale value V is less than the threshold Tg=0.05 is a black area, and the grayscale area [Tg,1] is evenly divided into VBINS= 10 intervals.
测地距离EMD(Earth Mover's Distance)用于计算直方图之间的距离,当特征间(bin与bin)的距离可以使用ground distance求得时,用Earth Mover's Distance做相似计算能够得到更精确的结果。The geodesic distance EMD (Earth Mover's Distance) is used to calculate the distance between histograms. When the distance between features (bin and bin) can be obtained using ground distance, using Earth Mover's Distance to do similar calculations can get more accurate results .
上述基于图像的行人目标检索方法通过基于学习的目标跟踪及前景分割,得到具有鲁棒性的行人检测结果,进而获取行人目标序列及前景序列。再通过计算获取行人目标序列的颜色特征直方图,即对行人目标的服饰颜色进行特征提取及匹配,最后根据公式计算行人目标直方图Hf,并使用测地距离获取行人目标直方图Hf之间的距离,按照测地距离的大小对行人目标序列进行排序。从而获取具有相关性排列的检索结果。上述基于图像的行人目标检索方法避免了分割服饰这一难题,而是在视频中提取出检测跟踪出来的一系列运动目标的前景。通过累计每幅图像的前景可以有效地提取出其服饰信息。因此,基于背景分割的行人检测提高了视频中行人检测的鲁棒性,避免服饰分割有效地提高了行人检索的准确性。The above image-based pedestrian target retrieval method obtains robust pedestrian detection results through learning-based target tracking and foreground segmentation, and then obtains pedestrian target sequences and foreground sequences. Then obtain the color feature histogram of the pedestrian target sequence through calculation, that is, perform feature extraction and matching on the clothing color of the pedestrian target, and finally calculate the pedestrian target histogram H f according to the formula, and use the geodesic distance to obtain the pedestrian target histogram H f . The pedestrian target sequence is sorted according to the size of the geodesic distance. In this way, retrieval results with correlation rankings can be obtained. The above-mentioned image-based pedestrian target retrieval method avoids the difficult problem of segmenting clothing, but extracts the foreground of a series of moving targets detected and tracked in the video. Clothing information can be effectively extracted by accumulating the foreground of each image. Therefore, background segmentation-based pedestrian detection improves the robustness of pedestrian detection in videos, and avoiding clothing segmentation effectively improves the accuracy of pedestrian retrieval.
如图5所示,一种基于图像的行人目标检索系统,包括图像获取模块500、行人序列多直方图特征计算模块540、行人序列特征计算模块550及测地距离排序模块560;所述行人序列多直方图特征计算模块540分别与所述图像获取模块500及所述行人序列特征计算模块550连接,所述行人序列特征计算模块550还与所述测地距离排序模块560连接。As shown in Figure 5, an image-based pedestrian target retrieval system includes an image acquisition module 500, a pedestrian sequence multi-histogram feature calculation module 540, a pedestrian sequence feature calculation module 550 and a geodesic distance sorting module 560; the pedestrian sequence The multi-histogram feature calculation module 540 is respectively connected to the image acquisition module 500 and the pedestrian sequence feature calculation module 550 , and the pedestrian sequence feature calculation module 550 is also connected to the geodetic distance sorting module 560 .
图像获取模块500用于从原始视频图像中获取行人目标序列和相应的前景序列。The image acquisition module 500 is used to acquire pedestrian target sequences and corresponding foreground sequences from original video images.
行人序列多直方图特征计算模块540用于采用公式The pedestrian sequence multi-histogram feature calculation module 540 is used to adopt the formula
所述行人序列多直方图特征计算模块540还用于将行人目标序列i的所有帧对应的前景序列的像素点按照所述权值进行累加,获得绝对区域直方图H1与模糊区域直方图H2,The pedestrian sequence multi-histogram feature calculation module 540 is also used to accumulate the pixels of the foreground sequence corresponding to all the frames of the pedestrian target sequence i according to the weights to obtain the absolute area histogram H1 and the blurred area histogram H 2 ,
所述行人序列特征计算模块550用于根据公式The pedestrian sequence feature calculation module 550 is used to calculate according to the formula
计算行人目标直方图Hf;其中,x∈{0,1},x=0为灰度优先,x=1表示彩色优先;颜色分量均匀分为HBINS个间隔,灰度值V小于阈值Tg为黑色区域,灰度区域[Tg,1]均匀分为VBINS个间隔;Calculate pedestrian target histogram H f ; among them, x∈{0,1}, x=0 means gray priority, x=1 means color priority; the color components are evenly divided into HBINS intervals, and the gray value V is less than the threshold Tg is The black area, the gray area [Tg,1] is evenly divided into VBINS intervals;
所述测地距离排序模块560用于使用测地距离计算行人目标直方图Hf之间的距离,按照测地距离的大小对行人目标序列进行排序。The geodesic distance sorting module 560 is used to calculate the distance between the pedestrian target histograms Hf using the geodesic distance, and sort the sequence of pedestrian targets according to the size of the geodesic distance.
图像获取模块500包括前景分割模块510、行人目标检测模块520及行人目标序列提取模块530。The image acquisition module 500 includes a foreground segmentation module 510 , a pedestrian target detection module 520 and a pedestrian target sequence extraction module 530 .
所述前景分割模块510分别与所述行人目标检测模块520及所述行人目标序列提取模块530连接。The foreground segmentation module 510 is connected to the pedestrian target detection module 520 and the pedestrian target sequence extraction module 530 respectively.
所述前景分割模块510用于采用混合高斯模型分割每帧行人目标的前景和背景。The foreground segmentation module 510 is used to segment the foreground and background of pedestrian targets in each frame by using a Gaussian mixture model.
所述行人目标检测模块520包括行人位置初步估计模块522及基于HOG的行人检测模块524。The pedestrian target detection module 520 includes a pedestrian position preliminary estimation module 522 and a pedestrian detection module 524 based on HOG.
所述行人位置初步估计模块522用于统计该帧对应的前景模板窗口内可能包含存在行人目标的窗口W={wi},其中,窗口内包含的前景面积占窗口面积一半以上,且窗口的高大于Hmin=60像素,宽大于Wmin=30像素,并将存在重合的窗口按照如下步骤进行合并:The pedestrian position preliminary estimation module 522 is used to count the window W={w i } that may contain pedestrian targets in the foreground template window corresponding to the frame, wherein the foreground area contained in the window accounts for more than half of the window area, and the window's The height is greater than Hmin=60 pixels, the width is greater than Wmin=30 pixels, and the overlapping windows are merged according to the following steps:
将任何两个有重合的窗口标记为同一个集合Si;Mark any two overlapping windows as the same set S i ;
判断集合Si,如果并且有i≠j∧wi1∩wi2≠φ,那么合并Si与Sj;重复判断直至所有包含有重合部分窗口的集合都被合并;To judge the set S i , if And there is i≠j∧w i1 ∩w i2 ≠φ, then merge S i and S j ; repeat the judgment until all sets containing overlapping windows are merged;
将一个面积最小的包含集合Si所有窗口的矩形Ti来代表集合Si中所有的窗口,重新形成窗口集合T={Ti}。A rectangle T i containing all the windows of the set S i with the smallest area is used to represent all the windows in the set S i , and the window set T={T i } is re-formed.
所述基于HOG的行人检测模块524是采用HOG特征在初步估计得到的行人位置处进行行人检测。The HOG-based pedestrian detection module 524 uses HOG features to perform pedestrian detection at the initially estimated pedestrian position.
行人目标序列提取模块530包括行人跟踪模块532及前景序列获取模块534。The pedestrian target sequence extraction module 530 includes a pedestrian tracking module 532 and a foreground sequence acquisition module 534 .
所述行人跟踪模块532用于使用基于学习方法的行人跟踪技术(TLD)对检测到的行人目标进行跟踪,获取行人目标序列;The pedestrian tracking module 532 is used to track the detected pedestrian targets by using the pedestrian tracking technology (TLD) based on the learning method, and acquire pedestrian target sequences;
所述前景序列获取模块534用于在行人目标序列中提取对应的前景序列。The foreground sequence acquisition module 534 is used to extract the corresponding foreground sequence in the pedestrian target sequence.
行人序列多直方图特征计算模块540包括颜色空间转化模块542和单帧图像颜色直方图计算模块544。The pedestrian sequence multi-histogram feature calculation module 540 includes a color space conversion module 542 and a single frame image color histogram calculation module 544 .
所述颜色空间转化模块542用于将行人目标图像转换到HSV颜色空间。The color space conversion module 542 is used to convert the pedestrian object image into HSV color space.
所述单帧图像颜色直方图计算模块544包括以下计算步骤:The single-frame image color histogram calculation module 544 includes the following calculation steps:
如果行人目标图像像素的灰度值V<Tg,则H1[1]=H1[1]+1;If the gray value of the pedestrian target image pixel V<Tg, then H 1 [1]=H 1 [1]+1;
如果像素灰度值V≥Tg并且饱和度值Sg<S<Sc,则计算模糊区域直方图H2;其中,模糊区域直方图计算包括颜色部分与灰度部分:If the pixel gray value V≥Tg and the saturation value S g < S < S c , then calculate the fuzzy area histogram H 2 ; wherein, the fuzzy area histogram calculation includes the color part and the gray part:
行人序列多直方图特征计算模块540还包括行人序列颜色直方图累积计算模块546,行人序列颜色直方图累积计算模块546包括如下计算步骤:The pedestrian sequence multi-histogram feature calculation module 540 also includes a pedestrian sequence color histogram accumulation calculation module 546, and the pedestrian sequence color histogram accumulation calculation module 546 includes the following calculation steps:
将绝对区域直方图H1与模糊区域直方图H2置零;Set the absolute area histogram H 1 and the fuzzy area histogram H 2 to zero;
计算行人目标序列i的第j帧前景Fi,j的内切椭圆,将内切椭圆与矩形之间部分置为背景;Calculate the inscribed ellipse of the j-th frame foreground F i,j of the pedestrian target sequence i, and set the part between the inscribed ellipse and the rectangle as the background;
将行人目标序列i的第j帧Ri,j对应的前景部分像素按照所述权值进行累加至第j帧Ri,j的绝对区域直方图H1与模糊区域直方图H2,累积值为weight(j),H1[index]=H1[index]+weight(j)或H2[index]=H2[index]+weight(j),直至行人目标序列中所有帧都计算完成;Accumulate the foreground pixels corresponding to the jth frame R i,j of the pedestrian target sequence i to the absolute area histogram H 1 and the blurred area histogram H 2 of the jth frame R i,j according to the weight value, and the accumulated value is weight(j), H 1 [index]=H 1 [index]+weight(j) or H 2 [index]=H 2 [index]+weight(j), until all frames in the pedestrian target sequence are calculated ;
保存绝对区域直方图H1与模糊区域直方图H2为序列i的颜色特征直方图。Save the absolute area histogram H 1 and the fuzzy area histogram H 2 as the color feature histogram of sequence i.
上述基于图像的行人目标检索系统通过提取视频中的行人目标,并提取行人目标的特征,快速地在本视频或其他视频中检索具有相似特征的行人目标。利用提取行人的梯度方向直方图(HOG)特征,然后采用这些特征来区分行人与非行人。在视频中行人是运动的,利用这一有效的信息可以将一般的行人检测算法(如HOG)在视频中更加鲁棒的实现。The image-based pedestrian target retrieval system extracts the pedestrian targets in the video and extracts the features of the pedestrian targets to quickly retrieve pedestrian targets with similar features in this video or other videos. The Histogram of Oriented Gradient (HOG) features of pedestrians are extracted, and then these features are used to distinguish pedestrians from non-pedestrians. Pedestrians are moving in the video, and the general pedestrian detection algorithm (such as HOG) can be implemented more robustly in the video by using this effective information.
上述基于图像的行人目标检索系统通过基于学习的目标跟踪及前景分割,得到具有鲁棒性的行人检测结果,进而获取行人目标序列及前景序列。再通过计算获取行人目标序列的颜色特征直方图,即对行人目标的服饰颜色进行特征提取及匹配,最后根据公式计算行人目标直方图Hf,并使用测地距离获取行人目标直方图Hf之间的距离,按照测地距离的大小对行人目标序列进行排序。从而获取具有相关性排列的检索结果。上述基于图像的行人目标检索系统避免了分割服饰这一难题,而是在视频中提取出检测跟踪出来的一系列运动目标的前景。通过累计每幅图像的前景可以有效地提取出其服饰信息。因此,基于背景分割的行人检测提高了视频中行人检测的鲁棒性,避免服饰分割有效地提高了行人检索的准确性。The above-mentioned image-based pedestrian target retrieval system obtains robust pedestrian detection results through learning-based target tracking and foreground segmentation, and then obtains pedestrian target sequences and foreground sequences. Then obtain the color feature histogram of the pedestrian target sequence through calculation, that is, perform feature extraction and matching on the clothing color of the pedestrian target, and finally calculate the pedestrian target histogram H f according to the formula, and use the geodesic distance to obtain the pedestrian target histogram H f . The pedestrian target sequence is sorted according to the size of the geodesic distance. In this way, retrieval results with correlation rankings can be obtained. The above-mentioned image-based pedestrian target retrieval system avoids the difficult problem of segmenting clothing, but extracts the foreground of a series of moving targets detected and tracked in the video. Clothing information can be effectively extracted by accumulating the foreground of each image. Therefore, background segmentation-based pedestrian detection improves the robustness of pedestrian detection in videos, and avoiding clothing segmentation effectively improves the accuracy of pedestrian retrieval.
以上所述实施例仅表达了本发明的几种实施方式,其描述较为具体和详细,但并不能因此而理解为对本发明专利范围的限制。应当指出的是,对于本领域的普通技术人员来说,在不脱离本发明构思的前提下,还可以做出若干变形和改进,这些都属于本发明的保护范围。因此,本发明专利的保护范围应以所附权利要求为准。The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but should not be construed as limiting the patent scope of the present invention. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention should be based on the appended claims.
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201310169245.2A CN104143077B (en) | 2013-05-09 | 2013-05-09 | Pedestrian target search method and system based on image |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201310169245.2A CN104143077B (en) | 2013-05-09 | 2013-05-09 | Pedestrian target search method and system based on image |
Publications (2)
Publication Number | Publication Date |
---|---|
CN104143077A CN104143077A (en) | 2014-11-12 |
CN104143077B true CN104143077B (en) | 2018-07-03 |
Family
ID=51852247
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201310169245.2A Active CN104143077B (en) | 2013-05-09 | 2013-05-09 | Pedestrian target search method and system based on image |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN104143077B (en) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104881662B (en) * | 2015-06-26 | 2019-03-08 | 北京畅景立达软件技术有限公司 | A kind of single image pedestrian detection method |
CN105893963B (en) * | 2016-03-31 | 2019-03-08 | 南京邮电大学 | A Method for Screening the Best Recognizable Frames for Single Pedestrian Objects in Videos |
CN106023245B (en) * | 2016-04-28 | 2019-01-01 | 绍兴文理学院 | Moving target detecting method under the static background measured based on middle intelligence collection similarity |
CN106250918B (en) * | 2016-07-26 | 2019-08-13 | 大连理工大学 | A kind of mixed Gauss model matching process based on improved soil-shifting distance |
CN106504264B (en) * | 2016-10-27 | 2019-09-20 | 锐捷网络股份有限公司 | Video foreground image extraction method and device |
CN110458045A (en) * | 2019-07-22 | 2019-11-15 | 浙江大华技术股份有限公司 | Acquisition methods, image processing method and the device of response probability histogram |
CN113239222B (en) * | 2021-01-19 | 2023-10-31 | 佳木斯大学 | An image retrieval method based on image information extraction and improved EMD distance |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101616309A (en) * | 2009-07-16 | 2009-12-30 | 上海交通大学 | Non-overlapping visual field multiple-camera human body target tracking method |
CN102194270A (en) * | 2011-06-02 | 2011-09-21 | 杭州电子科技大学 | Statistical method for pedestrian flow based on heuristic information |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8385599B2 (en) * | 2008-10-10 | 2013-02-26 | Sri International | System and method of detecting objects |
-
2013
- 2013-05-09 CN CN201310169245.2A patent/CN104143077B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101616309A (en) * | 2009-07-16 | 2009-12-30 | 上海交通大学 | Non-overlapping visual field multiple-camera human body target tracking method |
CN102194270A (en) * | 2011-06-02 | 2011-09-21 | 杭州电子科技大学 | Statistical method for pedestrian flow based on heuristic information |
Also Published As
Publication number | Publication date |
---|---|
CN104143077A (en) | 2014-11-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN104143077B (en) | Pedestrian target search method and system based on image | |
Yuan et al. | Robust traffic sign recognition based on color global and local oriented edge magnitude patterns | |
CN103761529B (en) | A kind of naked light detection method and system based on multicolour model and rectangular characteristic | |
CN102663354B (en) | Face calibration method and system thereof | |
CN104778453B (en) | A kind of night pedestrian detection method based on infrared pedestrian's brightness statistics feature | |
CN107491762B (en) | A kind of pedestrian detection method | |
CN103268489B (en) | Automotive number plate recognition methods based on sliding window search | |
CN102982350B (en) | A kind of station caption detection method based on color and histogram of gradients | |
Omachi et al. | Detection of traffic light using structural information | |
CN102999753B (en) | License plate locating method | |
CN103824081B (en) | Method for detecting rapid robustness traffic signs on outdoor bad illumination condition | |
CN105975929A (en) | Fast pedestrian detection method based on aggregated channel features | |
CN105608455B (en) | A kind of license plate sloped correcting method and device | |
CN102592288B (en) | Method for matching pursuit of pedestrian target under illumination environment change condition | |
CN104298969B (en) | Crowd size's statistical method based on color Yu HAAR Fusion Features | |
CN102915433B (en) | Character combination-based license plate positioning and identifying method | |
CN107491753A (en) | A kind of parking offense detection method based on background modeling | |
CN102902957A (en) | Video-stream-based automatic license plate recognition method | |
CN105913041A (en) | A Signal Light Recognition Method Based on Pre-calibration | |
CN103544484A (en) | Traffic sign identification method and system based on SURF | |
CN102663366A (en) | Method and system for identifying pedestrian target | |
CN106919910B (en) | Traffic sign identification method based on HOG-CTH combined features | |
CN104077605A (en) | Pedestrian search and recognition method based on color topological structure | |
CN107301378A (en) | The pedestrian detection method and system of Multi-classifers integrated in image | |
CN104182769B (en) | A license plate detection method and system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |