WO2018233205A1 - 使用高斯惩罚检测图像中行人的方法 - Google Patents

使用高斯惩罚检测图像中行人的方法 Download PDF

Info

Publication number
WO2018233205A1
WO2018233205A1 PCT/CN2017/112782 CN2017112782W WO2018233205A1 WO 2018233205 A1 WO2018233205 A1 WO 2018233205A1 CN 2017112782 W CN2017112782 W CN 2017112782W WO 2018233205 A1 WO2018233205 A1 WO 2018233205A1
Authority
WO
WIPO (PCT)
Prior art keywords
pedestrian
bounding box
confidence
detection
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/112782
Other languages
English (en)
French (fr)
Inventor
王文敏
董培磊
范梦迪
王荣刚
李革
董胜富
王振宇
李英
赵辉
高文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University Shenzhen Graduate School
Original Assignee
Peking University Shenzhen Graduate School
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University Shenzhen Graduate School filed Critical Peking University Shenzhen Graduate School
Priority to US16/621,385 priority Critical patent/US11030444B2/en
Publication of WO2018233205A1 publication Critical patent/WO2018233205A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/12—Bounding box

Definitions

  • the invention belongs to the field of information technology, and relates to technologies such as computer vision and pattern recognition, and particularly relates to a method for screening a pedestrian bounding box by using a Gaussian penalty to detect a pedestrian in an image.
  • Pedestrian detection determines whether a pedestrian appears in the input image or video and determines its location.
  • computer vision technology has been widely used in the field of intelligent driving, intelligent video surveillance and robotics.
  • pedestrian detection has the important significance and practical value of improving driving safety and ensuring the safety of pedestrians' lives and property. Therefore, pedestrian detection technology occupies an incomparable important position in the field of computer vision.
  • the current mainstream approach first detects enough bounding boxes in the image and generates a confidence level for each bounding box.
  • the preliminary generated bounding box is then further screened to obtain the final test results.
  • the general screening method is to delete the repeated bounding box directly according to the strategy of non-maximum value suppression according to the level of confidence and the degree of overlap.
  • this screening method ignores the handling of occlusion pedestrians. In densely populated areas, the direct deletion of multiple bounding boxes also removes the bounding box of the occluded pedestrian due to the mutual occlusion of the pedestrians. This makes the detection of occlusion pedestrians poor.
  • the present invention provides a bounding box screening method using Gaussian penalty, which can significantly improve the performance of pedestrian detection, especially for detecting pedestrians.
  • the training data and the test data are respectively obtained from the pedestrian data set (Caltech) disclosed by the California Institute of Technology.
  • the training sample or the test sample is marked by a pedestrian, and the labeling method is to use a rectangular frame to represent the position of the pedestrian, and the rectangle
  • the boxes are represented using the coordinates of the upper left and lower right corners in the image. That is, the coordinates of the upper left and lower right corners of the rectangular frame in the image are used to indicate the position of the pedestrian in the image.
  • the detection model is trained on the training set using a pedestrian detection method (an existing method such as Faster RCNN).
  • the detection model is used to classify the position of the pedestrian on the test data set to obtain a preliminary pedestrian bounding box and its confidence and coordinates.
  • the preliminary pedestrian bounding box can be classified according to the degree of overlap between the bounding boxes;
  • the classification of the preliminary pedestrian bounding box of the present invention includes: there is no overlap between the pedestrian bounding boxes, a single pedestrian bounding box, and a bounding box between the pedestrians.
  • the degree of confidence of the bounding box is punished to different degrees, and the penetrating boundary box confidence is obtained; thus, the repeated bounding box of the individual pedestrians is removed, and the occluded pedestrians are retained.
  • the purpose of the bounding box specifically:
  • ⁇ is the variance
  • score i is the confidence after the penalty of the bounding box bbox i
  • IoU(M, bbox i ) is the degree of overlap (proportion) of the bounding box bbox i and M.
  • the screening method of the present invention divides the pedestrian bounding box into three cases.
  • the first is that there is no overlap between the pedestrian bounding boxes, as shown in Figure 1.
  • the second type is a single pedestrian bounding box. In this case, the overlap between the bounding boxes is large and the confidence is high. as shown in picture 2.
  • the third case is to block the bounding box between pedestrians. Since the occlusion pedestrian area is not completely overlapping, the degree of overlap is smaller than the second case. Based on these three cases, the confidence of the bounding box is punished to varying degrees.
  • the penalty mode is: if there is no overlapping area, the penalty for the bounding box is not penalized; for the larger bounding box, the confidence is greatly penalized; for the bounding box with less overlap, the confidence is less penalized.
  • a pedestrian bounding box with a confidence greater than 0.5 is reserved for each image as the final detection result.
  • the present invention provides a bounding box screening method using Gaussian penalty. After obtaining the preliminary pedestrian bounding box, the confidence of the bounding box is penalized according to the degree of overlap between the bounding boxes. For pedestrians who are completely unobstructed, they can retain the original bounding box. A large confidence penalty is imposed on the repeated bounding box of a single pedestrian area, thereby removing multiple bounding boxes and repeatedly detecting the same pedestrian. A lower confidence penalty is imposed on the bounding box of the occlusion pedestrian area, thereby preserving the bounding box of the occluded pedestrian.
  • this invention can significantly reduce the missed detection rate of pedestrian detection, and improve the detection effect of blocking pedestrians without increasing the computational complexity. The present invention achieves a lower miss detection rate.
  • FIG. 1 is a flow chart of a method for screening a pedestrian bounding box provided by the present invention
  • Each pedestrian bounding box has a corresponding confidence degree; firstly, the bounding box is sorted according to the confidence degree, and the bounding box M with the largest confidence is obtained; then the overlap degree of other bounding boxes with M is calculated, and the boundary is compared according to the overlapping degree. The confidence level of the box is penalized; finally, the bounding box whose confidence is greater than the threshold (if set to 0.5) is selected as the final detection result.
  • FIG. 2 is a schematic diagram of the method for performing pedestrian boundary box screening using the Faster RCNN network structure
  • the entire Faster RCNN network is divided into three parts: input layer, middle layer and output layer; the middle layer includes convolution layer, pooling layer and fully connected layer; the network takes the original picture as input, through convolution, pooling and full The connection obtains the feature of the image, and outputs the category information and the location information of the classification; the output layer includes a classification layer and an alignment layer, a classification layer output category, and an alignment layer output location information.
  • Figure 3 is a pedestrian in the embodiment of the present invention with no overlapping areas at all.
  • FIG. 4 is a schematic diagram of a pedestrian boundary frame obtained by repeated detection of a single pedestrian in an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of detecting a bounding box of a pedestrian in the embodiment of the present invention.
  • Whether or not the above-mentioned pedestrians have occlusion or area overlap refers to the relationship of pedestrians in the black and gray border boxes in the image.
  • the present invention provides a bounding box screening method using Gaussian penalty. After obtaining a preliminary pedestrian bounding box, the confidence of the bounding box is penalized according to the degree of overlap between the bounding boxes; the present invention can significantly reduce the leakage of pedestrian detection.
  • the detection rate improves the detection effect of blocking pedestrians without substantially increasing the computational complexity.
  • each pedestrian bounding box has a corresponding confidence; first, the bounding box is sorted according to the confidence, and the bounding box M with the highest confidence is obtained; Then calculate the overlap degree of other bounding boxes with M, and penalize the confidence of the bounding box according to the degree of overlap; finally select the bounding box whose confidence is greater than the threshold (such as set to 0.5) as the final detection result.
  • the implementation steps of the present invention are specifically as follows:
  • the detection model may select any one of existing methods such as a Histogram of Oriented Gradient and a Faster Region-based Convolutional Neural Network.
  • the present invention is proposed in the literature [1] (S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," TPAMI (2016)). Take the Faster RCNN (Faster Region-based convolutional neural network) as an example to obtain a pedestrian detection model through training.
  • the neural network architecture of the Faster RCNN is shown in Figure 2.
  • the present invention uses the training image of the Caltech data set to fine tune the fully connected layer of the Faster RCNN network.
  • the categories of classification are set to two categories: background classes and rows of humans.
  • the training picture and the ground-truth label are input into the neural network, and the difference propagation between the detection and labeling of the current model is fed back to the network model by using the back propagation algorithm, and the network model is adjusted. parameter.
  • the pedestrian's detection model is obtained.
  • the trained pedestrian detection model is used to detect the preliminary bounding box.
  • the test image is entered into the test model. Detect if pedestrians are included on each image. If a pedestrian is included, the classification and positioning of the detected model is retained, and the coordinate information (x 1 , y 1 , x 2 , y 2 ) of the bounding box of the network output and its confidence are retained. Where (x 1 , y 1 ) is the coordinate of the upper left corner of the bounding box, and (x 2 , y 2 ) is the coordinate of the upper right corner of the bounding box. Each picture selects the top 300 bounding boxes with higher confidence as the initial bounding box.
  • a commonly used screening method is to first select the bounding box M with the highest confidence for each image, and calculate the degree of overlap (proportional) IoU with each of the other bounding boxes bbox i and M. The calculation method is as shown in Equation 1:
  • area (M ⁇ bbox i ) is the intersection of the bounding boxes bbox i and M
  • area (M ⁇ bbox i ) is the union of the bounding boxes bbox i and M.
  • the bounding box bbox i When the degree of overlap with the bounding box M is greater than the threshold thr, it is considered that the bounding box bbox i and the bounding box M are the same pedestrian area, so the bounding box bbox i is deleted.
  • the method of deleting is to set the confidence of the bounding box bbox i to 0, that is, Equation 2:
  • score i is the confidence of the bounding box bbox i , which is obtained by the detection model.
  • This screening method removes the repeated bounding box of a single pedestrian area as shown in FIG. However, if it is in a pedestrian-intensive area and the pedestrians are obscured by each other. Because the occluded pedestrian lacks part of the information, the confidence of the generally occluded pedestrian bounding box is less than the confidence of other pedestrian bounding boxes. As shown in Figure 4, the confidence of the black box is greater than the confidence of the gray box, and the gray box is Black boxes have a larger overlap. At this point, according to the above screening method, the gray bounding box will be deleted, thereby causing the missed detection of the blocked pedestrian.
  • the bounding box can be divided into three cases.
  • the first is that there is no overlap between the pedestrian bounding boxes, as shown in Figure 3.
  • the second type is a single pedestrian bounding box.
  • the overlap between the bounding boxes is large and the confidence is high, as shown in Figure 4.
  • the third case is to block the bounding box between pedestrians. Since the occlusion pedestrian area is not completely overlapping, the degree of overlap is smaller than the second case, as shown in FIG. Based on these three cases, the confidence of the bounding box is Gaussian based on the degree of overlap.
  • the penalty mode is: if there is no overlapping area, the penalty for the bounding box is not penalized; for the larger bounding box, the confidence is greatly penalized; for the bounding box with less overlap, the confidence is less penalized. This achieves the purpose of removing the repeated bounding box of a single pedestrian while retaining the bounding box of the occluded pedestrian.
  • the screening process of the bounding box specifically includes the following steps:
  • Equation 1 The degree of overlap (proportion) is calculated by Equation 1.
  • Equation 3 is used to update the confidence of the preliminary bounding box of each image, and the updated bounding box with the confidence greater than the set threshold (for example, setting the threshold of 0.5) is used as the final detection result.
  • Table 1 gives the pedestrian detection effect after using the present invention on the Caltech data set and comparison with the existing methods.
  • the existing methods in the first nine items in Table 1 correspond to the methods described in the following documents [2] to [10]:
  • the detection effect is measured by the average log miss rate–fppi (false positive per image) value, where miss rate is the missed detection rate and fppi is the false detection rate of each frame of image.
  • miss rate is the missed detection rate
  • fppi is the false detection rate of each frame of image. The lower the miss rate–fppi value, the better the detection.
  • the pedestrian detection effect using the method of the present invention achieves the lowest miss rate–fppi value compared to other methods.
  • the use of the bounding box screening method of the present invention reduces the 3% miss detection rate compared to the original method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

本发明公布了一种使用高斯惩罚检测图像中行人的方法，在行人检测过程中，使用高斯惩罚对获得的初步行人边界框进行筛选，从而提升对图像中行人尤其是遮挡行人的检测性能；包括：获取行人检测图像的训练数据集、测试数据集和行人标注；使用行人检测方法用训练数据集训练得到检测模型，获取初步的行人边界框及其置信度和坐标；对行人边界框的置信度进行高斯惩罚，得到惩罚后的行人边界框置信度；通过行人边界框筛选得到最终的行人边界框，从而达到去除单个行人的重复边界框，而保留被遮挡行人的边界框的目的，由此实现对图像中行人的检测。本发明能够显著降低行人检测的漏检率，提高遮挡行人的检出率。

Description

使用高斯惩罚检测图像中行人的方法 技术领域
本发明属于信息技术领域，涉及计算机视觉，模式识别等技术，具体涉及使用高斯惩罚的方式进行行人边界框的筛选，从而检测图像中行人的方法。
背景技术
行人检测即判断输入的图像或视频中是否出现行人，并确定其位置。近年来，随着计算机视觉领域飞速发展，计算机视觉技术在智能驾驶，智能视频监控和机器人领域得到大量地应用。行人检测作为保障汽车、行人安全的一种主动安全手段，具有提高驾驶安全性、保障行人生命财产安全的重要意义和实用价值。因此行人检测技术在计算机视觉领域占据着不可比拟的重要地位。
由于行人间的相互遮挡或者行人与其他物体之间的遮挡，造成被遮挡行人在图片或者视频中只有局部的信息，因此行人检测目前所面临的一个主要的挑战是遮挡行人。目前主流的方法首先在图像中检测到足够多的边界框，并对每个边界框生成一个置信度。之后对初步生成的边界框进一步的筛选，获得最后的检测结果。普遍的筛选方法是依据置信度的高低和重叠度大小按照非极大值抑制的策略将重复边界框直接删除。但是这种筛选方法忽略了对遮挡行人的处理。在人群密集的区域，由于行人间的相互遮挡，对多个边界框的直接删除也去除了被遮挡行人的边界框。使得对遮挡行人的检测效果表现不佳。
发明内容
为了克服现有技术的不足，本发明提供一种使用高斯惩罚的边界框筛选方法，能够显著提升行人检测的性能，尤其是对遮挡行人的检测。
本发明提供的技术方案是：
一种使用高斯惩罚检测图像中行人的方法，在行人检测过程中，使用高斯惩罚对获得的初步行人边界框进行筛选，从而提升对图像中行人尤其是遮挡行人的检测性能；包括如下步骤：
1)获取训练数据、测试数据及行人数据标签；
本发明实施例从加州理工大学公开的行人数据集(Caltech)分别得到训练数据和测试数据，训练样本或测试样本均有行人的标注，标注的方式是使用矩形框代表行人的位置，矩形 框使用在图像中的左上角和右下角的坐标来表示。即采用矩形框在图像中的左上角和右下角的坐标来表示行人在图像中的位置。
2)训练模型并检测，获取初步的行人边界框及其置信度和坐标；
使用行人检测方法(现有方法如Faster RCNN)在训练集上训练得到检测模型。使用检测模型在测试数据集上分类定位行人的位置，获得初步的行人边界框及其置信度和坐标。
3)对边界框的置信度进行高斯惩罚，通过行人边界框筛选得到最终的行人边界框，由此实现对图像中行人的检测；包括如下步骤：
31)可依据边界框之间重叠度的大小对初步的行人边界框进行分类；
具体地，本发明对初步的行人边界框的分类包括：行人边界框之间完全没有重叠、单个行人边界框、遮挡行人间的边界框。
32)针对不同种类的初步的行人边界框，对边界框的置信度进行不同程度的惩罚，得到惩罚后的行人边界框置信度；从而达到去除单个行人的重复边界框，而保留被遮挡行人的边界框的目的；具体为：
对于每一幅图像，根据置信度对边界框进行排序，选择置信度最大的边界框M；
通过式1计算其他每一个边界框bboxi与边界框M的重叠比例IoU(M,bboxi)：
Figure PCTCN2017112782-appb-000001
其中，area(M∩bboxi)是边界框bboxi与M的区域交集，area(M∪bboxi)是边界框bboxi与M的区域并集；
通过式3对边界框的置信度进行惩罚：
Figure PCTCN2017112782-appb-000002
其中，e为自然对数，σ是方差；scorei为边界框bboxi惩罚后的置信度；IoU(M,bboxi)是边界框bboxi与M的重叠度(比例)。
33)设置置信度阈值，对每张图像将惩罚后的置信度大于所设置的置信度阈值(例如0.5)的行人边界框，作为最终检测结果；由此检测得到图像中的行人。
本发明的筛选方法将行人边界框分为三种情况。第一种是行人边界框之间完全没有重叠，如图1所示。第二种是单个行人边界框，这种情况下边界框之间的重叠度较大且置信度较高， 如图2所示。第三种情况是遮挡行人间的边界框，由于遮挡行人间区域并不是完全的重叠，所以重叠度大小小于第二种情况。基于这三种情况，对边界框的置信度进行不同程度的惩罚。惩罚方式为：如果没有重叠区域，对边界框置信度不惩罚；对于重叠度较大边界框，对置信度进行较大惩罚；对于重叠度较小的边界框，对置信度进行较小惩罚。最后对每张图像保留置信度大于0.5的行人边界框作为最终检测结果。
与现有技术相比，本发明的有益效果是：
本发明提供一种使用高斯惩罚的边界框筛选方法。在获得初步的行人边界框之后，依据边界框之间重叠度的大小对边界框的置信度进行惩罚。对于完全没有遮挡的行人，能够保留原有的边界框。对单个行人区域的重复边界框进行较大的置信度惩罚，从而去除多个边界框重复检测同一个行人。对遮挡行人区域的边界框进行较小的置信度惩罚，从而保留有可能是被遮挡行人的边界框。实验表明，此发明方案能够显著降低行人检测的漏检率，在基本不增加计算复杂度的前提下，提高了遮挡行人的检测效果。本发明实现了较低的漏检率。
附图说明
图1是本发明提供的行人边界框筛选方法的流程框图；
其中，每个行人边界框都有一个对应的置信度；首先根据置信度对边界框进行排序，获得置信度最大的边界框M；然后计算其他边界框与M的重叠度，根据重叠度对边界框的置信度进行处罚；最后选择置信度大于阈值(如设置为0.5)的边界框作为最终的检测结果。
图2是本发明进行行人边界框筛选采用Faster RCNN网络结构的示意图；
其中，整个Faster RCNN网络分为三部分：输入层、中间层和输出层；中间层包括卷积层、池化层和全连接层；网络以原始图片作为输入，经过卷积、池化和全连接获得图像的特征，输出分类的类别信息和位置信息；输出层包括分类层和定位层，分类层输出类别，定位层输出位置信息。
图3是本发明实施例中完全没有重叠区域的行人。
图4是本发明实施例中单个行人的重复检测得到行人边界框的示意图。
图5是本发明实施例中遮挡行人的边界框检测示意图。
其中以上所述的行人是否存在遮挡或者区域重叠均指在图像中使用黑色和灰色边界框中的行人的关系。
具体实施方式
下面结合附图，通过实施例进一步描述本发明，但不以任何方式限制本发明的范围。
本发明提供一种使用高斯惩罚的边界框筛选方法，在获得初步的行人边界框之后，依据边界框之间重叠度的大小对边界框的置信度进行惩罚；本发明能够显著降低行人检测的漏检率，在基本不增加计算复杂度的前提下，提高了遮挡行人的检测效果。
图1是本发明提供的行人边界框筛选方法的流程框图；其中，每个行人边界框都有一个对应的置信度；首先根据置信度对边界框进行排序，获得置信度最大的边界框M；然后计算其他边界框与M的重叠度，根据重叠度对边界框的置信度进行处罚；最后选择置信度大于阈值(如设置为0.5)的边界框作为最终的检测结果。本发明实施步骤具体如下：
(一)训练检测模型并检测获得初步边界框；
1)训练检测模型；
检测模型可以选择现有方法如梯度直方图方法(Histogram of Oriented Gradient)、区域卷积神经网络(Faster Region-based convolutional neural network)等中的任意一种。本发明以文献[1](S.Ren,K.He,R.Girshick,and J.Sun,“Faster R-CNN:TowardsReal-Time Object Detection with Region Proposal Networks,”，TPAMI(2016))提出的检测模型Faster RCNN(Faster Region-based convolutional neural network)为例，通过训练得到行人检测模型。
Faster RCNN的神经网络架构如图2所示。本发明利用Caltech数据集的训练图像对Faster RCNN网络的全连接层进行微调。在行人检测中，将分类的类别设为两类：背景类和行人类。训练过程中，将训练图片及标注信息(ground-truth label)输入至神经网络中，利用反向传播算法将目前模型的检测和标注之间的差异损失回馈到网络模型中，并调整网络模型的参数。当在整个训练集上多次迭代完成后，获得行人的检测模型。
2)在Caltech数据集的测试图像上，利用训练好的行人检测模型，检测获得初步的边界框。
将测试图像输入至检测模型中。检测每张图像上是否包含行人。如果包含行人，经过检测模型的分类和定位，保留网络输出的边界框的坐标信息(x1,y1,x2,y2)及其置信度(score)。其中(x1,y1)是边界框的左上角的坐标，(x2,y2)是边界框的右上角的坐标。每张图片选取置信度较高的前300个边界框作为初步的边界框。
(二)边界框的筛选
由于检测模型分类的能力限制，初步获取的边界框中在一个行人周围分类模型会定位到 密集的重叠度较大且置信度较高的多个边界框。普遍采取的筛选方法是对于每一幅图像，首先选择置信度最大的边界框M，计算与其他每一个边界框bboxi与M的重叠度(比例)IoU。计算方法如式1：
Figure PCTCN2017112782-appb-000003
其中，area(M∩bboxi)是边界框bboxi与M的区域交集，area(M∪bboxi)是边界框bboxi与M的区域并集。
当与边界框M的重叠度大于阈值thr时，就认为边界框bboxi与边界框M为同一个行人区域，所以就删除掉边界框bboxi。删除的方法为将边界框bboxi的置信度置为0，即式2：
Figure PCTCN2017112782-appb-000004
其中，scorei为边界框bboxi的置信度，由检测模型获得。
这种筛选方式可以去除如图4所示的单个行人区域的重复边界框。但是如果在行人密集的区域且行人间相互遮挡。由于被遮挡行人缺少部分信息，所以一般被遮挡行人边界框的置信度小于其他行人边界框的置信度，如图4所示，黑色框的置信度大于灰色框的置信度，而灰色框又与黑色框有较大区域的重叠。此时按照以上筛选方式就会删除灰色边界框，从而造成被遮挡行人的漏检。
根据本发明的筛选方法，可将边界框分为三种情况。第一种是行人边界框之间完全没有重叠，如图3所示。第二种是单个行人边界框，这种情况下边界框之间的重叠度较大且置信度较高，如图4所示。第三种情况是遮挡行人间的边界框，由于遮挡行人间区域并不是完全的重叠，所以重叠度大小小于第二种情况，如图5所示。基于这三种情况，对边界框的置信度进行依据重叠度大小进行高斯惩罚。惩罚方式为：如果没有重叠区域，对边界框置信度不惩罚；对于重叠度较大边界框，对置信度进行较大惩罚；对于重叠度较小的边界框，对置信度进行较小惩罚。从而达到去除单个行人的重复边界框，而保留被遮挡行人的边界框的目的。
边界框的筛选过程具体包括如下步骤：
(一)对于每一幅图像，首先根据置信度对边界框排序，选择置信度最大的边界框M。
(二)通过式1计算与其他每一个边界框bboxi与M的重叠比例IoU。
(三)对每个边界框进行置信度处罚，对置信度的惩罚表示为：
Figure PCTCN2017112782-appb-000005
其中，e为自然对数，σ是方差；scorei为初始边界框的置信度；Si为边界框bboxi惩罚后的置信度；IoU(M,bboxi)是边界框bboxi与M的重叠度(比例)，由式1计算得到。
最后使用惩罚公式(式3)对每一张图像的初步边界框的置信度进行更新，将更新后的置信度大于设置阈值(比如设置阈值为0.5)的边界框作为最终检测结果。
表1给出了在Caltech数据集上使用本发明后的行人检测效果及与现有方法的对比。表1中前9项的现有方法，分别对应以下文献[2]～[10]记载的方法：
[2]Paul Viola,Michael J Jones,and Daniel Snow.2005.Detectingpedestrians using patterns of motion and appearance.International Journal of Computer Vision 63,2(2005),153–161.
[3]Pierre Sermanet,Koray Kavukcuoglu,Soumith Chintala,and Yann LeCun.2013.Pedestrian detection with unsupervised multistage feature learning.In Computer Vision and Pattern Recognition.IEEE,3626–3633.
[4]Navneet Dalal and Bill Triggs.2005.Histograms of orientedgradients for human detection.In Computer Vision and PatternRecognition.IEEE,886–893.
[5]Xiaoyu Wang,Tony X Han,and Shuicheng Yan.2009.An HOGLBP human detector with partial occlusion handling.In International Conference on Computer Vision.32–39.
[6]Piotr Doll′ar,Serge J Belongie,and Pietro Perona.2010.TheFastest Pedestrian Detector in the West.In British Machine Vision Conference,Vol.2.Citeseer,7.
[7]Piotr Doll′ar,Zhuowen Tu,Pietro Perona,and Serge Belongie.2009.Integral channel features.In British Machine Vision Conference.
[8]Piotr Doll′ar,Ron Appel,Serge Belongie,and Pietro Perona.2014.Fast feature pyramids for object detection.IEEE transactionson pattern analysis and machine intelligence 36,8(2014),1532–1545.
[9]Wanli Ouyang,Xingyu Zeng,and Xiaogang Wang.2013.Modeling mutual visibility relationship in pedestrian detection.InComputer Vision and Pattern Recognition.IEEE,3222–3229.
[10]Ping Luo,Yonglong Tian,Xiaogang Wang,and Xiaoou Tang.2014.Switchable deep network for pedestrian detection.In Computer Vision and Pattern Recognition.IEEE,899–906.
表1中，检测效果用平均对数miss rate–fppi(false positive per image)值衡量，其中miss rate为漏检率，fppi指每帧图像的误检率。该miss rate–fppi值越低，说明检测效果越好。
表1 Caltech数据集上的检测效果
检测方法 检测效果(miss rate–fppi值)
VJ(文献[2]) 0.95
ConvNet(文献[3]) 0.77
HOG(文献[4]) 0.68
HOGLBP(文献[5]) 0.68
FPDW(文献[6] 0.57
ChnFtrs(文献[7]) 0.56
ACF(文献[8]) 0.51
DBN-Mut(文献[9]) 0.48
SDN(文献[10]) 0.38
Faster RCNN(文献[1]) 0.27
本发明方法 0.24
从表中可以看出，采用本发明方法的行人检测效果和其他方法相比，达到了最低的miss rate–fppi值。采用Faster RCNN模型，使用本发明的边界框筛选方法比原有方法降低了3％的漏检率。
需要注意的是，公布实施例的目的在于帮助进一步理解本发明，但是本领域的技术人员可以理解：在不脱离本发明及所附权利要求的精神和范围内，各种替换和修改都是可能的。因此，本发明不应局限于实施例所公开的内容，本发明要求保护的范围以权利要求书界定的范围为准。

Claims (5)

  1. 一种使用高斯惩罚检测图像中行人的方法，在行人检测过程中，使用高斯惩罚对获得的初步行人边界框进行筛选，从而提升对图像中行人尤其是遮挡行人的检测性能；包括如下步骤：
    1)获取行人检测图像的训练数据集、测试数据集和行人标注；
    2)使用行人检测方法用训练数据集训练得到检测模型，获取初步的行人边界框的坐标及其置信度；
    3)对行人边界框的置信度进行高斯惩罚，得到惩罚后的行人边界框置信度；通过行人边界框筛选得到最终的行人边界框，从而达到去除单个行人的重复边界框，而保留被遮挡行人的边界框的目的，由此实现对图像中行人的检测；包括如下步骤：
    31)对于每一幅图像，根据置信度对边界框进行排序，选择置信度最大的边界框M；
    32)通过式1计算其他每一个边界框bboxi与边界框M的重叠度IoU(M,bboxi)：
    Figure PCTCN2017112782-appb-100001
    其中，area(M∩bboxi)是边界框bboxi与M的区域交集，area(M∪bboxi)是边界框bboxi与M的区域并集；
    通过式3对边界框的置信度进行惩罚，得到惩罚后的行人边界框置信度：
    Figure PCTCN2017112782-appb-100002
    其中，e为自然对数，σ是方差；scorei为初始边界框的置信度；Si为边界框bboxi惩罚后的置信度；IoU(M,bboxi)是边界框bboxi与M的重叠度；
    33)设置置信度阈值，对每张图像，将惩罚后的置信度大于所设置的置信度阈值的行人边界框，作为最终检测结果；由此检测得到图像中的行人。
  2. 如权利要求1所述方法，其特征是，步骤1)具体从行人数据集Caltech中分别得到训练数据和测试数据，所述训练数据和测试数据均有行人标注，所述行人标注采用矩形框在图像中的左上角和右下角的坐标来表示行人在图像中的位置。
  3. 如权利要求1所述方法，其特征是，步骤2)具体采用检测模型Faster RCNN进行训练，利用训练数据集的图像对Faster RCNN神经网络的全连接层进行微调；将训练图像和标注信息输入Faster RCNN神经网络中，通过反向传播算法将模型检测和标注之间的差异损失 回馈到模型中，并调整网络模型的参数；在训练数据集上多次迭代后，获得行人检测模型；再将测试数据集输入行人的检测模型，对行人的位置进行分类和定位，由此获得初步的行人边界框及其置信度和坐标。
  4. 如权利要求1所述方法，其特征是，步骤33)设置置信度阈值为0.5。
  5. 如权利要求1所述方法，其特征是，步骤2)所述获取的初步的行人边界框类别包括：行人边界框之间完全没有重叠的边界框、单个行人边界框、遮挡行人间的边界框。
PCT/CN2017/112782 2017-06-21 2017-11-24 使用高斯惩罚检测图像中行人的方法 Ceased WO2018233205A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/621,385 US11030444B2 (en) 2017-06-21 2017-11-24 Method for detecting pedestrians in image by using Gaussian penalty

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710474582.0 2017-06-21
CN201710474582.0A CN109101859A (zh) 2017-06-21 2017-06-21 使用高斯惩罚检测图像中行人的方法

Publications (1)

Publication Number Publication Date
WO2018233205A1 true WO2018233205A1 (zh) 2018-12-27

Family

ID=64735476

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/112782 Ceased WO2018233205A1 (zh) 2017-06-21 2017-11-24 使用高斯惩罚检测图像中行人的方法

Country Status (3)

Country Link
US (1) US11030444B2 (zh)
CN (1) CN109101859A (zh)
WO (1) WO2018233205A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110059584A (zh) * 2019-03-28 2019-07-26 中山大学 一种结合边界分布与纠正的事件提名方法
CN110765951A (zh) * 2019-10-24 2020-02-07 西安电子科技大学 基于边界框修正算法的遥感图像飞机目标检测方法
CN112528851A (zh) * 2020-12-09 2021-03-19 南京航空航天大学 一种人脸检测方法及系统
CN114078226A (zh) * 2021-11-25 2022-02-22 北京理工大学 一种基于动作管道在线关联的智联生产线行为识别方法
CN116092123A (zh) * 2023-02-07 2023-05-09 广联达科技股份有限公司 行人图像质量的评估方法、装置、设备和可读存储介质

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108009544B (zh) * 2017-12-13 2021-08-31 北京小米移动软件有限公司 目标检测方法及装置
CN110033472B (zh) * 2019-03-15 2021-05-11 电子科技大学 一种复杂红外地面环境下的稳定目标跟踪方法
CN110046600B (zh) * 2019-04-24 2021-02-26 北京京东尚科信息技术有限公司 用于人体检测的方法和装置
CN110298238B (zh) * 2019-05-20 2023-06-30 平安科技（深圳）有限公司 行人视觉跟踪方法、模型训练方法、装置、设备及存储介质
CN110414447B (zh) * 2019-07-31 2022-04-15 京东方科技集团股份有限公司 行人跟踪方法、装置及设备
US11043003B2 (en) 2019-11-18 2021-06-22 Waymo Llc Interacted object detection neural network
CN111881714B (zh) * 2020-05-22 2023-11-21 北京交通大学 一种无监督跨域行人再识别方法
CN111986252B (zh) * 2020-07-16 2024-03-29 浙江工业大学 一种用于目标分割网络中候选边界框精准定位的方法
US12045992B2 (en) * 2020-11-10 2024-07-23 Nec Corporation Multi-domain semantic segmentation with label shifts
CN112613564A (zh) * 2020-12-25 2021-04-06 桂林汉璟智能仪器有限公司 一种剔除重叠框的目标检测后处理方法
CN112949630B (zh) * 2021-03-01 2024-03-19 北京交通大学 基于边框分级筛选的弱监督目标检测方法
CN112949526B (zh) * 2021-03-12 2024-03-29 深圳海翼智新科技有限公司 人脸检测方法和装置
CN113420682B (zh) * 2021-06-28 2023-08-15 阿波罗智联(北京)科技有限公司 车路协同中目标检测方法、装置和路侧设备
WO2024185106A1 (ja) * 2023-03-09 2024-09-12 三菱電機株式会社 物体検出装置及び物体検出方法
US20250095201A1 (en) * 2023-09-15 2025-03-20 Qualcomm Incorporated Machine-learning based object localization from images
CN118155293B (zh) * 2024-05-09 2024-07-05 东南大学 一种基于小目标追踪的行人过街危险行为判别方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103093214A (zh) * 2013-02-01 2013-05-08 浙江捷尚视觉科技有限公司 一种基于车载红外相机的行人检测方法
EP2860661A1 (en) * 2013-10-09 2015-04-15 Delphi Technologies, Inc. Mean shift tracking method
CN105975925A (zh) * 2016-05-03 2016-09-28 电子科技大学 基于联合检测模型的部分遮挡行人检测方法
CN106845374A (zh) * 2017-01-06 2017-06-13 清华大学 基于深度学习的行人检测方法及检测装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9881234B2 (en) * 2015-11-25 2018-01-30 Baidu Usa Llc. Systems and methods for end-to-end object detection
US9996772B2 (en) * 2016-04-28 2018-06-12 International Business Machines Corporation Detection of objects in images using region-based convolutional neural networks
CN106022237B (zh) * 2016-05-13 2019-07-12 电子科技大学 一种端到端的卷积神经网络的行人检测方法
CN106203506B (zh) * 2016-07-11 2019-06-21 上海凌科智能科技有限公司 一种基于深度学习技术的行人检测方法
US10192314B2 (en) * 2016-12-14 2019-01-29 Canon Kabushiki Kaisha Method, system and apparatus for determining a lowest point of a target object in an image

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103093214A (zh) * 2013-02-01 2013-05-08 浙江捷尚视觉科技有限公司 一种基于车载红外相机的行人检测方法
EP2860661A1 (en) * 2013-10-09 2015-04-15 Delphi Technologies, Inc. Mean shift tracking method
CN105975925A (zh) * 2016-05-03 2016-09-28 电子科技大学 基于联合检测模型的部分遮挡行人检测方法
CN106845374A (zh) * 2017-01-06 2017-06-13 清华大学 基于深度学习的行人检测方法及检测装置

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110059584A (zh) * 2019-03-28 2019-07-26 中山大学 一种结合边界分布与纠正的事件提名方法
CN110765951A (zh) * 2019-10-24 2020-02-07 西安电子科技大学 基于边界框修正算法的遥感图像飞机目标检测方法
CN110765951B (zh) * 2019-10-24 2023-03-10 西安电子科技大学 基于边界框修正算法的遥感图像飞机目标检测方法
CN112528851A (zh) * 2020-12-09 2021-03-19 南京航空航天大学 一种人脸检测方法及系统
CN112528851B (zh) * 2020-12-09 2024-03-29 南京航空航天大学 一种人脸检测方法及系统
CN114078226A (zh) * 2021-11-25 2022-02-22 北京理工大学 一种基于动作管道在线关联的智联生产线行为识别方法
CN116092123A (zh) * 2023-02-07 2023-05-09 广联达科技股份有限公司 行人图像质量的评估方法、装置、设备和可读存储介质

Also Published As

Publication number Publication date
US20200160048A1 (en) 2020-05-21
CN109101859A (zh) 2018-12-28
US11030444B2 (en) 2021-06-08

Similar Documents

Publication Publication Date Title
WO2018233205A1 (zh) 使用高斯惩罚检测图像中行人的方法
CN110427905B (zh) 行人跟踪方法、装置以及终端
CN107230202B (zh) 路面病害图像的自动识别方法和系统
CN106651846B (zh) 视网膜血管图像的分割方法
CN103854273B (zh) 一种近正向俯视监控视频行人跟踪计数方法和装置
CN103116756B (zh) 一种人脸检测与跟踪方法及装置
CN105893946B (zh) 一种正面人脸图像的检测方法
CN110222644A (zh) 基于图像分割的森林火灾烟雾检测方法
CN107657225B (zh) 一种基于聚合通道特征的行人检测方法
WO2019114036A1 (zh) 人脸检测方法及装置、计算机装置和计算机可读存储介质
CN103955949B (zh) 基于Mean-shift算法的运动目标检测方法
CN110415208A (zh) 一种自适应目标检测方法及其装置、设备、存储介质
CN105447503B (zh) 基于稀疏表示lbp和hog融合的行人检测方法
CN107609512A (zh) 一种基于神经网络的视频人脸捕捉方法
CN104268583A (zh) 基于颜色区域特征的行人重识别方法及系统
CN108805897A (zh) 一种改进的运动目标检测vibe算法
CN106897673A (zh) 一种基于retinex算法和卷积神经网络的行人再识别方法
CN114863198B (zh) 一种基于神经网络的小龙虾质量分级方法
CN109886128A (zh) 一种低分辨率下的人脸检测方法
CN111611907A (zh) 一种图像增强的红外目标检测方法
CN104599291B (zh) 基于结构相似度和显著性分析的红外运动目标检测方法
CN108538052A (zh) 基于车头灯轨迹跟踪和动态配对的夜间交通流量检测方法
CN109190456A (zh) 基于聚合通道特征和灰度共生矩阵的多特征融合俯视行人检测方法
CN112446417B (zh) 基于多层超像素分割的纺锤形果实图像分割方法及系统
CN110263670A (zh) 一种面部局部特征分析系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17914608

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17914608

Country of ref document: EP

Kind code of ref document: A1