TWI729454B - Open scene real-time crowd flow statistics method and device, computer equipment and computer readable storage medium - Google Patents

Open scene real-time crowd flow statistics method and device, computer equipment and computer readable storage medium Download PDF

Info

Publication number
TWI729454B
TWI729454B TW108128880A TW108128880A TWI729454B TW I729454 B TWI729454 B TW I729454B TW 108128880 A TW108128880 A TW 108128880A TW 108128880 A TW108128880 A TW 108128880A TW I729454 B TWI729454 B TW I729454B
Authority
TW
Taiwan
Prior art keywords
pedestrian
statistical area
frame
valid frame
open scene
Prior art date
Application number
TW108128880A
Other languages
Chinese (zh)
Other versions
TW202036375A (en
Inventor
張曉博
侯章軍
楊旭東
Original Assignee
開曼群島商創新先進技術有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 開曼群島商創新先進技術有限公司 filed Critical 開曼群島商創新先進技術有限公司
Publication of TW202036375A publication Critical patent/TW202036375A/en
Application granted granted Critical
Publication of TWI729454B publication Critical patent/TWI729454B/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition

Abstract

本說明書提供一種開放場景的即時人群流量統計方法，包括：從拍攝所述開放場景的即時視訊串流中提取當前有效框；所述視訊串流由安置在開放場景上方的攝影機拍攝；檢測當前有效框中的每個行人；採用行人重識別演算法，識別當前有效框中與之前的至少一個有效框中相同的行人；根據行人重識別演算法的結果，計算經過所述開放場景的人群流量數量。 This manual provides an open scene real-time crowd flow statistics method, including: extracting the currently valid frame from the real-time video stream for shooting the open scene; the video stream is taken by a camera placed above the open scene; detecting the current validity Each pedestrian in the frame; using a pedestrian re-recognition algorithm to identify the same pedestrian in the current valid frame as at least one of the previous valid frames; according to the result of the pedestrian re-recognition algorithm, calculate the amount of crowd flow through the open scene .

Description

開放場景的即時人群流量統計方法和裝置、電腦設備及電腦可讀儲存媒體 Open scene real-time crowd flow statistics method and device, computer equipment and computer readable storage medium

本說明書涉及資料處理技術領域，尤其涉及開放場景的即時人群流量統計方法和裝置。 This specification relates to the field of data processing technology, and in particular to methods and devices for real-time crowd flow statistics in open scenes.

人群流量統計在多種商業應用場合發揮著作用，例如，可以統計商場內部按不同時段分佈的人數、人群流動方向等資訊，為商場舉辦活動提供參考；可以獲得商場外人行道上的人群流動量統計資訊，有利於評估商場選址是否適當；等等。 Crowd flow statistics play a role in a variety of commercial applications. For example, it can count the number of people distributed at different times in a shopping mall, the direction of crowd flow, etc., to provide reference for shopping malls to organize activities; obtain statistical information on crowd flow on the sidewalk outside the mall , It is helpful to evaluate whether the location of the shopping mall is appropriate; and so on.

即時人群流量的統計具有更為重要的意義，對監控區域即時流量的統計能夠及時得到現場的人數和人群流量資料，有利於管理單位更高效的組織工作，為科學決策提供資料支持。基於視訊的即時人群流量統計通常要求攝影機具有垂直向下的視角，而在很多應用場合都難以滿足這一條件。 The statistics of real-time crowd flow are of even more important significance. The statistics of real-time flow in the monitoring area can obtain information on the number of people and crowd flow on site in time, which is conducive to more efficient organization of management units and provides data support for scientific decision-making. Video-based real-time crowd flow statistics usually require a camera to have a vertical downward viewing angle, which is difficult to meet in many applications.

有鑑於此，本說明書提供一種開放場景的即時人群流量統計方法，包括： 從拍攝所述開放場景的即時視訊串流中提取當前有效框；所述視訊串流由安置在開放場景上方的攝影機拍攝；檢測當前有效框中的每個行人；採用行人重識別演算法，識別當前有效框中與之前的至少一個有效框中相同的行人；根據行人重識別演算法的結果，計算經過所述開放場景的人群流量數量。 In view of this, this manual provides an open-scene real-time crowd flow statistics method, including: Extract the current valid frame from the real-time video stream for shooting the open scene; the video stream is taken by a camera placed above the open scene; each pedestrian in the current valid frame is detected; a pedestrian re-recognition algorithm is used to identify Pedestrians in the current valid frame that are the same as at least one of the previous valid frames; according to the result of the pedestrian re-identification algorithm, the amount of crowd flow passing through the open scene is calculated.

本說明書還提供了一種開放場景的即時人群流量統計裝置，包括：有效框提取單元，用於從拍攝所述開放場景的即時視訊串流中提取當前有效框；所述視訊串流由安置在開放場景上方的攝影機拍攝；行人檢測單元，用於檢測當前有效框中的每個行人；行人重識別單元，用於採用行人重識別演算法，識別當前有效框中與之前的至少一個有效框中相同的行人；流量計算單元，用於根據行人重識別演算法的結果，計算經過所述開放場景的人群流量數量。 This specification also provides an open scene real-time crowd flow statistics device, including: a valid frame extraction unit for extracting the current valid frame from the real-time video stream for shooting the open scene; the video stream is installed in the open scene The camera above the scene is taken; the pedestrian detection unit is used to detect each pedestrian in the current valid frame; the pedestrian re-recognition unit is used to adopt the pedestrian re-recognition algorithm to recognize that the current valid frame is the same as at least one of the previous valid frames The pedestrian flow calculation unit is used to calculate the amount of crowd flow through the open scene according to the result of the pedestrian re-identification algorithm.

本說明書提供的一種電腦設備，包括：儲存器和處理器；所述儲存器上儲存有可由處理器運行的電腦程式；所述處理器運行所述電腦程式時，執行上述web存取實現方法所述的步驟。 The computer device provided in this specification includes: a memory and a processor; the memory stores a computer program that can be run by the processor; when the processor runs the computer program, it executes the above-mentioned web access implementation method The steps described.

本說明書提供的一種電腦可讀儲存媒體，其上儲存有電腦程式，所述電腦程式被處理器運行時，執行上述應用在CDN節點上的web存取的實現方法所述的步驟。 This specification provides a computer-readable storage medium on which a computer program is stored. When the computer program is run by a processor, it executes the steps described in the method for implementing web access on a CDN node.

由以上技術方案可見，本說明書的實施例中，基於攝影機在開放場景上方拍攝的視訊串流，藉由檢測當前有效框中的行人，以及採用行人重識別演算法識別出當前有效框中的行人與之前有效框中相同的行人，得出經過開放場景的人群流量數量，從而解決了需要垂直向下的攝影機視角進行人群流量統計的局限性，並且在減少運算代價的同時既能提供檢測的時效性，又能提高檢測的準確性。It can be seen from the above technical solutions that in the embodiments of this specification, based on the video stream shot by the camera above the open scene, the pedestrian in the current valid frame is detected and the pedestrian re-recognition algorithm is used to identify the pedestrian in the current valid frame. The same pedestrians as the previous valid frame can obtain the number of crowd flow through the open scene, which solves the limitation of requiring a vertically downward camera perspective for crowd flow statistics, and can provide detection timeliness while reducing the computational cost. It can improve the accuracy of detection.

本說明書的實施例提出一種新的開放場景的即時人群流量統計方法，從安置在開放場景上方的攝影機拍攝的即時視訊串流中提取當前有效框，檢測當前有效框中的行人並識別與之前有效框中的行人是否相同，來統計經過開放場景的人群流量數量。本說明書的實施例無需垂直向下的攝影機視角，能夠適用於絕大多數的應用場合，並且運算負荷低，在具有良好時效性的前提下還具有很高的準確性。 本說明書的實施例可以運行在任何具有計算和儲存能力的設備上，如手機、平板電腦、PC（Personal Computer，個人電腦）、筆記型電腦、伺服器等設備；還可以由運行在兩個或兩個以上設備的邏輯節點來實現本說明書實施例中的各項功能。 本說明書的實施例用來統計在開放場景中經過即時人群流量。攝影機安置在開放場景的上方，以斜向下的角度對開放場景上的人群進行拍攝，產生即時視訊串流。在一些實際應用中，需要統計的是經過開放場景中某個預定的統計區域的人群流量數量。統計區域是開放場景內的一個既定區域，攝影機的拍攝範圍能夠完全覆蓋統計區域。一種開放場景、統計區域與攝影機角度的示例如圖1所示，其中實線框內部為統計區域。 本說明書的實施例中，開放場景的即時人群流量統計方法的流程如圖2所示。 步驟210，從拍攝開放場景的即時視訊串流中提取當前有效框。 安裝在開放場景上方的攝影機將持續輸出以斜向下視角拍攝的視訊串流，視訊串流由連續的一框框圖像構成。可以基於一定的條件，持續不斷的從視訊串流中將符合該條件的各框圖像提取出來作為有效框，藉由連續的辨識有效框中的行人來進行人群流量的統計。將最後一個從視訊串流中提取的有效框作為當前有效框。 提取有效框的條件可以根據實際應用場景對統計時間精度的要求、運行本實施例的設備的處理能力等因素來設置，例如，可以將與上一個有效框間隔N（N為自然數）框的一框圖像作為下一個有效框，也可以從每M（M為大於1的自然數）個連續框中提取一框作為有效框。 步驟220，檢測當前有效框中的每個行人。 在提取當前有效框後，藉由深度學習的目標檢測演算法判斷當前有效框中是否存在人體，如果存在則定位每個行人的位置、以及該行人所佔據的部分圖像區域。 本說明書實施例中對採用的目標檢測演算法不做限定，如可以採用Faster R-CNN（Faster Regions with Convolutional Neural Network features，採用卷積神經網路特徵的快速目標區域識別）、SSD（Single Shot MultiBox Detector，單次目標多框預測）等。 在既要求較低計算量，又要求檢測準確率的應用場景中，可以採用YOLO（You Only Live Once）目標檢測演算法，從當前有效框中提取每個行人的圖像範圍和位置資訊，往往可以達到更好的效果。 步驟230，採用行人重識別演算法，識別當前有效框中與之前的至少一個有效框中相同的行人。 當某個行人從開放場景經過時，會被拍攝到多個有效框中。在進行人群流量統計時，需要在各個有效框中找出相同的行人，避免對同一個人多次計數，才能得到準確的資料。 行人重識別（Person ReID，Person Re-identification）能夠利用電腦視覺技術來判斷圖像中是否存在特定行人，可以用來進行同一個攝影機或跨攝影機的人物追蹤。本說明書的實施例中，採用行人重識別演算法來判斷在當前有效框中檢測到的所有行人中，哪些是已經出現在之前有效框中的行人，哪些是在當前有效框中新出現的行人。 可以藉由查找當前有效框中的某個行人是否出現在當前有效框之前的N個有效框裡，來判斷該行人是否是新出現的行人。由於本說明書實施例中攝影機以傾斜向下的角度拍攝開放場景，行人較為密集時，可能會出現某個行人在某個有效框或某幾個連續的有效框中被他人遮擋而沒有被檢測到的情形，選取較大的N值可以避免在這種情況下錯誤的將該行人重複計數，但較大的N值也會帶來更大的運算負荷。實際應用場景中，可以根據開放場景的行人密集程度、相鄰有效框的間隔時間、運行本實施例的設備的處理能力等因素，來選擇適當的N值。 在一種實現方式中，可以獲取當前有效框中每個行人的外觀特徵和位置特徵，再根據行人的外觀特徵和位置特徵，判定當前有效框中的某個行人是否是之前的N個有效框中已存在的行人，如果不是，產生新的人物標識標記所述行人；如果是，以已有的人物標識標記所述行人。 具體而言，對目標檢測演算法輸出的每個行人的位置、以及該行人所佔據的部分圖像區域，由每個行人的位置產生該行人的位置特徵（如該行人佔據的部分區域在圖片坐標系中的坐標），由該行人所佔據的部分區域的圖像產生該行人的外觀特徵（如衣服顏色、衣服紋理、手提包、背包、帽子等）。採用當前有效框中某個行人的位置特徵和外觀特徵，在之前的N個有效框中查找是否已經存在該行人，如果不存在，為該行人產生新的人物標識並用產生的人物標識標記該行人，人物標識用來唯一的代表一個行人，可以是索引號、字串等，不做限定。如果已經存在該行人，則該行人已經具有自己的人物標識，沿用之前已有的人物標識標記當前有效框中的該行人即可。對當前有效框中檢測出的所有行人逐個執行上述查找過程，直到檢測出的每個行人都標記有人物標識。 可以根據實際應用場景的需要，來選擇識別不同有效框中的行人是否是同一個人時採用的演算法，不做限定。例如，可以採用匈牙利演算法，來根據行人的外觀特徵和位置特徵進行當前有效框與之前有效框中的行人匹配。 步驟240，根據行人重識別演算法的結果，計算經過所述開放場景的人群流量數量。 可以根據實際應用的需要來確定統計開放場景人群流量數量的具體方式，本說明書的實施例不做限定。以下舉例說明（以下的例子中，統計時間段是對人群流量進行累計的時間段）： 第一個例子：可以將在統計時間段內的各個有效框中新出現的行人進行累加，將累加結果作為統計時間段內的人群流量數量。如果當前有效框中的某個行人與之前N個有效框中的行人均不同，則該行人為當前有效框中新出現的行人；統計時間段內所有有效框中新出現的行人數量總和，可以視為統計時間段的行人總數量。 第二個例子：在以人物標識對每個行人進行標記的實現方式中，可以得到統計時間段內在每個有效框中新出現的行人和離開的行人。其中，離開的行人可以是出現在某個有效框之前的相鄰有效框中、並且在該有效框以及之後的N個有效框中未曾出現的行人，以避免該行人被其他行人遮擋而暫時消失導致的統計偏差。統計時間段內的人群流量數量，可以是統計時間段內所有有效框中新出現的行人數量總和，也可以是統計時間段內所有有效框中離開的行人數量總和，還可以是對上述兩個值進行數學運算的結果（如新出現的行人數量總和與離開的行人數量總和的均值）。 在第二個例子中，還可以統計每個行人處於開放場景中的時間長度，如將某個行人離開的有效框、與該行人新出現的有效框之間經過的時間，作為該行人處於開放場景中的時間長度。 在需要統計的是經過開放場景中某個預定的統計區域的人群流量數量的應用場合，對某個有效框，如果某個行人在該有效框中出現在統計區域內、並且在該有效框之前的N個有效框中該行人均未出現在統計區域內，則將該行人作為該有效框中新出現的行人（即進入統計區域的行人）；如果某個行人出現在某個有效框之前的相鄰有效框的統計區域、並且在該有效框及其後的N個有效框中均為出現在統計區域，則將該行人作為該有效框中離開的行人（即離開統計區域的行人）。 在上述應用場合，對進入和離開統計區域可以採用更加嚴格的判斷標準，以得到更為準確的人群流量數量。例如，可以在某個行人在之前的N個有效框中出現在統計區域外而不曾出現在統計區域內、並且在當前有效框中出現在統計區域內時，認為該行人在當前有效框進入統計區域；當某個行人在之前的N個有效框中曾出現在統計區域內而不曾出現在統計區域外、並且在當前有效框中出現在統計區域外時，認為該行人在當前有效框離開統計區域。經過統計區域的人群流量數量可以根據進入統計區域的人數來計算，也可以根據離開統計區域的人數來計算，還可以根據進入統計區域的人數和離開統計區域的人數來計算。 可見，本說明書的實施例中，從安置在開放場景上方的攝影機拍攝的即時視訊串流中提取當前有效框，藉由檢測當前有效框中的行人，以及採用行人重識別演算法識別出當前有效框中的行人與之前有效框中相同的行人，得出經過開放場景的人群流量數量，不僅無需垂直向下的攝影機視角，能夠適用於絕大多數的應用場合，而且運算負荷低，既能提供檢測的時效性，又能提高檢測的準確性。 由於運算負荷較低，本說明書實施例的方法適於運行在嵌入式開發板上，並且對嵌入式開發板的硬體環境沒有特別要求。運行本說明書實施例的嵌入式開發板可以安裝在攝像機附近，將即時統計出的人群流量資料藉由自身的通信模組發送給負責採集流量資料的伺服器，而無需上傳攝像機拍攝的視訊或者圖像，能夠在不侵犯行人的隱私的條件下得到精確的人群流量統計資料。 上述對本說明書特定實施例進行了描述。其它實施例在所附申請專利範圍的範圍內。在一些情況下，在申請專利範圍中記載的動作或步驟可以按照不同於實施例中的順序來執行並且仍然可以實現期望的結果。另外，在圖式中描繪的過程不一定要求示出的特定順序或者連續順序才能實現期望的結果。在某些實施方式中，多任務處理和並行處理也是可以的或者可能是有利的。 在本說明書的一個應用示例中，需要對經過室內開放場景中特定區域的人群流量數量進行統計。RGB（Red Green Blue，紅綠藍）攝影機安裝在牆壁上較高的地點，能夠以斜向下的視角對室內的開放場景進行拍攝，統計區域（即特定區域）位於拍攝範圍的中央部分，與拍攝範圍的邊界相隔一定距離。 人群流量數量統計由運行在NVIDIA（英偉達）嵌入式開發板上的程式進行，嵌入式開發板安裝在攝影機附近，其上包括通信單元，可以與攝影機藉由近距離無線方式連接，從攝影機獲取其拍攝的視訊資料。嵌入式開發板還可以藉由通信單元將統計得出的人群流量資料上傳給預定的伺服器。 嵌入式開發板上運行的人群流量數量統計軟體的結構如圖3所示。 RGB攝影機以25框/秒的速度持續拍攝開放場景的圖像，形成視訊串流。嵌入式開發板從拍攝的視訊串流中每隔固定數量的框提取一框RGB圖像，作為當前有效框。 人群流量數量統計軟體採用YOLO目標檢測演算法，識別出當前有效框中的每個行人，確定每個行人在圖像坐標系中的位置坐標（一種位置特徵），以及每個行人所佔據的部分區域的圖像。 針對當前有效框中的每個行人，行人重識別演算法從該行人佔據區域的圖像中提取該行人的外觀特徵，以該行人的外觀特徵和位置坐標為依據，行人重識別演算法採用匈牙利演算法判斷該行人是否與當前有效框之前的3個有效框中的各個行人是否匹配，鑒別出該行人是否在之前的3個有效框中出現過，如果未曾出現過，則為該行人產生新的人物標識來唯一代表該行人，並用新的人物標識標記該行人；如果曾經出現過，則採用該行人已有的人物標識來標記該行人。 基於行人重識別演算法給當前有效框中每個行人標記的人物標識，以及YOLO目標檢測演算法輸出的每個行人的位置坐標，如果某個行人在之前的3個有效框中出現在統計區域外而不曾出現在統計區域內、並且在當前有效框中出現在統計區域內時，將該行人標記為在當前有效框進入統計區域的行人；如果某個行人在之前的3個有效框中曾出現在統計區域內而不曾出現在統計區域外、並且在當前有效框中出現在統計區域外時，將該行人標記為在當前有效框離開統計區域的行人。 人群流量數量統計軟體以預定的統計時間段為週期，累加在一個週期中的所有有效框中離開統計區域的行人數量作為人群流量數量，並且將單個行人離開統計區域的有效框與進入統計區域的有效框之間的時間間隔作為該行人在統計區域的停留時間長度，累加所有這些行人的停留時間長度。在一個週期結束後，人群流量數量統計軟體向預定的伺服器發送該週期的人群流量數量和這些人群流量的停留總時間長度。 與上述流程實現對應，本說明書的實施例還提供了一種開放場景的即時人群流量統計裝置。該裝置可以藉由軟體實現，也可以藉由硬體或者軟硬體結合的方式實現。以軟體實現為例，作為邏輯意義上的裝置，是藉由所在設備的CPU（Central Process Unit，中央處理器）將對應的電腦程式指令讀取到內部儲存器中運行形成的。從硬體層面而言，除了圖4所示的CPU、內部儲存器以及儲存器之外，開放場景的即時人群流量統計裝置所在的設備通常還包括用於進行無線信號收發的晶片等其他硬體，和/或用於實現網路通信功能的板卡等其他硬體。 圖5所示為本說明書實施例提供的一種開放場景的即時人群流量統計裝置，包括有效框提取單元、行人檢測單元、行人重識別單元和流量計算單元，其中：有效框提取單元用於從拍攝所述開放場景的即時視訊串流中提取當前有效框；所述視訊串流由安置在開放場景上方的攝影機拍攝；行人檢測單元用於檢測當前有效框中的每個行人；行人重識別單元用於採用行人重識別演算法，識別當前有效框中與之前的至少一個有效框中相同的行人；流量計算單元用於根據行人重識別演算法的結果，計算經過所述開放場景的人群流量數量。 可選的，所述經過開放場景的人群流量數量包括：經過所述開放場景中預定的統計區域的人群流量數量；所述流量計算單元具體用於以下至少一項：當某個行人在之前有效框中出現在統計區域外而不曾出現在統計區域內、並且在當前有效框中出現在統計區域內時，認為該行人進入統計區域；根據進入統計區域的人數計算經過所述統計區域的人群流量數量；當某個行人在之前有效框中曾出現在統計區域內而不曾出現在統計區域外、並且在當前有效框中出現在統計區域外時，認為該行人離開統計區域；根據離開統計區域的人數計算經過所述統計區域的人群流量數量。 一種實現方式中，所述行人重識別單元具體用於：獲取當前有效框中每個行人的外觀特徵、或外觀特徵和位置特徵；根據行人的外觀特徵、或外觀特徵和位置特徵，判定當前有效框中的每個行人是否是之前的至少一個有效框中已存在的行人，如果不是則產生新的人物標識標記所述行人，如果是則以已有的人物標識標記所述行人。 上述實現方式中，所述行人重識別單元根據行人的外觀特徵、或外觀特徵和位置特徵，判定當前有效框中的某個行人是否是之前有效框中已存在的行人，包括：採用匈牙利演算法，根據行人的外觀特徵、或外觀特徵和位置特徵進行當前有效框與之前有效框中的行人匹配。 可選的，所述行人檢測單元具體用於：採用YOLO目標檢測方法，從當前有效框中提取每個行人的圖像範圍和位置資訊。 可選的，所述攝影機為紅綠藍RGB攝影機。 可選的，所述裝置運行在嵌入式開發板上。 本說明書的實施例提供了一種電腦設備，該電腦設備包括儲存器和處理器。其中，儲存器上儲存有能夠由處理器運行的電腦程式；處理器在運行儲存的電腦程式時，執行本說明書實施例中開放場景的即時人群流量統計方法的各個步驟。對開放場景的即時人群流量統計方法的各個步驟的詳細描述請參見之前的內容，不再重複。 本說明書的實施例提供了一種電腦可讀儲存媒體，該儲存媒體上儲存有電腦程式，這些電腦程式在被處理器運行時，執行本說明書實施例中開放場景的即時人群流量統計方法的各個步驟。對開放場景的即時人群流量統計方法的各個步驟的詳細描述請參見之前的內容，不再重複。 以上所述僅為本說明書的較佳實施例而已，並不用以限制本申請，凡在本申請的精神和原則之內，所做的任何修改、等同替換、改進等，均應包含在本申請保護的範圍之內。 在一個典型的配置中，計算設備包括一個或多個處理器（CPU）、輸入/輸出介面、網路介面和內部儲存器。 內部儲存器可能包括電腦可讀媒體中的非永久性儲存器，隨機存取記憶體（RAM）和/或非易失性內部儲存器等形式，如唯讀記憶體（ROM）或快閃記憶體（flash RAM）。內部儲存器是電腦可讀媒體的示例。 電腦可讀媒體包括永久性和非永久性、可移動和非可移動媒體可以由任何方法或技術來實現資訊儲存。資訊可以是電腦可讀指令、資料結構、程式的模組或其他資料。電腦的儲存媒體的例子包括，但不限於相變記憶體（PRAM）、靜態隨機存取記憶體（SRAM）、動態隨機存取記憶體（DRAM）、其他類型的隨機存取記憶體（RAM）、唯讀記憶體（ROM）、電可抹除可程式化唯讀記憶體（EEPROM）、快閃記憶體或其他內部儲存器技術、唯讀光碟（CD-ROM）、數位化多功能光碟（DVD）或其他光學儲存、磁盒式磁帶，磁帶磁磁碟儲存或其他磁性儲存設備或任何其他非傳輸媒體，可用於儲存可以被計算設備存取的資訊。按照本文中的界定，電腦可讀媒體不包括暫存電腦可讀媒體（transitory media），如調變的資料信號和載波。 還需要說明的是，術語“包括”、“包含”或者其任何其他變體意在涵蓋非排他性的包含，從而使得包括一系列要素的過程、方法、商品或者設備不僅包括那些要素，而且還包括沒有明確列出的其他要素，或者是還包括為這種過程、方法、商品或者設備所固有的要素。在沒有更多限制的情況下，由語句“包括一個……”限定的要素，並不排除在包括所述要素的過程、方法、商品或者設備中還存在另外的相同要素。 本領域技術人員應明白，本說明書的實施例可提供為方法、系統或電腦程式產品。因此，本說明書的實施例可採用完全硬體實施例、完全軟體實施例或結合軟體和硬體方面的實施例的形式。而且，本說明書的實施例可採用在一個或多個其中包含有電腦可用程式碼的電腦可用儲存媒體（包括但不限於磁碟儲存器、CD-ROM、光學儲存器等）上實施的電腦程式產品的形式。The embodiment of this specification proposes a new open scene real-time crowd flow statistics method, which extracts the current valid frame from the real-time video stream shot by a camera placed above the open scene, detects the pedestrians in the current valid frame and identifies the previous valid Whether the pedestrians in the boxes are the same, to count the number of people passing through the open scene. The embodiments of this specification do not require a vertically downward camera angle of view, can be applied to most applications, have low computational load, and have high accuracy on the premise of good timeliness. The embodiments of this specification can run on any device with computing and storage capabilities, such as mobile phones, tablets, PCs (Personal Computers, personal computers), notebook computers, servers and other devices; it can also be run on two or The logical nodes of more than two devices implement various functions in the embodiments of this specification. The embodiments of this specification are used to count the instant crowd flow in an open scene. The camera is placed above the open scene and shoots the crowd in the open scene from an oblique downward angle to produce a real-time video stream. In some practical applications, what needs to be counted is the amount of crowd traffic passing through a predetermined statistical area in an open scene. The statistical area is a predetermined area in the open scene, and the shooting range of the camera can completely cover the statistical area. An example of an open scene, statistical area, and camera angle is shown in Figure 1, where the solid-line frame is the statistical area. In the embodiment of this specification, the flow of the method for real-time crowd flow statistics in an open scene is shown in FIG. 2. Step 210: Extract the currently valid frame from the real-time video stream of the shooting open scene. The camera installed above the open scene will continuously output a video stream shot from a diagonally downward perspective. The video stream consists of a continuous frame of images. Based on certain conditions, each frame image that meets the conditions can be continuously extracted from the video stream as a valid frame, and crowd flow statistics can be performed by continuously identifying pedestrians in the valid frame. Use the last valid frame extracted from the video stream as the current valid frame. The conditions for extracting a valid frame can be set according to the actual application scenario’s requirements for statistical time accuracy, the processing capability of the device running this embodiment, and other factors. For example, the frame interval from the last valid frame by N (N is a natural number) can be set. A frame of image is used as the next valid frame, and a frame can also be extracted from every M (M is a natural number greater than 1) consecutive frames as a valid frame. Step 220: Detect each pedestrian in the currently valid frame. After extracting the current valid frame, a deep learning target detection algorithm is used to determine whether there is a human body in the current valid frame, and if so, the location of each pedestrian and the part of the image area occupied by the pedestrian are located. In the embodiments of this specification, the target detection algorithm used is not limited. For example, Faster R-CNN (Faster Regions with Convolutional Neural Network features, fast target region recognition using convolutional neural network features), SSD (Single Shot MultiBox Detector, single target multi-box prediction) and so on. In application scenarios that require both low calculation and detection accuracy, the YOLO (You Only Live Once) target detection algorithm can be used to extract the image range and location information of each pedestrian from the current valid frame. Can achieve better results. In step 230, a pedestrian re-recognition algorithm is used to identify pedestrians in the current valid frame that are the same as at least one previous valid frame. When a pedestrian passes through an open scene, it will be shot into multiple effective frames. When performing crowd flow statistics, it is necessary to find the same pedestrians in each valid box and avoid counting the same person multiple times to obtain accurate information. Pedestrian ReID (Person Re-identification) can use computer vision technology to determine whether there is a specific pedestrian in the image, which can be used to track people on the same camera or across cameras. In the embodiment of this specification, a pedestrian re-recognition algorithm is used to determine among all the pedestrians detected in the current valid frame, which are the pedestrians that have appeared in the previous valid frame, and which are the newly appeared pedestrians in the current valid frame . It is possible to determine whether the pedestrian is a new pedestrian by searching whether a pedestrian in the current valid frame appears in the N valid frames before the current valid frame. Since the camera in the embodiment of this specification shoots the open scene at an oblique downward angle, when the pedestrians are dense, it may happen that a pedestrian is blocked by others in a certain effective frame or a few consecutive effective frames without being detected. In this case, choosing a larger value of N can avoid falsely counting the pedestrian in this case, but a larger value of N will also bring greater computational load. In an actual application scenario, an appropriate value of N can be selected according to factors such as the pedestrian density of the open scene, the interval time between adjacent effective frames, and the processing capability of the device running this embodiment. In one implementation, the appearance feature and location feature of each pedestrian in the current valid frame can be obtained, and then according to the appearance feature and location feature of the pedestrian, it is determined whether a pedestrian in the current valid frame is the previous N valid frames. If the existing pedestrian is not, a new person identification is generated to mark the pedestrian; if it is, the pedestrian is marked with the existing person identification. Specifically, for the position of each pedestrian output by the target detection algorithm and the partial image area occupied by the pedestrian, the position feature of the pedestrian is generated from the position of each pedestrian (for example, the partial area occupied by the pedestrian is in the picture Coordinates in the coordinate system), the appearance characteristics of the pedestrian (such as clothes color, clothes texture, handbag, backpack, hat, etc.) are generated from the image of the part of the area occupied by the pedestrian. Use the location feature and appearance feature of a pedestrian in the current valid box to find out whether the pedestrian already exists in the previous N valid boxes. If not, generate a new person identification for the pedestrian and mark the pedestrian with the generated person identification , The person ID is used to uniquely represent a pedestrian, it can be an index number, a character string, etc., without limitation. If the pedestrian already exists, the pedestrian already has its own personal identification, and the existing personal identification can be used to mark the pedestrian in the current valid frame. The above search process is performed one by one for all pedestrians detected in the current valid frame, until each detected pedestrian is marked with a person ID. The algorithm used to identify whether the pedestrians in different effective frames are the same person can be selected according to the needs of the actual application scenario, and there is no limitation. For example, the Hungarian algorithm can be used to match the pedestrians in the current valid frame with the previous valid frame according to the appearance and location characteristics of the pedestrian. Step 240: Calculate the amount of crowd flow passing through the open scene according to the result of the pedestrian re-identification algorithm. The specific method for counting the amount of crowd traffic in the open scene can be determined according to the needs of actual applications, which is not limited in the embodiment of this specification. The following is an example (in the following example, the statistical time period is the time period during which the crowd flow is accumulated): The first example: the pedestrians newly appearing in each valid frame in the statistical time period can be accumulated, and the accumulated result can be used as the number of crowd flow in the statistical time period. If a pedestrian in the current valid box is different from the pedestrians in the previous N valid boxes, then the pedestrian is a new pedestrian in the current valid box; the total number of new pedestrians in all valid boxes in the statistical time period, you can The total number of pedestrians in the statistical time period. The second example: in the implementation of marking each pedestrian with a person identifier, the newly appeared pedestrians and the pedestrians leaving in each valid frame within the statistical time period can be obtained. Among them, the leaving pedestrian can be a pedestrian who appears in the adjacent effective frame before a certain effective frame and has not appeared in the effective frame and the next N effective frames, so as to avoid the pedestrian being blocked by other pedestrians and disappearing temporarily The resulting statistical bias. The number of crowd flows in the statistical time period can be the sum of the number of pedestrians newly appearing in all valid boxes during the statistical time period, or the sum of the number of pedestrians leaving all valid boxes during the statistical time period, or the sum of the above two Value is the result of a mathematical operation (such as the average of the sum of the number of new pedestrians and the sum of the number of leaving pedestrians). In the second example, the length of time each pedestrian is in the open scene can also be counted. For example, the time elapsed between the effective frame that a certain pedestrian leaves and the effective frame that the pedestrian newly appears is regarded as the pedestrian in the open scene. The length of time in the scene. In applications where statistics are required to pass through a predetermined statistical area in an open scene, for an effective frame, if a pedestrian appears in the effective frame in the statistical area and before the effective frame If the pedestrian does not appear in the statistical area in the N valid boxes, the pedestrian will be regarded as the new pedestrian in the effective box (that is, the pedestrian entering the statistical area); if a pedestrian appears before a valid box If the statistical area adjacent to the effective frame and the N effective frames after the effective frame appear in the statistical area, the pedestrian is regarded as the pedestrian leaving the effective frame (that is, the pedestrian leaving the statistical area). In the above-mentioned applications, stricter criteria can be adopted for entering and leaving the statistical area to obtain a more accurate number of crowds. For example, when a pedestrian appears outside the statistical area in the previous N valid frames but never appeared in the statistical area, and appears in the statistical area in the current valid frame, it can be considered that the pedestrian enters the statistical area in the current valid frame. Area; when a pedestrian has appeared in the statistical area but not outside the statistical area in the previous N valid frames, and appears outside the statistical area in the current valid frame, it is considered that the pedestrian has left the statistical area in the current valid frame area. The number of people passing through the statistical area can be calculated based on the number of people entering the statistical area, can also be calculated based on the number of people leaving the statistical area, and can also be calculated based on the number of people entering the statistical area and the number of people leaving the statistical area. It can be seen that in the embodiment of this specification, the current valid frame is extracted from the real-time video stream taken by the camera placed above the open scene, the current valid frame is identified by detecting the pedestrian in the current valid frame, and the pedestrian re-recognition algorithm is used to identify the current valid frame. Pedestrians in the frame are the same as the pedestrians in the previous effective frame, and the amount of crowd traffic passing through the open scene is obtained. Not only does it require a vertical downward camera perspective, it can be applied to most applications, but also has a low computing load and can provide The timeliness of detection can also improve the accuracy of detection. Due to the low computational load, the method in the embodiment of this specification is suitable for running on an embedded development board, and there is no special requirement for the hardware environment of the embedded development board. The embedded development board running the embodiments of this specification can be installed near the camera, and the real-time statistics of crowd flow data can be sent to the server responsible for collecting the flow data through its own communication module, without uploading the video or pictures taken by the camera. Like, accurate crowd flow statistics can be obtained without infringing on the privacy of pedestrians. The foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the attached patent application. In some cases, the actions or steps described in the scope of the patent application may be performed in a different order than in the embodiments and still achieve desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown in order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. In an application example of this specification, it is necessary to count the number of crowd traffic passing through a specific area in an indoor open scene. The RGB (Red Green Blue, red, green, and blue) camera is installed at a high place on the wall and can shoot indoor open scenes from a diagonally downward perspective. The statistical area (ie, a specific area) is located in the center of the shooting range. The boundaries of the shooting range are separated by a certain distance. Crowd flow statistics are carried out by programs running on the NVIDIA embedded development board. The embedded development board is installed near the camera. It includes a communication unit, which can be connected to the camera in a short-distance wireless manner, and it can be obtained from the camera. Video data taken. The embedded development board can also upload the statistics of crowd flow data to a predetermined server through the communication unit. Figure 3 shows the structure of the crowd flow statistics software running on the embedded development board. The RGB camera continuously shoots images of open scenes at a rate of 25 frames per second to form a video stream. The embedded development board extracts a frame of RGB image every fixed number of frames from the captured video stream as the current valid frame. Crowd flow statistics software uses the YOLO target detection algorithm to identify each pedestrian in the current valid frame, determine the position coordinates of each pedestrian in the image coordinate system (a position feature), and the part occupied by each pedestrian The image of the area. For each pedestrian in the current valid frame, the pedestrian re-recognition algorithm extracts the appearance characteristics of the pedestrian from the image of the area occupied by the pedestrian. Based on the appearance characteristics and position coordinates of the pedestrian, the pedestrian re-recognition algorithm adopts Hungary The algorithm judges whether the pedestrian matches each of the pedestrians in the three valid boxes before the current valid box, and identifies whether the pedestrian has appeared in the previous three valid boxes, and if it has not appeared before, a new one is generated for the pedestrian To uniquely represent the pedestrian, and mark the pedestrian with a new character identifier; if it has appeared before, use the existing character identifier of the pedestrian to mark the pedestrian. Based on the pedestrian re-recognition algorithm to mark the person identification of each pedestrian in the current effective frame, and the position coordinates of each pedestrian output by the YOLO target detection algorithm, if a pedestrian appears in the statistics area in the previous 3 effective frames When a pedestrian has never appeared in the statistical area and appears in the statistical area in the current valid frame, mark the pedestrian as a pedestrian who entered the statistical area in the current valid frame; if a pedestrian has been in the previous 3 valid frames When it appears in the statistical area but never outside the statistical area, and appears outside the statistical area in the current valid frame, mark the pedestrian as a pedestrian who leaves the statistical area in the current valid frame. Crowd flow statistics software uses a predetermined statistical time period as the cycle, accumulates the number of pedestrians leaving the statistical area in all valid frames in a cycle as the number of crowd flow, and compares the effective frame of a single pedestrian leaving the statistical area with the number of pedestrians entering the statistical area. The time interval between valid frames is used as the length of stay time of the pedestrian in the statistical area, and the length of stay time of all these pedestrians is accumulated. After the end of a period, the crowd flow statistics software sends the number of crowd flows in the period and the total length of stay of these crowd flows to a predetermined server. Corresponding to the foregoing process implementation, the embodiments of this specification also provide an open scene real-time crowd flow statistics device. The device can be implemented by software, or by hardware or a combination of software and hardware. Take software implementation as an example. As a logical device, it is formed by the CPU (Central Process Unit, central processing unit) of the device that reads the corresponding computer program instructions into the internal storage and runs. From the hardware level, in addition to the CPU, internal storage, and storage shown in Figure 4, the equipment where the real-time crowd flow statistics device of the open scene is located usually also includes other hardware such as chips for wireless signal transmission and reception. , And/or other hardware such as boards used to implement network communication functions. Figure 5 shows an open-scene real-time crowd flow statistics device provided by an embodiment of this specification, including an effective frame extraction unit, a pedestrian detection unit, a pedestrian re-identification unit, and a flow calculation unit. The current valid frame is extracted from the real-time video stream of the open scene; the video stream is taken by a camera placed above the open scene; the pedestrian detection unit is used to detect each pedestrian in the current valid frame; the pedestrian re-recognition unit is used A pedestrian re-recognition algorithm is used to identify pedestrians in the current valid frame that are the same as at least one previous valid frame; the flow calculation unit is used to calculate the amount of crowd flow through the open scene according to the result of the pedestrian re-recognition algorithm. Optionally, the amount of crowd flow passing through the open scene includes: the amount of crowd flow passing through a predetermined statistical area in the open scene; the flow calculation unit is specifically used for at least one of the following: when a certain pedestrian is valid before When the box appears outside the statistical area but never appeared in the statistical area, and appears in the statistical area in the current valid box, the pedestrian is considered to have entered the statistical area; the flow of people passing through the statistical area is calculated based on the number of people entering the statistical area Quantity; when a pedestrian has appeared in the statistical area but not outside the statistical area in the previous valid frame, and appears outside the statistical area in the current valid frame, it is considered that the pedestrian has left the statistical area; The number of people calculates the number of people who pass through the statistical area. In an implementation manner, the pedestrian re-identification unit is specifically configured to: obtain the appearance feature, or appearance feature and location feature of each pedestrian in the current valid frame; determine the current valid according to the appearance feature, or appearance feature and location feature of the pedestrian Whether each pedestrian in the frame is a pedestrian that already exists in at least one previous valid frame, if not, a new person identification is generated to mark the pedestrian, and if so, the pedestrian is marked with an existing person identification. In the foregoing implementation manner, the pedestrian re-identification unit determines whether a pedestrian in the current valid frame is a pedestrian that already exists in the previous valid frame according to the appearance feature, or appearance feature and location feature of the pedestrian, including: adopting the Hungarian algorithm , According to the appearance characteristics of pedestrians, or appearance characteristics and location characteristics, the current valid frame is matched with the pedestrians in the previous valid frame. Optionally, the pedestrian detection unit is specifically configured to use the YOLO target detection method to extract the image range and position information of each pedestrian from the current valid frame. Optionally, the camera is a red, green, and blue RGB camera. Optionally, the device runs on an embedded development board. The embodiments of this specification provide a computer device including a storage and a processor. Wherein, a computer program that can be run by the processor is stored in the storage; when the processor runs the stored computer program, each step of the open scene real-time crowd flow statistics method in the embodiment of this specification is executed. For a detailed description of each step of the instant crowd flow statistics method for open scenarios, please refer to the previous content and will not be repeated. The embodiment of this specification provides a computer-readable storage medium on which computer programs are stored. These computer programs, when run by a processor, execute each step of the open scene real-time crowd flow statistics method in the embodiment of this specification . For a detailed description of each step of the instant crowd flow statistics method for open scenarios, please refer to the previous content and will not be repeated. The above are only the preferred embodiments of this specification and are not intended to limit this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in this application Within the scope of protection. In a typical configuration, the computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and internal storage. Internal storage may include non-permanent storage in computer-readable media, random access memory (RAM) and/or non-volatile internal storage, such as read-only memory (ROM) or flash memory Body (flash RAM). Internal storage is an example of computer-readable media. Computer-readable media includes permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), and other types of random access memory (RAM) , Read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other internal storage technology, CD-ROM, digital multi-function disc ( DVD) or other optical storage, magnetic cassettes, magnetic tape storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by computing devices. According to the definition in this article, computer-readable media does not include transitory media, such as modulated data signals and carrier waves. It should also be noted that the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or equipment including a series of elements not only includes those elements, but also includes Other elements that are not explicitly listed, or also include elements inherent to such processes, methods, commodities, or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, commodity, or equipment that includes the element. Those skilled in the art should understand that the embodiments of this specification can be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may adopt the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of this specification can adopt computer programs implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The form of the product.

210~240:步驟210~240: Step

圖1是本說明書實施例中一種開放場景、攝影機角度與統計區域的示例圖； 圖2是本說明書實施例中一種開放場景的即時人群流量統計方法的流程圖； 圖3是本說明書應用示例中嵌入式開發板上運行的人群流量數量統計軟體的結構示意圖； 圖4是運行本說明書實施例的設備的一種硬體結構圖； 圖5是本說明書實施例中一種開放場景的即時人群流量統計裝置的邏輯結構圖。FIG. 1 is an example diagram of an open scene, camera angle, and statistical area in an embodiment of this specification; 2 is a flowchart of an open scene real-time crowd flow statistics method in an embodiment of this specification; Figure 3 is a schematic diagram of the structure of the crowd flow statistics software running on the embedded development board in the application example of this manual; Figure 4 is a hardware structure diagram of the device running the embodiment of this specification; Fig. 5 is a logical structure diagram of an open scene real-time crowd flow statistics device in an embodiment of this specification.

Claims (14)

一種開放場景的即時人群流量統計方法，包括：從拍攝該開放場景的即時視訊串流中提取當前有效框；該視訊串流由安置在開放場景上方的攝影機拍攝；檢測當前有效框中的每個行人；採用行人重識別演算法，識別當前有效框中與之前的至少一個有效框中相同的行人；根據行人重識別演算法的結果，計算經過該開放場景的人群流量數量，其中，該採用行人重識別演算法，識別當前有效框中與之前的至少一個有效框中相同的行人，包括：獲取當前有效框中每個行人的外觀特徵、或外觀特徵和位置特徵；根據行人的外觀特徵、或外觀特徵和位置特徵，判定當前有效框中的每個行人是否是之前的至少一個有效框中已存在的行人，如果不是則產生新的人物標識標記該行人，如果是則以已有的人物標識標記該行人。 An open scene real-time crowd flow statistics method includes: extracting the current valid frame from the real-time video stream of the open scene; the video stream is taken by a camera placed above the open scene; detecting each of the current valid frames Pedestrians; Pedestrian re-recognition algorithm is used to identify the pedestrians in the current valid frame and at least one of the previous valid frames; according to the results of the pedestrian re-recognition algorithm, the amount of crowd flow passing through the open scene is calculated, among which, pedestrians should be used Re-recognition algorithm to identify pedestrians in the current valid frame that are the same as at least one of the previous valid frames, including: obtaining the appearance feature, or appearance feature and location feature of each pedestrian in the current valid frame; according to the appearance feature of the pedestrian, or Appearance feature and location feature, determine whether each pedestrian in the current valid frame is an existing pedestrian in at least one of the previous valid frames, if not, generate a new person identification to mark the pedestrian, if so, use the existing person identification Mark the pedestrian. 根據請求項1所述的方法，該根據行人的外觀特徵、或外觀特徵和位置特徵，判定當前有效框中的某個行人是否是之前有效框中已存在的行人，包括：採用匈牙利演算法，根據行人的外觀特徵、或外觀特徵和位置特徵進行當前有效框與之前有效框中的行人匹配。 According to the method described in claim 1, determining whether a pedestrian in the current valid frame is a pedestrian in the previous valid frame based on the appearance feature, or appearance feature and location feature of the pedestrian, includes: adopting the Hungarian algorithm, The current valid frame is matched with the pedestrians in the previous valid frame according to the appearance feature, or appearance feature and location feature of the pedestrian. 根據請求項1所述的方法，該檢測當前有效框中的每個行人，包括：採用YOLO目標檢測方法，從當前有效框中提取每個行人的圖像範圍和位置資訊。 According to the method described in claim 1, the detecting each pedestrian in the current valid frame includes: using the YOLO target detection method to extract the image range and location information of each pedestrian from the current valid frame. 根據請求項1所述的方法，該攝影機為紅綠藍RGB攝影機。 According to the method described in claim 1, the camera is a red, green, and blue RGB camera. 根據請求項1所述的方法，該方法運行在嵌入式開發板上。 According to the method described in claim 1, the method runs on an embedded development board. 一種開放場景的即時人群流量統計方法，包括：從拍攝該開放場景的即時視訊串流中提取當前有效框；該視訊串流由安置在開放場景上方的攝影機拍攝；檢測當前有效框中的每個行人；採用行人重識別演算法，識別當前有效框中與之前的至少一個有效框中相同的行人；根據行人重識別演算法的結果，計算經過該開放場景的人群流量數量，其中，該經過開放場景的人群流量數量包括：經過該開放場景中預定的統計區域的人群流量數量；該根據行人重識別演算法的結果，計算經過該開放場景的人群流量數量，包括以下至少一項：當某個行人在之前有效框中出現在統計區域外而不曾出現在統計區域內、並且在當前有效框中出現在統計區域 內時，認為該行人進入統計區域；根據進入統計區域的人數計算經過該統計區域的人群流量數量；當某個行人在之前有效框中曾出現在統計區域內而不曾出現在統計區域外、並且在當前有效框中出現在統計區域外時，認為該行人離開統計區域；根據離開統計區域的人數計算經過該統計區域的人群流量數量。 An open scene real-time crowd flow statistics method includes: extracting the current valid frame from the real-time video stream of the open scene; the video stream is taken by a camera placed above the open scene; detecting each of the current valid frames Pedestrians; the pedestrian re-recognition algorithm is used to identify the pedestrians in the current valid frame and at least one of the previous valid frames; according to the result of the pedestrian re-recognition algorithm, the amount of crowd flow passing through the open scene is calculated. The amount of crowd flow in the scene includes: the amount of crowd flow passing through the predetermined statistical area in the open scene; the calculation of the amount of crowd flow passing through the open scene according to the result of the pedestrian re-identification algorithm includes at least one of the following: Pedestrians appeared outside the statistical area in the previous valid box but never appeared in the statistical area, and appeared in the statistical area in the current valid box When inside, the pedestrian is considered to have entered the statistical area; the number of people passing through the statistical area is calculated based on the number of people entering the statistical area; when a pedestrian has appeared in the statistical area but never appeared outside the statistical area in the previous valid box, and When the current valid box appears outside the statistical area, the pedestrian is considered to have left the statistical area; the number of people passing through the statistical area is calculated based on the number of people leaving the statistical area. 一種開放場景的即時人群流量統計裝置，包括：有效框提取單元，用於從拍攝該開放場景的即時視訊串流中提取當前有效框；該視訊串流由安置在開放場景上方的攝影機拍攝；行人檢測單元，用於檢測當前有效框中的每個行人；行人重識別單元，用於採用行人重識別演算法，識別當前有效框中與之前的至少一個有效框中相同的行人；流量計算單元，用於根據行人重識別演算法的結果，計算經過該開放場景的人群流量數量，其中，該行人重識別單元具體用於：獲取當前有效框中每個行人的外觀特徵、或外觀特徵和位置特徵；根據行人的外觀特徵、或外觀特徵和位置特徵，判定當前有效框中的每個行人是否是之前的至少一個有效框中已存在的行人，如果不是則產生新的人物標識標記該行人，如果是則以已有的人物標識標記該行人。 An open scene real-time crowd flow statistics device, including: an effective frame extraction unit for extracting the current effective frame from the real-time video stream for shooting the open scene; the video stream is taken by a camera placed above the open scene; pedestrians The detection unit is used to detect each pedestrian in the current valid frame; the pedestrian re-identification unit is used to adopt a pedestrian re-recognition algorithm to identify pedestrians in the current valid frame and at least one of the previous valid frames; the flow calculation unit, It is used to calculate the amount of crowd flow passing through the open scene according to the result of the pedestrian re-recognition algorithm, where the pedestrian re-recognition unit is specifically used to: obtain the appearance feature, or appearance feature and location feature of each pedestrian in the current valid frame ; According to the appearance characteristics of the pedestrian, or appearance characteristics and location characteristics, determine whether each pedestrian in the current valid frame is a pedestrian that already exists in at least one previous valid frame, if not, a new person ID will be generated to mark the pedestrian, if If yes, mark the pedestrian with the existing person ID. 根據請求項7所述的裝置，該行人重識別單元根據行 人的外觀特徵、或外觀特徵和位置特徵，判定當前有效框中的某個行人是否是之前有效框中已存在的行人，包括：採用匈牙利演算法，根據行人的外觀特徵、或外觀特徵和位置特徵進行當前有效框與之前有效框中的行人匹配。 According to the device according to claim 7, the pedestrian re-identification unit People’s appearance characteristics, or appearance characteristics and location characteristics, to determine whether a pedestrian in the current valid frame is a pedestrian that already exists in the previous valid frame, including: using the Hungarian algorithm, according to the appearance characteristics, or appearance characteristics and location of the pedestrian The feature matches the pedestrians in the current valid frame with the previous valid frame. 根據請求項7所述的裝置，該行人檢測單元具體用於：採用YOLO目標檢測方法，從當前有效框中提取每個行人的圖像範圍和位置資訊。 According to the device described in claim 7, the pedestrian detection unit is specifically configured to use the YOLO target detection method to extract the image range and location information of each pedestrian from the current valid frame. 根據請求項7所述的裝置，該攝影機為紅綠藍RGB攝影機。 According to the device according to claim 7, the camera is a red, green, and blue RGB camera. 根據請求項7所述的裝置，該裝置運行在嵌入式開發板上。 According to the device described in claim 7, the device runs on an embedded development board. 一種開放場景的即時人群流量統計裝置，包括：有效框提取單元，用於從拍攝該開放場景的即時視訊串流中提取當前有效框；該視訊串流由安置在開放場景上方的攝影機拍攝；行人檢測單元，用於檢測當前有效框中的每個行人；行人重識別單元，用於採用行人重識別演算法，識別當前有效框中與之前的至少一個有效框中相同的行人；流量計算單元，用於根據行人重識別演算法的結果，計算經過該開放場景的人群流量數量， 其中，該經過開放場景的人群流量數量包括：經過該開放場景中預定的統計區域的人群流量數量；該流量計算單元具體用於以下至少一項：當某個行人在之前有效框中出現在統計區域外而不曾出現在統計區域內、並且在當前有效框中出現在統計區域內時，認為該行人進入統計區域；根據進入統計區域的人數計算經過該統計區域的人群流量數量；當某個行人在之前有效框中曾出現在統計區域內而不曾出現在統計區域外、並且在當前有效框中出現在統計區域外時，認為該行人離開統計區域；根據離開統計區域的人數計算經過該統計區域的人群流量數量。 An open scene real-time crowd flow statistics device, including: an effective frame extraction unit for extracting the current effective frame from the real-time video stream for shooting the open scene; the video stream is taken by a camera placed above the open scene; pedestrians The detection unit is used to detect each pedestrian in the current valid frame; the pedestrian re-identification unit is used to adopt a pedestrian re-recognition algorithm to identify pedestrians in the current valid frame and at least one of the previous valid frames; the flow calculation unit, It is used to calculate the amount of crowd flow through the open scene based on the result of the pedestrian re-identification algorithm, Wherein, the amount of crowd flow passing through the open scene includes: the amount of crowd flow passing through a predetermined statistical area in the open scene; the flow calculation unit is specifically used for at least one of the following: when a pedestrian appears in the statistics in the previous valid box When outside the area and does not appear in the statistical area, and appears in the statistical area in the current valid box, the pedestrian is considered to have entered the statistical area; the number of people passing through the statistical area is calculated based on the number of people entering the statistical area; when a pedestrian When the previous valid box has appeared in the statistical area but never outside the statistical area, and appears outside the statistical area in the current valid box, the pedestrian is considered to have left the statistical area; the number of people who have left the statistical area is calculated based on the number of people who have passed through the statistical area The amount of crowd traffic. 一種電腦設備，包括：儲存器和處理器；該儲存器上儲存有可由處理器運行的電腦程式；該處理器運行該電腦程式時，執行如請求項1到6中任意一項所述的步驟。 A computer device comprising: a memory and a processor; the memory is stored with a computer program that can be run by the processor; when the processor runs the computer program, the processor executes the steps described in any one of claim items 1 to 6 . 一種電腦可讀儲存媒體，其上儲存有電腦程式，該電腦程式被處理器運行時，執行如請求項1到6中任意一項所述的步驟。 A computer-readable storage medium has a computer program stored thereon, and when the computer program is run by a processor, it executes the steps described in any one of request items 1 to 6.
TW108128880A 2018-11-09 2019-08-14 Open scene real-time crowd flow statistics method and device, computer equipment and computer readable storage medium TWI729454B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811331786.XA CN109902551A (en) 2018-11-09 2018-11-09 The real-time stream of people's statistical method and device of open scene
CN201811331786.X 2018-11-09

Publications (2)

Publication Number Publication Date
TW202036375A TW202036375A (en) 2020-10-01
TWI729454B true TWI729454B (en) 2021-06-01

Family

ID=66943300

Family Applications (1)

Application Number Title Priority Date Filing Date
TW108128880A TWI729454B (en) 2018-11-09 2019-08-14 Open scene real-time crowd flow statistics method and device, computer equipment and computer readable storage medium

Country Status (3)

Country Link
CN (1) CN109902551A (en)
TW (1) TWI729454B (en)
WO (1) WO2020093829A1 (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109902551A (en) * 2018-11-09 2019-06-18 阿里巴巴集团控股有限公司 The real-time stream of people's statistical method and device of open scene
CN112149457A (en) * 2019-06-27 2020-12-29 西安光启未来技术研究院 People flow statistical method, device, server and computer readable storage medium
CN110659588A (en) * 2019-09-02 2020-01-07 平安科技（深圳）有限公司 Passenger flow volume statistical method and device and computer readable storage medium
CN110705494A (en) * 2019-10-10 2020-01-17 北京东软望海科技有限公司 People flow monitoring method and device, electronic equipment and computer readable storage medium
CN112005247A (en) * 2020-06-12 2020-11-27 深圳盈天下视觉科技有限公司 People flow data monitoring system and people flow data display method and device thereof
CN113297888A (en) * 2020-09-18 2021-08-24 阿里巴巴集团控股有限公司 Method and device for checking image content detection result
CN112560765A (en) * 2020-12-24 2021-03-26 上海明略人工智能(集团)有限公司 Pedestrian flow statistical method, system, equipment and storage medium based on pedestrian re-identification
CN116665243A (en) * 2021-03-23 2023-08-29 福建诺诚数字科技有限公司 Method, device and equipment for measuring pedestrian distance based on TOF depth map
TWI796033B (en) * 2021-12-07 2023-03-11 巨鷗科技股份有限公司 People flow analysis and identification system
CN116385969B (en) * 2023-04-07 2024-03-12 暨南大学 Personnel gathering detection system based on multi-camera cooperation and human feedback

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070098303A1 (en) * 2005-10-31 2007-05-03 Eastman Kodak Company Determining a particular person from a collection
CN101231755A (en) * 2007-01-25 2008-07-30 上海遥薇实业有限公司 Moving target tracking and quantity statistics method
TW201033908A (en) * 2009-03-12 2010-09-16 Micro Star Int Co Ltd System and method for counting people flow
CN108664946A (en) * 2018-05-18 2018-10-16 上海极歌企业管理咨询中心（有限合伙） Stream of people's characteristic-acquisition method based on image and device

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103425967B (en) * 2013-07-21 2016-06-01 浙江大学 A kind of based on stream of people's monitoring method of pedestrian detection and tracking
CN104318263A (en) * 2014-09-24 2015-01-28 南京邮电大学 Real-time high-precision people stream counting method
CN105809178A (en) * 2014-12-31 2016-07-27 中国科学院深圳先进技术研究院 Population analyzing method based on human face attribute and device
US9996752B2 (en) * 2016-08-30 2018-06-12 Canon Kabushiki Kaisha Method, system and apparatus for processing an image
CN108021848B (en) * 2016-11-03 2021-06-01 浙江宇视科技有限公司 Passenger flow volume statistical method and device
CN106650695A (en) * 2016-12-30 2017-05-10 苏州万店掌网络科技有限公司 Video analysis technology-based people flow tracking statistics system
CN107423708A (en) * 2017-07-25 2017-12-01 成都通甲优博科技有限责任公司 The method and its device of pedestrian's flow of the people in a kind of determination video
CN108388851A (en) * 2018-02-09 2018-08-10 北京京东金融科技控股有限公司 information statistical method, device, storage medium and electronic equipment
CN108764167B (en) * 2018-05-30 2020-09-29 上海交通大学 Space-time correlated target re-identification method and system
CN109902551A (en) * 2018-11-09 2019-06-18 阿里巴巴集团控股有限公司 The real-time stream of people's statistical method and device of open scene

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070098303A1 (en) * 2005-10-31 2007-05-03 Eastman Kodak Company Determining a particular person from a collection
CN101231755A (en) * 2007-01-25 2008-07-30 上海遥薇实业有限公司 Moving target tracking and quantity statistics method
TW201033908A (en) * 2009-03-12 2010-09-16 Micro Star Int Co Ltd System and method for counting people flow
CN108664946A (en) * 2018-05-18 2018-10-16 上海极歌企业管理咨询中心（有限合伙） Stream of people's characteristic-acquisition method based on image and device

Also Published As

Publication number Publication date
TW202036375A (en) 2020-10-01
WO2020093829A1 (en) 2020-05-14
CN109902551A (en) 2019-06-18

Similar Documents

Publication Publication Date Title
TWI729454B (en) Open scene real-time crowd flow statistics method and device, computer equipment and computer readable storage medium
JP7036863B2 (en) Systems and methods for activity monitoring using video data
TW202018594A (en) People flow condition estimation method and device for designated area
CN108629791B (en) Pedestrian tracking method and device and cross-camera pedestrian tracking method and device
US10380431B2 (en) Systems and methods for processing video streams
US9530221B2 (en) Context aware moving object detection
WO2020094088A1 (en) Image capturing method, monitoring camera, and monitoring system
CN107871114A (en) A kind of method, apparatus and system for pushing target person tracking information
Cetin et al. Methods and techniques for fire detection: signal, image and video processing perspectives
CN106570449B (en) A kind of flow of the people defined based on region and popularity detection method and detection system
Fu et al. Scene-adaptive accurate and fast vertical crowd counting via joint using depth and color information
Alipour et al. An adaptive background subtraction approach based on frame differences in video surveillance
Sandifort et al. An entropy model for loiterer retrieval across multiple surveillance cameras
WO2016172262A1 (en) Systems and methods for processing video data for activity monitoring
CN105989063B (en) Video retrieval method and device
CN112689120A (en) Monitoring method and device
US11334751B2 (en) Systems and methods for processing video data for activity monitoring
Luo et al. A real-time people counting approach in indoor environment
Wang Distributed multi-object tracking with multi-camera systems composed of overlapping and non-overlapping cameras
Ma et al. Location and Fusion Algorithm of High-Rise Building Rescue Drill Scene Based on Binocular Vision
Feng et al. Crowd Anomaly Scattering Detection Based on Information Entropy
Turchini et al. Open Set Recognition for Unique Person Counting via Virtual Gates
Xing et al. Coarse-to-Fine Multi-camera Network Topology Estimation
CN115346147A (en) Cross-lens real-time pedestrian trajectory analysis method and system