WO2020093830A1 - 指定区域的人流状况估算方法和装置 - Google Patents

指定区域的人流状况估算方法和装置 Download PDF

Info

Publication number
WO2020093830A1
WO2020093830A1 PCT/CN2019/110016 CN2019110016W WO2020093830A1 WO 2020093830 A1 WO2020093830 A1 WO 2020093830A1 CN 2019110016 W CN2019110016 W CN 2019110016W WO 2020093830 A1 WO2020093830 A1 WO 2020093830A1
Authority
WO
WIPO (PCT)
Prior art keywords
pedestrian
frame
designated area
pedestrians
valid
Prior art date
Application number
PCT/CN2019/110016
Other languages
English (en)
French (fr)
Inventor
张晓博
侯章军
杨旭东
Original Assignee
阿里巴巴集团控股有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴集团控股有限公司 filed Critical 阿里巴巴集团控股有限公司
Publication of WO2020093830A1 publication Critical patent/WO2020093830A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion

Definitions

  • This specification relates to the technical field of data processing, and in particular to a method and device for estimating the flow of people in a designated area.
  • Advertisement placement in real scenes has a different influence on people than online advertising, so even in the Internet era, various outdoor advertisements and indoor advertisements are still favored by advertisers.
  • the selection of advertising spots has a great influence on the effectiveness of advertising in real scenes. The greater the crowd flow in the area near the advertising spots, the more people are likely to see the advertising and the wider the reach of the advertising.
  • crowd traffic is not the only determinant of advertising effectiveness. Even in the same area, the specific position and orientation of the advertising screen and the flow of people passing through the area will also affect the delivery of a particular advertisement. If you can estimate the statistical data of the flow of people in a certain area, such as the age and gender of the crowd, it will make the determination of the advertisement position more accurate, or make the advertisements placed in a certain advertisement position more closely match the audience .
  • this specification provides a method for estimating the flow of people in designated areas, including:
  • Extract the currently valid frame from the real-time video stream that shoots the specified area the video stream is shot by a camera placed above the specified area, and its shooting range covers the entire specified area;
  • N is a natural number
  • Pedestrian re-identification algorithm is used to identify the same pedestrian in the current effective frame as at least one previous effective frame according to the characteristic information
  • the pedestrians passing through the designated area are determined, and the statistical values of N personal attributes of the pedestrian passing through the designated area are obtained from the single frame attribute statistics of the pedestrians passing through the designated area.
  • This specification also provides a device for estimating the flow of people in a designated area, including:
  • An effective frame extraction unit used to extract the current effective frame from the real-time video stream shooting the specified area; the video stream is shot by a camera arranged above the specified area, and its shooting range covers the entire specified area;
  • Pedestrian detection and feature unit used to detect each pedestrian in the current valid frame, generate feature information of each pedestrian, and identify N single frame attributes of each pedestrian according to the feature information; N is a natural number;
  • Pedestrian re-identification unit which is used to identify the same pedestrian in the current valid frame and at least one valid frame in the current valid frame according to the characteristic information
  • the attribute statistics unit is used to determine the pedestrians passing through the specified area based on the result of the pedestrian re-identification algorithm, and obtain the statistical values of N personal attributes of the pedestrian flow through the specified area from the single-frame attribute statistics of the pedestrians passing through the specified area.
  • a computer device provided in this specification includes: a memory and a processor; a computer program that can be executed by the processor is stored on the memory; when the processor runs the computer program, the method for estimating the flow of people in the designated area described above is executed The steps described.
  • This specification also provides a computer-readable storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the steps described in the method for estimating a flow of people in a designated area described above are performed.
  • the pedestrian re-identification algorithm Identify the pedestrians in the current valid frame and the same pedestrians in the previous valid frame, get the pedestrians passing through the specified area, and count the personal attribute statistics of the pedestrian flow through the specified area according to the pedestrian's single frame attribute, thus achieving a low
  • the computational cost of the algorithm accurately estimates the flow of people passing through the designated area, and provides a data basis for the implementation of various service items near the designated area.
  • FIG. 1 is an exemplary diagram of a designated area of an advertising screen and a camera installation position in an embodiment of this specification;
  • FIG. 2 is a flowchart of a method for estimating a flow of people in a designated area in an embodiment of this specification
  • Figure 3 is a schematic diagram of the structure of the software for estimating the flow of people running on the embedded development board in the application example of this specification;
  • FIG. 5 is a logical structure diagram of a device for estimating a flow of people in a designated area in an embodiment of the present specification.
  • the embodiment of this specification proposes a new method for estimating the flow of people in a specified area, extracting the current valid frame from the real-time video stream captured by a camera placed above the specified area, detecting pedestrians in the current effective frame, and generating single-frame attributes of pedestrian And identify whether it is the same as the pedestrian in the previous valid frame to determine the pedestrian passing through the designated area and to count the personal attribute statistics of the pedestrian flow through the designated area.
  • the embodiments of the present specification accurately estimate the flow of people passing through the designated area with a low calculation load, and can play an important role in guiding the setting and implementation of service items near the designated area.
  • the embodiments of this specification can run on any device with computing and storage capabilities, such as mobile phones, tablet computers, PCs (Personal Computers), notebooks, servers, etc .; it can also be run on two or more
  • the logical node of the device implements various functions in the embodiments of this specification.
  • the embodiments of the present specification use the video stream captured by the camera to estimate the flow of people.
  • the camera is placed above the designated area, and the people in the designated area are photographed at an oblique downward angle to generate a real-time video stream.
  • the shooting range of the camera can completely cover the designated area.
  • the designated area inside the solid line frame
  • the camera can be placed above the advertising screen to continuously shoot the entire designated area at an oblique downward angle.
  • FIG. 2 the flow of the method for estimating the flow of people in a designated area is shown in FIG. 2.
  • Step 210 Extract the currently valid frame from the real-time video stream of the designated area.
  • the camera installed above the designated area will continue to output a video stream taken at an obliquely downward viewing angle.
  • the video stream consists of a continuous frame of images. Based on a certain condition, each frame image that meets this condition can be continuously extracted from the video stream as an effective frame, and the pedestrian status can be estimated by continuously identifying the pedestrians in the effective frame. Take the last valid frame extracted from the video stream as the current valid frame.
  • the conditions for extracting effective frames can be set according to factors such as the requirements of the actual application scenario for statistical time accuracy, the processing capabilities of the device running this embodiment, and so on.
  • the frame can be separated from the last effective frame by K (K is a natural number)
  • K is a natural number
  • the frame image is taken as the next effective frame, and one frame can be extracted as an effective frame from every M (M is a natural number greater than 1) consecutive frames.
  • Step 220 Detect each pedestrian in the current valid frame, generate feature information of each pedestrian, and identify N (N is a natural number) single frame attributes of each pedestrian according to the feature information.
  • the target detection algorithm of deep learning determines whether there is a human body in the current effective frame, and if so, locates the position of each pedestrian and part of the image area occupied by the pedestrian.
  • the target detection algorithm is not limited.
  • Faster R-CNN Full Regions with Convolutional Neural Network Features
  • SSD Single Shot MultiBox Detector
  • SSD Single target multi-frame prediction
  • the position characteristics of the pedestrian can be generated from the position of each pedestrian (such as the coordinates of the partial area occupied by the pedestrian in the picture coordinate system ),
  • the pedestrian ’s appearance features (such as clothing color, clothing texture, handbag, backpack, hat, etc.) are generated from the image of the partial area occupied by the pedestrian, and the appearance features, or the location features and appearance features are regarded as the pedestrian ’s Feature information.
  • the attribute is used to describe a specific aspect of the pedestrian.
  • the attribute may be one or more of gender, age range, body orientation, clothing, etc. According to the requirements of actual application scenarios, camera resolution and other factors, it can be determined which attributes are the objects of identification and statistics.
  • the attribute of the pedestrian identified by the image of a pedestrian in a valid frame is called the single-frame attribute of the pedestrian.
  • the specific values of the N single-frame attributes of each pedestrian can be identified, such as gender attributes (male or female), age attributes (such as under 20, 20 to 30 years old, 30 to 40 years old, which age group is over 40 years old), body orientation attribute (such as front, side or back facing the camera).
  • a machine learning classification model can be used to identify the attributes of each single frame. Specifically, using the feature information of pedestrians as input, and the probability of each specific value of the attribute as the output to build a machine learning classification model, after training with labeled samples, you can use the classification model to obtain individual frames for pedestrians The estimated value of the attribute.
  • the embodiments of this specification do not limit the types of machine learning classification models, such as binary classification models, multi-classification models, etc .; the machine learning algorithms adopted by classification models are also not limited, such as linear regression, decision trees, Random forest etc.
  • a pedestrian re-identification algorithm is used to identify the same pedestrian in the current effective frame and at least one previous effective frame according to the characteristic information.
  • Pedestrian ReID can use computer vision technology to determine whether there is a specific pedestrian in the image, which can be used to track people in the same camera or across cameras.
  • a pedestrian re-identification algorithm is used to determine which of all pedestrians detected in the current effective frame are pedestrians that have already appeared in the previous effective frame, and which are new pedestrians that have appeared in the current effective frame.
  • K is a natural number
  • each pedestrian in the current effective frame is an existing pedestrian in the previous K effective frames according to the appearance characteristics and position characteristics of the pedestrian, and if not, a new character identification mark is generated Pedestrian, if it is, then mark the pedestrian with the existing character identification.
  • the identifier marks the pedestrian, and the character identifier is used to uniquely represent a pedestrian, which may be an index number, a character string, etc., without limitation. If the pedestrian already exists, the pedestrian already has his own character identification, and it is sufficient to mark the pedestrian in the currently valid frame with the character identification that has been previously available.
  • the above search process is performed on all pedestrians detected in the current valid frame one by one until each detected pedestrian is marked with a person identification.
  • the algorithm used when identifying whether the pedestrians in different valid frames are the same person can be selected according to the needs of the actual application scenario, without limitation.
  • a Hungarian algorithm can be used to match the pedestrians in the current effective frame with the previous effective frames according to the appearance characteristics and position characteristics of the pedestrians.
  • Step 240 Determine the pedestrians passing through the designated area based on the result of the pedestrian re-identification algorithm, and obtain the statistical values of N personal attributes of the pedestrian flow through the designated area from the single-frame attribute statistics of the pedestrians passing through the designated area.
  • pedestrians entering the designated area in each valid frame can be regarded as pedestrians passing through the designated area. If a pedestrian appears in the designated area in a valid frame, and the K valid frames before the valid frame do not appear in the designated area, the pedestrian is considered to enter the designated area in the valid frame.
  • Pedestrians leaving the designated area in each valid frame can be regarded as pedestrians passing through the designated area. If a pedestrian has appeared in the specified area in the K effective frames before a certain effective frame, and does not appear in the specified area in the effective frame and 1 to K effective frames after the effective frame, then It is considered that the pedestrian leaves the designated area at the valid frame.
  • the flow of people passing through the designated area is a collection of these pedestrians. From the N single-frame attributes of these pedestrians in each effective frame, the statistical values of the N personal attributes of the pedestrian flow through the designated area can be counted.
  • the statistical value of the personal attribute is the statistical result of the specific value of the personal attribute of each pedestrian in a pedestrian collection, for example, the gender attribute (the total number of men and the number of women), the age attribute (such as the number of people under 20, 20 to 30 The number of people aged 30, 40 to 40 years old, the number of people over 40 years old), body orientation attributes (such as the number of people facing the camera from the front, the number of people facing the camera from the side, and the number of people facing the camera from the back).
  • the gender attribute the total number of men and the number of women
  • the age attribute such as the number of people under 20, 20 to 30 The number of people aged 30, 40 to 40 years old, the number of people over 40 years old
  • body orientation attributes such as the number of people facing the camera from the front, the number of people facing the camera from the side, and the number of people facing the camera from the back.
  • the N personal attributes of the pedestrian can be determined according to the N single frame attributes of the same pedestrian in several valid frames.
  • N personal attributes of all pedestrians passing through the designated area and statistically obtaining statistical values of N personal attributes of the pedestrian flow passing through the designated area in the statistical period.
  • the statistical time period is a time period for accumulating the statistical values of personal attributes.
  • Pedestrians usually appear in multiple effective frames when they pass through the designated area.
  • the pedestrian's attributes based on the images of the pedestrians in these effective frames are called personal attributes of the pedestrian.
  • the pedestrian In each valid frame in which the pedestrian appears, the pedestrian will have N specific values of the single-frame attributes.
  • the specific values of the individual single-frame attributes may be used to generate specific values of the corresponding personal attributes.
  • a specific value of a single frame attribute that has the highest probability among all valid frames where the pedestrian appears can be used as the specific value of the personal attribute corresponding to the pedestrian; in the second In an implementation manner, it is possible to accumulate the possibility of a specific value of a single frame attribute in all valid frames where the pedestrian appears, and take the specific value with the largest accumulation result as the specific value of the personal attribute corresponding to the pedestrian.
  • a pedestrian appears in 3 effective frames, and the specific values of the gender attribute in the single frame attribute are: male 0.33, female 0.67 in the first effective frame, male 0.52, female 0.48 in the second effective frame, the first In the two effective frames, male 0.42 and female 0.58, in the first implementation, the specific value with the highest probability is female 0.67, then the gender attribute of the pedestrian's personal attribute is female; in the second implementation, The total probability of males is 1.27 and the total probability of females is 1.73, so the gender attribute of the pedestrian's personal attributes is female.
  • the N personal attributes of all pedestrians passing through the specified area during the statistical period are accumulated separately to obtain the statistical values of the N personal attributes of the pedestrians passing through the specified area during the statistical period.
  • the length of time that each pedestrian enters the effective frame of the designated area and the effective frame where the pedestrian leaves the designated area can be used as the length of time the pedestrian stays in the designated area .
  • the current effective frame is extracted from the real-time video stream captured by the camera placed above the designated area, and by detecting the pedestrians in the current effective frame, the feature information and single-frame attributes of each pedestrian are generated, using Pedestrian re-identification algorithm identifies the pedestrians in the current valid frame and the same pedestrians in the previous valid frame, determines the pedestrians passing through the specified area and counts the statistical values of the personal attributes of the flow of people passing through the specified area, which can be accurate at a lower cost Estimates the flow of people passing through the designated area, thus guiding the setting and implementation of service projects near the designated area.
  • the method of the embodiment of the present specification is suitable for running on an embedded development board, and has no special requirements on the hardware environment of the embedded development board.
  • the embedded development board running the embodiment of this specification can be installed near the camera, and the flow status data collected at a certain period can be sent to the server responsible for collecting the flow status data through its own communication module without uploading the video or image captured by the camera , Can get accurate statistical data without violating the privacy of pedestrians.
  • the quality of an advertising spot where an advertising screen is located needs to be evaluated.
  • the downward viewing angle shoots the designated area, which is located in the central part of the shooting range and is separated from the boundary of the shooting range by a certain distance.
  • the flow statistics are carried out by a program running on the embedded development board.
  • the embedded development board is installed near the camera, which includes a communication unit, and can be connected to the camera in a short-range wireless manner to obtain the video data captured by the camera.
  • the embedded development board can also upload the estimated flow status data to a predetermined server through the communication unit.
  • the embedded development board pre-stores a machine learning classification model that has been trained using labeled samples.
  • the classification model takes the appearance characteristics of pedestrians in a frame of images as input, and takes the four attributes of gender, age, body orientation, and clothing. The probability of the specific value is the output.
  • the RGB camera continuously captures images of open scenes at a rate of 25 frames per second to form a video stream.
  • the embedded development board extracts a frame of RGB images every fixed number of frames from the captured video stream as the current effective frame.
  • the flow statistics software uses the YOLO target detection algorithm to identify each pedestrian in the current valid frame, determine the position coordinates (a type of position feature) of each pedestrian in the image coordinate system, and the partial area occupied by each pedestrian. image.
  • the appearance characteristics of the pedestrian are extracted from the image of the area occupied by the pedestrian.
  • the probability of each specific value of the pedestrian's 4 single-frame attributes can be obtained.
  • the pedestrian re-identification algorithm uses the Hungarian algorithm to determine whether the pedestrian matches each pedestrian in the three effective frames before the current effective frame, and identify the pedestrian Whether the pedestrian has appeared in the previous 3 valid frames, if it has not appeared before, a new character logo is generated for the pedestrian to uniquely represent the pedestrian, and the pedestrian is marked with the new character logo; if it has ever appeared, the Pedestrians are already marked with a personal identity.
  • the pedestrians entering and leaving the designated area in each valid frame are determined. If a pedestrian appears in the designated area in a valid frame and does not appear in the designated area in the 3 valid frames before the valid frame, the pedestrian is considered to enter the designated area in the valid frame; if a certain Pedestrians have appeared in the designated area in the 3 valid frames before a valid frame, and have not appeared in the designated area in the valid frame and the 2 valid frames after the valid frame. The frame leaves the designated area.
  • Attributes and specific values accumulate the number of pedestrians leaving the designated area in all valid frames in a cycle, and take the cumulative results of the individual specific values of each individual attribute as the statistical results of the flow of people in the statistical time period; The length of stay of pedestrians leaving the designated area in the valid frame, and calculating the average length of stay of these pedestrians.
  • the embedded development board After the end of a period, the embedded development board sends to the predetermined server the statistical results of the flow status of the period and the average length of stay of the flow.
  • the server After receiving the information uploaded by the embedded development board, the server can evaluate the quality of the advertising spots based on the personal attribute statistics of the flow of people passing through the designated area.
  • the server can obtain detailed information about the number of people in various statistical time periods near the point, how long the person stays in front of the advertising point, whether the flow is forward, sideways or back to the screen, etc., not only can the advertiser be presented with accurate advertising Recommendations, but also to provide fine quote basis for advertising operators.
  • the embodiments of the present specification also provide an apparatus for estimating the flow of people in a designated area.
  • the device can be implemented by software, or by hardware or a combination of hardware and software. Taking software implementation as an example, as a logical device, it is formed by reading the corresponding computer program instructions into the memory through the CPU (Central Processing Unit) of the device where it is located. From the hardware level, in addition to the CPU, memory, and storage shown in FIG. 4, the device where the device for estimating the flow of people in the designated area usually includes other hardware such as chips for wireless signal transmission and reception, and / or Other hardware such as boards that implement network communication functions.
  • CPU Central Processing Unit
  • FIG. 5 shows a device for estimating the flow of people in a specified area provided by an embodiment of the present specification, which includes an effective frame extraction unit, a pedestrian detection and feature unit, a pedestrian re-identification unit, and an attribute statistical unit, where the effective frame extraction unit is used to Extract the current effective frame from the real-time video stream that shoots the specified area; the video stream is shot by a camera placed above the specified area, and its shooting range covers the entire specified area; pedestrian detection and feature unit is used to detect the current effective frame For each pedestrian, generate the feature information of each pedestrian, and identify the N single frame attributes of each pedestrian according to the feature information; N is a natural number; the pedestrian re-identification unit is used to adopt the pedestrian re-identification algorithm to identify the current valid frame based on the feature information The same pedestrian in at least one valid frame before; the attribute statistical unit is used to determine the pedestrian passing through the specified area based on the result of the pedestrian re-identification algorithm, and the single frame attribute statistics of the pedestrian passing through the specified area are used to obtain the flow
  • the pedestrian detection and feature unit recognizes each pedestrian's single-frame attribute based on the feature information, including: using a machine learning classification model to identify several single-frame attributes of each pedestrian; the machine learning classification model uses pedestrian's Feature information is input, and labeled samples are used for training.
  • the attribute statistical unit obtains statistical values of N personal attributes of pedestrians passing through the specified area from single frame attribute statistics of pedestrians passing through the specified area, including: according to N of the same pedestrian in several valid frames
  • the single frame attribute determines the N personal attributes of the pedestrian, and the N personal attributes of all pedestrians passing through the specified area in the statistical time period are counted to obtain the statistical values of the N personal attributes of the pedestrian flow through the specified area in the statistical time period .
  • the single-frame attribute and the personal attribute respectively include one or more of the following: gender, age group, body orientation, and clothing.
  • the feature information includes: appearance characteristics and position characteristics;
  • the pedestrian re-identification unit is specifically configured to: according to the appearance characteristics and position characteristics of pedestrians, determine whether each pedestrian in the current valid frame is at least one before If the pedestrian already exists in the valid frame, a new character identification is generated to mark the pedestrian, if not, the pedestrian is marked with an existing character identification.
  • the pedestrian re-identification unit determines whether a pedestrian in the current effective frame is an existing pedestrian in the previous effective frame according to the appearance characteristics and position characteristics of the pedestrian, including: using the Hungarian algorithm according to the appearance characteristics of the pedestrian And position feature to match the current valid frame with the pedestrian in the previous valid frame.
  • the pedestrians passing through the designated area include: pedestrians entering the designated area, or pedestrians leaving the designated area; the pedestrians entering the designated area include: appearing in the designated area in a valid frame, and Pedestrians that do not appear in the designated area in at least one valid frame before the valid frame; the pedestrians leaving the designated area include: appearing in the designated area in the adjacent valid frame before a valid frame, and in Pedestrians that do not appear in the designated area in the effective frame and at least one subsequent effective frame.
  • the device further includes: a stay time statistics unit for counting the stay time of the pedestrian flow passing through the specified area in the specified area, wherein the stay time of each pedestrian passing through the specified area is entered by the pedestrian into the specified area The length of time between a valid frame and a valid frame leaving a specified area is determined.
  • the device runs on an embedded development board.
  • the designated area includes: a designated area before an advertising spot; the device further includes: a spot evaluation unit, configured to perform quality assessment of the advertising spot based on the personal attribute statistical value attribute of the flow of people passing through the designated area Assessment.
  • a spot evaluation unit configured to perform quality assessment of the advertising spot based on the personal attribute statistical value attribute of the flow of people passing through the designated area Assessment.
  • the embodiments of the present specification provide a computer device including a memory and a processor.
  • the memory stores a computer program that can be executed by the processor; when the processor runs the stored computer program, it executes the steps of the method for estimating the flow of people in the designated area in the embodiment of the present specification.
  • the processor runs the stored computer program, it executes the steps of the method for estimating the flow of people in the designated area in the embodiment of the present specification.
  • the embodiments of the present specification provide a computer-readable storage medium on which computer programs are stored.
  • each step of the method for estimating the flow of people in a specified area in the embodiments of the present specification is executed .
  • each step of the method for estimating the flow of people in the designated area please refer to the previous content and it will not be repeated.
  • the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.
  • processors CPUs
  • input / output interfaces output interfaces
  • network interfaces network interfaces
  • memory volatile and non-volatile memory
  • the memory may include non-permanent memory, random access memory (RAM) and / or non-volatile memory in computer-readable media, such as read only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
  • RAM random access memory
  • ROM read only memory
  • flash RAM flash memory
  • Computer-readable media including permanent and non-permanent, removable and non-removable media, can store information by any method or technology.
  • the information may be computer readable instructions, data structures, modules of programs, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices.
  • computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.
  • the embodiments of the present specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of this specification may take the form of computer program products implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code .
  • computer usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

一种指定区域的人流状况估算方法，包括：从拍摄指定区域的实时视频流中提取当前有效帧（210）；检测当前有效帧中的每个行人，生成每个行人的特征信息，根据特征信息辨识每个行人的N个单帧属性（220）；N为自然数；采用行人重识别算法，根据特征信息识别当前有效帧中与之前的至少一个有效帧中相同的行人（230）；基于行人重识别算法的结果确定经过指定区域的行人，由经过指定区域的行人的单帧属性统计得到经过指定区域的人流的N个个人属性的统计值（240）。

Description

指定区域的人流状况估算方法和装置 技术领域
本说明书涉及数据处理技术领域，尤其涉及一种指定区域的人流状况估算方法和装置。
背景技术
在现实场景的广告投放对人们具有不同于网络广告的影响力，因而即使在互联网时代，各种户外广告、室内广告仍然受到广告主的青睐。广告点位的选择对现实场景中广告的效果具有很大的影响，广告点位附近区域的人群流量越大，意味着可能见到广告的人越多，广告波及的范围越广。
但是，人群流量并不是广告效果唯一的决定因素。即使在同一个的区域，广告屏幕的具体位置和朝向、经过该区域的人流状况，也会影响某个特定广告的投放效果。如果能够估算出经过某个区域人流状况，如人群的年龄、性别等属性的统计数据，将会使广告点位的确定更加准确，或者使得在某个广告点位投放的广告与受众更为匹配。
发明内容
有鉴于此，本说明书提供一种指定区域的人流状况估算方法，包括：
从拍摄所述指定区域的实时视频流中提取当前有效帧；所述视频流由安置在指定区域上方的摄像头拍摄，其拍摄范围覆盖整个指定区域；
检测当前有效帧中的每个行人，生成每个行人的特征信息，根据特征信息辨识每个行人的N个单帧属性；N为自然数；
采用行人重识别算法，根据特征信息识别当前有效帧中与之前的至少一个有效帧中相同的行人；
基于行人重识别算法的结果确定经过指定区域的行人，由经过指定区域的行人的单帧属性统计得到经过指定区域的人流的N个个人属性的统计值。
本说明书还提供了一种指定区域的人流状况估算装置，包括：
有效帧提取单元，用于从拍摄所述指定区域的实时视频流中提取当前有效帧；所 述视频流由安置在指定区域上方的摄像头拍摄，其拍摄范围覆盖整个指定区域；
行人检测及特征单元，用于检测当前有效帧中的每个行人，生成每个行人的特征信息，根据特征信息辨识每个行人的N个单帧属性；N为自然数；
行人重识别单元，用于采用行人重识别算法，根据特征信息识别当前有效帧中与之前的至少一个有效帧中相同的行人；
属性统计单元，用于基于行人重识别算法的结果确定经过指定区域的行人，由经过指定区域的行人的单帧属性统计得到经过指定区域的人流的N个个人属性的统计值。
本说明书提供的一种计算机设备，包括：存储器和处理器；所述存储器上存储有可由处理器运行的计算机程序；所述处理器运行所述计算机程序时，执行上述指定区域的人流状况估算方法所述的步骤。
本说明书还提供了一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器运行时，执行上述指定区域的人流状况估算方法所述的步骤。
由以上技术方案可见，本说明书的实施例中，基于摄像头在指定区域上方拍摄的视频流，通过检测当前有效帧中的行人，生成每个行人的特征信息和单帧属性，采用行人重识别算法识别出当前有效帧中的行人与之前有效帧中相同的行人，得出经过指定区域的行人，并按照行人的单帧属性统计经过指定区域的人流的个人属性统计值，从而实现了以较低的运算代价，准确的估算出经过指定区域的人流状况，为指定区域附近的各种服务项目的实施提供了数据基础。
附图说明
图1是本说明书实施例中一种广告屏的指定区域与摄像头安装位置的示例图；
图2是本说明书实施例中一种指定区域的人流状况估算方法的流程图；
图3是本说明书应用示例中嵌入式开发板上运行的人流状况估算软件的结构示意图；
图4是运行本说明书实施例的设备的一种硬件结构图；
图5是本说明书实施例中一种指定区域的人流状况估算装置的逻辑结构图。
具体实施方式
本说明书的实施例提出一种新的指定区域的人流状况估算，从安置在指定区域上方的摄像头拍摄的实时视频流中提取当前有效帧，检测当前有效帧中的行人、生成行人的单帧属性并识别与之前有效帧中的行人是否相同，以确定经过指定区域的行人并统计经过指定区域的人流的个人属性统计值。本说明书的实施例以较低的运算负荷，准确的估算出经过指定区域的人流状况，对指定区域附近服务项目的设置和实施能够起到重要的指导作用。
本说明书的实施例可以运行在任何具有计算和存储能力的设备上，如手机、平板电脑、PC(Personal Computer，个人电脑)、笔记本、服务器等设备；还可以由运行在两个或两个以上设备的逻辑节点来实现本说明书实施例中的各项功能。
本说明书的实施例采用摄像头拍摄的视频流来进行人流状况估算。摄像头安置在指定区域的上方，以斜向下的角度对指定区域内的人群进行拍摄，生成实时视频流，摄像头的拍摄范围能够完全覆盖指定区域。以图1中的广告屏为例，指定区域(实线框内部)位于该广告屏的前方，摄像头可以安置在广告屏幕的上方，以倾斜向下的角度持续拍摄整个指定区域。
本说明书的实施例中，指定区域的人流状况估算方法的流程如图2所示。
步骤210，从拍摄指定区域的实时视频流中提取当前有效帧。
安装在指定区域上方的摄像头将持续输出以斜向下视角拍摄的视频流，视频流由连续的一帧帧图像构成。可以基于一定的条件，持续不断的从视频流中将符合该条件的各帧图像提取出来作为有效帧，通过连续的辨识有效帧中的行人来进行人流状况的估计。将最后一个从视频流中提取的有效帧作为当前有效帧。
提取有效帧的条件可以根据实际应用场景对统计时间精度的要求、运行本实施例的设备的处理能力等因素来设置，例如，可以将与上一个有效帧间隔K(K为自然数)帧的一帧图像作为下一个有效帧，也可以从每M(M为大于1的自然数)个连续帧中提取一帧作为有效帧。
步骤220，检测当前有效帧中的每个行人，生成每个行人的特征信息，根据特征信息辨识每个行人的N(N为自然数)个单帧属性。
在提取当前有效帧后，通过深度学习的目标检测算法判断当前有效帧中是否存在 人体，如果存在则定位每个行人的位置、以及该行人所占据的部分图像区域。
本说明书实施例中对采用的目标检测算法不做限定，如可以采用Faster R-CNN(Faster Regions with Convolutional Neural Network features，卷积神经网络特征的快速目标区域识别)、SSD(Single Shot MultiBox Detector，单次目标多框预测)等。
在既要求较低计算量，又要求检测准确率的应用场景中，可以采用YOLO(You Only Live Once)目标检测算法，从当前有效帧中提取每个行人的图像范围和位置信息，往往可以达到更好的效果。
对目标检测算法输出的每个行人的位置、以及该行人所占据的部分图像区域，可以由每个行人的位置生成该行人的位置特征(如该行人占据的部分区域在图片坐标系中的坐标)，由该行人所占据的部分区域的图像生成该行人的外观特征(如衣服颜色、衣服纹理、手提包、背包、帽子等)，将外观特征、或者将位置特征和外观特征作为该行人的特征信息。
本说明书实施例中，属性用来描述行人的某个特定方面，例如，属性可以是性别、年龄段、身体朝向、衣着等等中的一个到多个。可以根据实际应用场景的需求、摄像头分辨率等因素，来确定将哪些属性作为辨识和统计的对象。根据某一个有效帧中某个行人的图像所辨识出的该行人的属性，称为该行人的单帧属性。
依据当前有效帧中每个行人的特征信息，可以辨识出每个行人的N个单帧属性的具体值，例如性别属性(是男是女)、年龄段属性(如处于20岁以下、20到30岁、30到40岁、40岁以上的哪个年龄段)、身体朝向属性(如正面、侧面还是背面朝向摄像头)。
可以采用机器学习分类模型来进行各个单帧属性的辨识。具体而言，以行人的特征信息为输入，以属性各个具体值的概率为输出构建机器学习分类模型，采用有标记的样本进行训练后，即可使用该分类模型得出对行人的各个单帧属性的估计值。本说明书实施例对机器学习分类模型的种类不做限定，例如可以是二分类模型、多分类模型等；对分类模型所采取的机器学习算法也不做限定，例如可以是线性回归、决策树、随机森林等。
步骤230，采用行人重识别算法，根据特征信息识别当前有效帧中与之前的至少一个有效帧中相同的行人。
当某个行人从指定区域经过时，会被拍摄到多个有效帧中。在进行人流状况估算 时，需要在各个有效帧中找出相同的行人，避免对同一个行人多次计数，才能得到准确的数据。
行人重识别(Person ReID，Person Re-identification)能够利用计算机视觉技术来判断图像中是否存在特定行人，可以用来进行同一个摄像头或跨摄像头的人物追踪。本说明书的实施例中，采用行人重识别算法来判断在当前有效帧中检测到的所有行人中，哪些是已经出现在之前有效帧中的行人，哪些是在当前有效帧中新出现的行人。
可以通过查找当前有效帧中的某个行人是否出现在当前有效帧之前的K(K为自然数)个有效帧里，来判断该行人是否是新出现的行人。由于本说明书实施例中摄像头以倾斜向下的角度拍摄指定区域，行人较为密集时，可能会出现某个行人在某个有效帧或某几个连续的有效帧中被他人遮挡而没有被检测到的情形，选取较大的K值可以避免在这种情况下错误的将该行人重复计数，但较大的K值也会带来更大的运算负荷。实际应用场景中，可以根据指定区域的行人密集程度、相邻有效帧的间隔时间、运行本实施例的设备的处理能力等因素，来选择适当的K值。
在一种实现方式中，可以根据行人的外观特征和位置特征，判定当前有效帧中的每个行人是否是之前的K个有效帧中已存在的行人，如果不是则生成新的人物标识标记该行人，如果是则以已有的人物标识标记该行人。
具体而言，采用当前有效帧中某个行人的位置特征和外观特征，在之前的K个有效帧中查找是否已经存在该行人，如果不存在，为该行人生成新的人物标识并用生成的人物标识标记该行人，人物标识用来唯一的代表一个行人，可以是索引号、字符串等，不做限定。如果已经存在该行人，则该行人已经具有自己的人物标识，沿用之前已有的人物标识标记当前有效帧中的该行人即可。对当前有效帧中检测出的所有行人逐个执行上述查找过程，直到检测出的每个行人都标记有人物标识。
可以根据实际应用场景的需要，来选择识别不同有效帧中的行人是否是同一个人时采用的算法，不做限定。例如，可以采用匈牙利算法，来根据行人的外观特征和位置特征进行当前有效帧与之前有效帧中的行人匹配。
步骤240，基于行人重识别算法的结果确定经过指定区域的行人，由经过指定区域的行人的单帧属性统计得到经过指定区域的人流的N个个人属性的统计值。
可以根据实际应用的需要，来确定将哪些行人作为经过指定区域的行人，不做限定。以下举例说明：
第一个例子：可以将在各个有效帧中进入指定区域的行人作为经过指定区域的行人。如果在某个有效帧中某个行人出现在指定区域内、并且在该有效帧之前的K个有效帧中未出现在指定区域内，则认为该行人在该有效帧进入指定区域。
第二个例子：可以将在各个有效帧中离开指定区域的行人作为经过指定区域的行人。如果某个行人在某个有效帧之前的K个有效帧中曾经出现在指定区域内、并且在该有效帧以及该有效帧之后的1个到K个有效帧中未出现在指定区域内，则认为该行人在该有效帧离开指定区域。
在确定经过指定区域的行人后，经过指定区域的人流即是这些行人的集合。由这些行人在各个有效帧中的N个单帧属性，可以统计出经过指定区域的人流的N个个人属性的统计值。其中，个人属性的统计值是对某个行人集合中各个行人的个人属性具体值的统计结果，例如性别属性(男性总数和女性人数)、年龄段属性(如20岁以下的人数、20到30岁的人数、30到40岁的人数、40岁以上的人数)、身体朝向属性(如正面朝向摄像头的人数、侧面朝向摄像头的人数、背面朝向摄像头的人数)。
在由经过指定区域的行人的单帧属性生成个人属性的统计值时，可以根据同一个行人在若干个有效帧中的N个单帧属性确定该行人的N个个人属性，由统计时间段内经过指定区域的所有行人的N个个人属性，统计得到该统计时间段经过指定区域的人流的N个个人属性的统计值。其中，统计时间段是对个人属性的统计值进行累计的时间段。
在行人经过指定区域时，通常会出现在多个有效帧中，以这些有效帧中该行人的图像为依据所估计的该行人的属性，称为该行人的个人属性。在该行人出现的每个有效帧中，该行人都会有N个单帧属性的具体值，可以由这些各个单帧属性的具体值，生成对应的个人属性的具体值。
举例说明，在第一种实现方式中，可以将某个单帧属性在该行人出现的所有有效帧中具有最大可能性的具体值，作为该行人对应的个人属性的具体值；在第二种实现方式中，可以将该行人出现的所有有效帧中某个单帧属性具体值的可能性进行累加，将累加结果最大的一个具体值作为该行人对应的个人属性的具体值。如，某行人在3个有效帧中出现，其单帧属性中性别属性的具体值分别是：第一个有效帧中男性0.33、女性0.67，第二个有效帧中男性0.52、女性0.48，第二个有效帧中男性0.42、女性0.58，则在第一种实现方式中，具有最大可能性的具体值为女性0.67，则该行人个人属性中性别属性为女性；在第二种实现方式中，男性的可能性总和为1.27，女性的可能性总和为1.73，则该行人个人属性中性别属性为女性。
对统计时间段内经过指定区域的所有行人的N个个人属性分别进行累计，即可得到统计时间段经过指定区域的人流的N个个人属性的统计值。
在以人物标识对每个行人进行标记的实现方式中，可以将每个行人进入指定区域的有效帧、与该行人离开指定区域的有效帧之间的时长，作为该行人在指定区域的停留时长。在统计经过指定区域的人流的个人属性的统计值时，还可以统计这些行人的停留时长，如停留时长总和或平均停留时长。
需要说明的是，在统计经过指定区域的人流的个人属性统计值时，累加每个个人属性每个具体值的统计值，即可得到经过指定区域的人流数量，因而可以不必单独对人群流量做累计。
可见，本说明书的实施例中，从安置在指定区域上方的摄像头拍摄的实时视频流中提取当前有效帧，通过检测当前有效帧中的行人，生成每个行人的特征信息和单帧属性，采用行人重识别算法识别出当前有效帧中的行人与之前有效帧中相同的行人，确定经过指定区域的行人并统计经过指定区域的人流的个人属性的统计值，能够以较低的运算代价，准确的估算出经过指定区域的人流状况，从而对指定区域附近服务项目的设置和实施起到指导作用。
由于运算负荷较低，本说明书实施例的方法适于运行在嵌入式开发板上，并且对嵌入式开发板的硬件环境没有特别要求。运行本说明书实施例的嵌入式开发板可以安装在摄像机附近，将以一定周期统计出的人流状况数据通过自身的通信模块发送给负责采集人流状况数据的服务器，而无需上传摄像机拍摄的视频或者图像，能够在不侵犯行人的隐私的条件下得到精确的统计数据。
上述对本说明书特定实施例进行了描述。其它实施例在所附权利要求书的范围内。在一些情况下，在权利要求书中记载的动作或步骤可以按照不同于实施例中的顺序来执行并且仍然可以实现期望的结果。另外，在附图中描绘的过程不一定要求示出的特定顺序或者连续顺序才能实现期望的结果。在某些实施方式中，多任务处理和并行处理也是可以的或者可能是有利的。
在本说明书的一个应用示例中，需要对一块广告屏所在的广告点位质量进行评估。将该广告点位前行人可以停留或通过、并且能够了解该广告屏幕上播放内容的的区域作为指定区域，在广告屏幕的顶部中间位置安装RGB(Red Green Blue，红绿蓝)摄像头，以斜向下的视角对指定区域进行拍摄，指定区域位于拍摄范围的中央部分，与拍摄范围 的边界相隔一定距离。
人流状况统计由运行在嵌入式开发板上的程序进行，嵌入式开发板安装在摄像头附近，其上包括通信单元，可以与摄像头通过近距离无线方式连接，从摄像头获取其拍摄的视频数据。嵌入式开发板还可以通过通信单元将估算得出的人流状况数据上传给预定的服务器。
嵌入式开发板上预先保存有采用有标记样本训练完毕的机器学习分类模型，该分类模型以行人在一帧图像中的外观特征作为输入，以性别、年龄段、身体朝向、衣着这4个属性的具体值的概率为输出。
嵌入式开发板上运行的人流状况统计软件的结构如图3所示。
RGB摄像头以25帧/秒的速度持续拍摄开放场景的图像，形成视频流。嵌入式开发板从拍摄的视频流中每隔固定数量的帧提取一帧RGB图像，作为当前有效帧。
人流状况统计软件采用YOLO目标检测算法，识别出当前有效帧中的每个行人，确定每个行人在图像坐标系中的位置坐标(一种位置特征)，以及每个行人所占据的部分区域的图像。
针对当前有效帧中的每个行人，从该行人占据区域的图像中提取该行人的外观特征。将外观特征输入到机器学习分类模型，可以得到该行人的4个单帧属性的每个具体值的概率。
分别以当前有效帧中的每个行人的外观特征和位置坐标为依据，行人重识别算法采用匈牙利算法判断该行人是否与当前有效帧之前的3个有效帧中的各个行人是否匹配，鉴别出该行人是否在之前的3个有效帧中出现过，如果未曾出现过，则为该行人生成新的人物标识来唯一代表该行人，并用新的人物标识标记该行人；如果曾经出现过，则采用该行人已有的人物标识来标记该行人。
基于行人重识别算法给当前有效帧中每个行人标记的人物标识，以及YOLO目标检测算法输出的每个行人的位置坐标，判断在各个有效帧中进入和离开指定区域的行人。如果在某个有效帧中某个行人出现在指定区域内、并且在该有效帧之前的3个有效帧中未出现在指定区域内，则认为该行人在该有效帧进入指定区域；如果某个行人在某个有效帧之前的3个有效帧中曾经出现在指定区域内、并且在该有效帧以及该有效帧之后的2个有效帧中未出现在指定区域内，则认为该行人在该有效帧离开指定区域。
以预定的统计时间段为周期，对在该统计时间段的所有有效帧离开指定区域的每 个行人，累加该行人出现的各个有效帧的各个单帧属性的各个具体值，将某个属性的累加和最高的具体值作为该行人对应的个人属性的具体值；将该行人离开指定区域的有效帧和该行人进入指定区域的有效帧之间的时间间隔作为该行人在指定区域的停留时长。
分属性分具体值累计在一个周期内所有有效帧中离开指定区域的行人的数量，将各个个人属性的各个具体值累计结果作为该统计时间段内的人流状况统计结果；累加在一个周期内所有有效帧中离开指定区域的行人的停留时长，计算这些行人的平均停留时长。
在一个周期结束后，嵌入式开发板向预定的服务器发送该周期的人流状况统计结果和人流的平均停留时长。
服务器在收到嵌入式开发板上传的信息后，可以根据经过指定区域的人流的个人属性统计值来对广告点位的质量进行评估。服务器可以获得点位附近在各个统计时间段的人流数量、人流在广告点位前停留的时间、人流是正向、侧向还是背向屏幕等的详细信息，不仅可以向广告主提出准确的广告投放建议，同时也能为广告运营商提供精细的报价依据。
与上述流程实现对应，本说明书的实施例还提供了一种指定区域的人流状况估算装置。该装置可以通过软件实现，也可以通过硬件或者软硬件结合的方式实现。以软件实现为例，作为逻辑意义上的装置，是通过所在设备的CPU(Central Process Unit，中央处理器)将对应的计算机程序指令读取到内存中运行形成的。从硬件层面而言，除了图4所示的CPU、内存以及存储器之外，指定区域的人流状况估算装置所在的设备通常还包括用于进行无线信号收发的芯片等其他硬件，和/或用于实现网络通信功能的板卡等其他硬件。
图5所示为本说明书实施例提供的一种指定区域的人流状况估算装置，包括有效帧提取单元、行人检测及特征单元、行人重识别单元和属性统计单元，其中：有效帧提取单元用于从拍摄所述指定区域的实时视频流中提取当前有效帧；所述视频流由安置在指定区域上方的摄像头拍摄，其拍摄范围覆盖整个指定区域；行人检测及特征单元用于检测当前有效帧中的每个行人，生成每个行人的特征信息，根据特征信息辨识每个行人的N个单帧属性；N为自然数；行人重识别单元用于采用行人重识别算法，根据特征信息识别当前有效帧中与之前的至少一个有效帧中相同的行人；属性统计单元用于基于行人重识别算法的结果确定经过指定区域的行人，由经过指定区域的行人的单帧属性统计得到经过指定区域的人流的N个个人属性的统计值。
可选的，所述行人检测及特征单元根据特征信息辨识每个行人的单帧属性，包括：采用机器学习分类模型辨识每个行人的若干个单帧属性；所述机器学习分类模型以行人的特征信息为输入，采用有标记的样本进行训练。
可选的，所述属性统计单元由经过指定区域的行人的单帧属性统计得到经过指定区域的人流的N个个人属性的统计值，包括：根据同一个行人在若干个有效帧中的N个单帧属性确定所述行人的N个个人属性，由统计时间段内经过指定区域的所有行人的N个个人属性，统计得到所述统计时间段经过指定区域的人流的N个个人属性的统计值。
可选的，所述单帧属性和个人属性分别包括以下一项到多项：性别、年龄段、身体朝向、衣着。
一个例子中，所述特征信息包括：外观特征和位置特征；所述行人重识别单元具体用于：根据行人的外观特征和位置特征，判定当前有效帧中的每个行人是否是之前的至少一个有效帧中已存在的行人，如果不是则生成新的人物标识标记所述行人，如果是则以已有的人物标识标记所述行人。
上述例子中，所述行人重识别单元根据行人的外观特征和位置特征，判定当前有效帧中的某个行人是否是之前有效帧中已存在的行人，包括：采用匈牙利算法，根据行人的外观特征和位置特征进行当前有效帧与之前有效帧中的行人匹配。
一种实现方式中，所述经过指定区域的行人包括：进入指定区域的行人、或离开指定区域的行人；所述进入指定区域的行人包括：在某个有效帧中出现在指定区域内、并且在所述有效帧之前的至少一个有效帧中未出现在指定区域内的行人；所述离开指定区域的行人包括：在某个有效帧之前的相邻有效帧中出现在指定区域内、并且在所述有效帧及其后的至少一个有效帧中未出现在指定区域内的行人。
上述实现方式中，所述装置还包括：停留时长统计单元，用于统计经过指定区域的人流在指定区域的停留时长，其中经过指定区域的每个行人的停留时长由所述行人进入指定区域的有效帧与离开指定区域的有效帧之间的时长确定。
可选的，所述装置运行在嵌入式开发板上。
可选的，所述指定区域包括：广告点位前的指定区域；所述装置还包括：点位评估单元，用于根据经过指定区域的人流的个人属性统计值属性对广告点位的质量进行评估。
本说明书的实施例提供了一种计算机设备，该计算机设备包括存储器和处理器。 其中，存储器上存储有能够由处理器运行的计算机程序；处理器在运行存储的计算机程序时，执行本说明书实施例中指定区域的人流状况估算方法的各个步骤。对指定区域的人流状况估算方法的各个步骤的详细描述请参见之前的内容，不再重复。
本说明书的实施例提供了一种计算机可读存储介质，该存储介质上存储有计算机程序，这些计算机程序在被处理器运行时，执行本说明书实施例中指定区域的人流状况估算方法的各个步骤。对指定区域的人流状况估算方法的各个步骤的详细描述请参见之前的内容，不再重复。
以上所述仅为本说明书的较佳实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。
在一个典型的配置中，计算设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。
还需要说明的是，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、商品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、商品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、商品或者设备中还存在另外的相同要素。
本领域技术人员应明白，本说明书的实施例可提供为方法、系统或计算机程序产品。因此，本说明书的实施例可采用完全硬件实施例、完全软件实施例或结合软件和硬件方面的实施例的形式。而且，本说明书的实施例可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。

Claims (22)

  1. 一种指定区域的人流状况估算方法，包括：
    从拍摄所述指定区域的实时视频流中提取当前有效帧；所述视频流由安置在指定区域上方的摄像头拍摄，其拍摄范围覆盖整个指定区域；
    检测当前有效帧中的每个行人，生成每个行人的特征信息，根据特征信息辨识每个行人的N个单帧属性；N为自然数；
    采用行人重识别算法，根据特征信息识别当前有效帧中与之前的至少一个有效帧中相同的行人；
    基于行人重识别算法的结果确定经过指定区域的行人，由经过指定区域的行人的单帧属性得到经过指定区域的人流的N个个人属性的统计值。
  2. 根据权利要求1所述的方法，所述根据特征信息辨识每个行人的单帧属性，包括：采用机器学习分类模型辨识每个行人的若干个单帧属性；所述机器学习分类模型以行人的特征信息为输入，采用有标记的样本进行训练。
  3. 根据权利要求1所述的方法，所述由经过指定区域的行人的单帧属性统计得到经过指定区域的人流的N个个人属性的统计值，包括：根据同一个行人在若干个有效帧中的N个单帧属性确定所述行人的N个个人属性，由统计时间段内经过指定区域的所有行人的N个个人属性，统计得到所述统计时间段经过指定区域的人流的N个个人属性的统计值。
  4. 根据权利要求1所述的方法，所述单帧属性和个人属性分别包括以下一项到多项：性别、年龄段、身体朝向、衣着。
  5. 根据权利要求1所述的方法，所述特征信息包括：外观特征和位置特征；
    所述采用行人重识别算法，根据特征信息识别当前有效帧中与之前的至少一个有效帧中相同的行人，包括：根据行人的外观特征和位置特征，判定当前有效帧中的每个行人是否是之前的至少一个有效帧中已存在的行人，如果不是则生成新的人物标识标记所述行人，如果是则以已有的人物标识标记所述行人。
  6. 根据权利要求5所述的方法，所述根据行人的外观特征和位置特征，判定当前有效帧中的某个行人是否是之前有效帧中已存在的行人，包括：采用匈牙利算法，根据行人的外观特征和位置特征进行当前有效帧与之前有效帧中的行人匹配。
  7. 根据权利要求1所述的方法，所述经过指定区域的行人包括：进入指定区域的行人、或离开指定区域的行人；
    所述进入指定区域的行人包括：在某个有效帧中出现在指定区域内、并且在所述有 效帧之前的至少一个有效帧中未出现在指定区域内的行人；
    所述离开指定区域的行人包括：在某个有效帧之前的相邻有效帧中出现在指定区域内、并且在所述有效帧及其后的至少一个有效帧中未出现在指定区域内的行人。
  8. 根据权利要求7所述的方法，所述方法还包括：统计经过指定区域的人流在指定区域的停留时长，其中经过指定区域的每个行人的停留时长由所述行人进入指定区域的有效帧与离开指定区域的有效帧之间的时长确定。
  9. 根据权利要求1所述的方法，所述方法运行在嵌入式开发板上。
  10. 根据权利要求1所述的方法，所述指定区域包括：广告点位前的指定区域；
    所述方法还包括：根据经过指定区域的人流的个人属性统计值对广告点位的质量进行评估。
  11. 一种指定区域的人流状况估算装置，包括：
    有效帧提取单元，用于从拍摄所述指定区域的实时视频流中提取当前有效帧；所述视频流由安置在指定区域上方的摄像头拍摄，其拍摄范围覆盖整个指定区域；
    行人检测及特征单元，用于检测当前有效帧中的每个行人，生成每个行人的特征信息，根据特征信息辨识每个行人的N个单帧属性；N为自然数；
    行人重识别单元，用于采用行人重识别算法，根据特征信息识别当前有效帧中与之前的至少一个有效帧中相同的行人；
    属性统计单元，用于基于行人重识别算法的结果确定经过指定区域的行人，由经过指定区域的行人的单帧属性统计得到经过指定区域的人流的N个个人属性的统计值。
  12. 根据权利要求11所述的装置，所述行人检测及特征单元根据特征信息辨识每个行人的单帧属性，包括：采用机器学习分类模型辨识每个行人的若干个单帧属性；所述机器学习分类模型以行人的特征信息为输入，采用有标记的样本进行训练。
  13. 根据权利要求11所述的装置，所述属性统计单元由经过指定区域的行人的单帧属性统计得到经过指定区域的人流的N个个人属性的统计值，包括：根据同一个行人在若干个有效帧中的N个单帧属性确定所述行人的N个个人属性，由统计时间段内经过指定区域的所有行人的N个个人属性，统计得到所述统计时间段经过指定区域的人流的N个个人属性的统计值。
  14. 根据权利要求11所述的装置，所述单帧属性和个人属性分别包括以下一项到多项：性别、年龄段、身体朝向、衣着。
  15. 根据权利要求11所述的装置，所述特征信息包括：外观特征和位置特征；
    所述行人重识别单元具体用于：根据行人的外观特征和位置特征，判定当前有效帧 中的每个行人是否是之前的至少一个有效帧中已存在的行人，如果不是则生成新的人物标识标记所述行人，如果是则以已有的人物标识标记所述行人。
  16. 根据权利要求15所述的装置，所述行人重识别单元根据行人的外观特征和位置特征，判定当前有效帧中的某个行人是否是之前有效帧中已存在的行人，包括：采用匈牙利算法，根据行人的外观特征和位置特征进行当前有效帧与之前有效帧中的行人匹配。
  17. 根据权利要求11所述的装置，所述经过指定区域的行人包括：进入指定区域的行人、或离开指定区域的行人；
    所述进入指定区域的行人包括：在某个有效帧中出现在指定区域内、并且在所述有效帧之前的至少一个有效帧中未出现在指定区域内的行人；
    所述离开指定区域的行人包括：在某个有效帧之前的相邻有效帧中出现在指定区域内、并且在所述有效帧及其后的至少一个有效帧中未出现在指定区域内的行人。
  18. 根据权利要求17所述的装置，所述装置还包括：停留时长统计单元，用于统计经过指定区域的人流在指定区域的停留时长，其中经过指定区域的每个行人的停留时长由所述行人进入指定区域的有效帧与离开指定区域的有效帧之间的时长确定。
  19. 根据权利要求11所述的装置，所述装置运行在嵌入式开发板上。
  20. 根据权利要求11所述的装置，所述指定区域包括：广告点位前的指定区域；
    所述装置还包括：点位评估单元，用于根据经过指定区域的人流的个人属性统计值对广告点位的质量进行评估。
  21. 一种计算机设备，包括：存储器和处理器；所述存储器上存储有可由处理器运行的计算机程序；所述处理器运行所述计算机程序时，执行如权利要求1到10任意一项所述的步骤。
  22. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器运行时，执行如权利要求1到10任意一项所述的步骤。
PCT/CN2019/110016 2018-11-09 2019-10-08 指定区域的人流状况估算方法和装置 WO2020093830A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811332839.XA CN110008799A (zh) 2018-11-09 2018-11-09 指定区域的人流状况估算方法和装置
CN201811332839.X 2018-11-09

Publications (1)

Publication Number Publication Date
WO2020093830A1 true WO2020093830A1 (zh) 2020-05-14

Family

ID=67164879

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/110016 WO2020093830A1 (zh) 2018-11-09 2019-10-08 指定区域的人流状况估算方法和装置

Country Status (3)

Country Link
CN (1) CN110008799A (zh)
TW (1) TW202018594A (zh)
WO (1) WO2020093830A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113807200A (zh) * 2021-08-26 2021-12-17 青岛文达通科技股份有限公司 基于动态拟合多任务推理网络的多行人识别方法及系统
CN114332778A (zh) * 2022-03-08 2022-04-12 深圳市万物云科技有限公司 基于人流密度的智慧告警工单生成方法、装置及相关介质
CN115631464A (zh) * 2022-11-17 2023-01-20 北京航空航天大学 面向大时空目标关联的行人立体表示方法

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110008799A (zh) * 2018-11-09 2019-07-12 阿里巴巴集团控股有限公司 指定区域的人流状况估算方法和装置
CN112005247A (zh) * 2020-06-12 2020-11-27 深圳盈天下视觉科技有限公司 人流量数据监控系统及其人流量数据的展示方法和装置
CN112308609B (zh) * 2020-10-31 2023-06-27 成都新潮传媒集团有限公司 一种广告投放计费方法、装置及计算机设备
TWI748723B (zh) * 2020-11-02 2021-12-01 鴻海精密工業股份有限公司 物體定位方法、裝置及存儲介質
CN112560765A (zh) * 2020-12-24 2021-03-26 上海明略人工智能(集团)有限公司 基于行人重识别的人流统计方法、系统、设备及存储介质
CN112906483B (zh) * 2021-01-25 2024-01-23 中国银联股份有限公司 一种目标重识别方法、装置及计算机可读存储介质
CN117423067B (zh) * 2023-12-18 2024-03-12 成都华芯智云科技有限公司 一种基于tof技术的客流统计终端

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101237563A (zh) * 2007-01-31 2008-08-06 华为技术有限公司 节目受众信息获取方法和装置及节目播放系统
US20150317517A1 (en) * 2012-11-26 2015-11-05 Zte Corporation Method, apparatus and system for acquiring headcount
CN105809178A (zh) * 2014-12-31 2016-07-27 中国科学院深圳先进技术研究院 一种基于人脸属性的人群分析方法及装置
CN108388851A (zh) * 2018-02-09 2018-08-10 北京京东金融科技控股有限公司 信息统计方法、装置、存储介质及电子设备
CN108647612A (zh) * 2018-04-28 2018-10-12 成都睿码科技有限责任公司 广告牌观看人流量分析系统
CN110008799A (zh) * 2018-11-09 2019-07-12 阿里巴巴集团控股有限公司 指定区域的人流状况估算方法和装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103559478B (zh) * 2013-10-07 2018-12-04 唐春晖 俯视行人视频监控中的客流计数与事件分析方法
CN108664946A (zh) * 2018-05-18 2018-10-16 上海极歌企业管理咨询中心（有限合伙） 基于图像的人流特征获取方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101237563A (zh) * 2007-01-31 2008-08-06 华为技术有限公司 节目受众信息获取方法和装置及节目播放系统
US20150317517A1 (en) * 2012-11-26 2015-11-05 Zte Corporation Method, apparatus and system for acquiring headcount
CN105809178A (zh) * 2014-12-31 2016-07-27 中国科学院深圳先进技术研究院 一种基于人脸属性的人群分析方法及装置
CN108388851A (zh) * 2018-02-09 2018-08-10 北京京东金融科技控股有限公司 信息统计方法、装置、存储介质及电子设备
CN108647612A (zh) * 2018-04-28 2018-10-12 成都睿码科技有限责任公司 广告牌观看人流量分析系统
CN110008799A (zh) * 2018-11-09 2019-07-12 阿里巴巴集团控股有限公司 指定区域的人流状况估算方法和装置

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113807200A (zh) * 2021-08-26 2021-12-17 青岛文达通科技股份有限公司 基于动态拟合多任务推理网络的多行人识别方法及系统
CN113807200B (zh) * 2021-08-26 2024-04-19 青岛文达通科技股份有限公司 基于动态拟合多任务推理网络的多行人识别方法及系统
CN114332778A (zh) * 2022-03-08 2022-04-12 深圳市万物云科技有限公司 基于人流密度的智慧告警工单生成方法、装置及相关介质
CN114332778B (zh) * 2022-03-08 2022-06-21 深圳市万物云科技有限公司 基于人流密度的智慧告警工单生成方法、装置及相关介质
CN115631464A (zh) * 2022-11-17 2023-01-20 北京航空航天大学 面向大时空目标关联的行人立体表示方法
CN115631464B (zh) * 2022-11-17 2023-04-04 北京航空航天大学 面向大时空目标关联的行人立体表示方法

Also Published As

Publication number Publication date
TW202018594A (zh) 2020-05-16
CN110008799A (zh) 2019-07-12

Similar Documents

Publication Publication Date Title
WO2020093830A1 (zh) 指定区域的人流状况估算方法和装置
US10880524B2 (en) System and method for activity monitoring using video data
WO2020093829A1 (zh) 开放场景的实时人流统计方法和装置
US11138442B2 (en) Robust, adaptive and efficient object detection, classification and tracking
US11861850B2 (en) System and method for player reidentification in broadcast video
US10043078B2 (en) Virtual turnstile system and method
KR102097781B1 (ko) 객체 정보 제공 장치 및 방법
CN114783037B (zh) 目标重识别方法、目标重识别装置和计算机可读存储介质
WO2016172262A1 (en) Systems and methods for processing video data for activity monitoring
US11334751B2 (en) Systems and methods for processing video data for activity monitoring
CN113947795A (zh) 口罩佩戴检测方法、装置、设备及存储介质
CN114387648A (zh) 异常图像检测方法、装置、计算机设备及存储介质
JP2015162241A (ja) 情報提供効果を測定する測定システム及び情報提供効果測定方法
CN110956644A (zh) 一种运动轨迹确定方法及系统
CN110222942B (zh) 一种识别导购员的方法、装置、电子设备及存储介质
TWM488698U (zh) 智慧型影像式顧客分析系統
CN112949538A (zh) 一种目标关联方法、装置、电子设备及机器可读存储介质
CN112560557A (zh) 人数检测方法、人脸检测方法、装置及电子设备
Rother et al. What Can Casual Walkers Tell Us About a 3D Scene?(PREPRINT)

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19882946

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19882946

Country of ref document: EP

Kind code of ref document: A1