CN112580633B - Public transport passenger flow statistics device and method based on deep learning - Google Patents
Public transport passenger flow statistics device and method based on deep learning Download PDFInfo
- Publication number
- CN112580633B CN112580633B CN202011560622.1A CN202011560622A CN112580633B CN 112580633 B CN112580633 B CN 112580633B CN 202011560622 A CN202011560622 A CN 202011560622A CN 112580633 B CN112580633 B CN 112580633B
- Authority
- CN
- China
- Prior art keywords
- queue
- cur
- personnel
- area
- personnel area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000013135 deep learning Methods 0.000 title claims abstract description 20
- 238000001514 detection method Methods 0.000 claims abstract description 37
- 238000005070 sampling Methods 0.000 claims abstract description 17
- 238000004422 calculation algorithm Methods 0.000 claims description 25
- 238000001914 filtration Methods 0.000 claims description 17
- 238000004364 calculation method Methods 0.000 claims description 16
- 238000005304 joining Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 7
- 230000001788 irregular Effects 0.000 claims description 6
- 238000000354 decomposition reaction Methods 0.000 claims description 4
- 238000007619 statistical method Methods 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 abstract description 5
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000013461 design Methods 0.000 abstract description 2
- 238000010191 image analysis Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 16
- 230000008569 process Effects 0.000 description 8
- 238000004590 computer program Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000003860 storage Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000003416 augmentation Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 229910052799 carbon Inorganic materials 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000008685 targeting Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30242—Counting objects in image
Abstract
The invention relates to the field of video image analysis and artificial intelligence, in particular to a device and a method for dynamically counting public transportation passenger flow based on video analysis deep learning. The method adopts the region of interest to divide the acquired image video, carries out secondary sampling on the image video, carries out image recognition, object tracking, data association matching, detection region setting and geometric relationship comparison, judges the people flow on line in real time, has reasonable flow design, high operation efficiency and high accuracy, and can also improve the accuracy by adopting secondary sampling.
Description
Technical Field
The invention belongs to the field of video image analysis and artificial intelligence, and relates to a device and a method for dynamically carrying out public transportation passenger flow statistics based on video analysis based on deep learning.
Background
The increasing urban population makes urban traffic pressure greater and greater, and the preferential development of public traffic is the primary task for solving urban traffic pressure. Bus priority is an important development strategy for urban traffic construction. This is both a requirement for the mass travel and for low-carbon cities. The riding experience of public transportation means is improved, and the method is an important link of public transportation construction. Besides the powerful construction of the public transport hardware system, the riding comfort of passengers is increasingly valued by traffic decision makers and travelers, and the public transport travel crowding degree becomes a hot spot of most concern for people. Therefore, during the operation of the passenger car, if the operation company, the passenger car driver and the passenger can acquire the number of passengers of the vehicle, the density of the passengers of the vehicle and the current boarding and disembarking conditions of the passengers in real time, and dynamically acquire the real-time full load rate in the bus, a very effective communication channel can be established among the operation company of the passenger car, the passenger car driver and the passenger, important bus operation information is provided for the bus traveler and the operation manager, the passenger is helped to solve the problem of poor experience of the trip difficulty, the service quality of the passenger is prompted, and the riding satisfaction of the passenger is improved.
In recent years, with the development of computer graphic analysis technology and deep learning technology, the technology of performing real-time online statistics on people flow and passenger flow by utilizing video graphic analysis has been advanced, and particularly, a neural network system based on convolution operation, namely a Convolutional Neural Network (CNN), has been rapidly developed, and is applied to the fields of computer vision, natural language processing and the like, and the technology of installing image acquisition equipment such as a camera and the like near a door of a public transportation means to acquire video and images of getting on and off passengers and detecting the flow of the passengers by utilizing the video graphic technology has been realized.
For example, patent document 1 (CN 104156983 a) provides a method for counting bus passenger flow based on video graphics processing, which uses a video collected by a camera installed in the vertical direction of the roof of a bus to process, uses the combination of a human body detection algorithm of the HOG feature of the head and shoulder of a human body and a tracking algorithm based on SURF features to detect and track passengers on the head and the lower part of the bus, and realizes counting of bus passenger flow according to the movement direction of the passengers, thereby solving the technical problems that the passenger detection accuracy is affected under the conditions of mutual shielding and obvious light change of the passengers, and the bus passenger flow can still be counted accurately under the conditions that the color of the human body clothes is similar to the color of the Beijing, the object similar to the human body contour is in Beijing, the weather or the illumination change is generated, and the like, so as to reduce the false detection rate of the passengers and simultaneously reduce the bus passenger flow counting cost. However, in the technical solution of patent document 1, a head-shoulder HOG feature human detection algorithm and a SURF feature tracking algorithm are required to be combined, but when a strong classifier is obtained in target detection training, window traversal is required to be performed on a foreground and then target passenger judgment is performed, SURF feature extraction is performed on adjacent frame videos through the SURF algorithm, the two algorithms are long in time, if errors exist in early traversal judgment, target passenger samples are lost, the accuracy of the following SURF algorithm cannot be improved, the accuracy of the whole algorithm is low, time consumption is high, and real-time online calculation is not facilitated.
For another example, patent document 2 (CN 108573497 a) provides an apparatus and method for dynamically counting passenger flows based on video analysis, which counts passenger flows from videos captured by an image pickup element, comprising: a region-of-interest setting step of setting a predetermined region of interest in an image of the video; a motion information calculation step of calculating motion information for all pixels of the region of interest of each frame image in the video; a foreground region judgment step of judging a foreground region composed of foreground pixels moving therein within the region of interest of each frame image of the video, based on the motion information calculated by the motion information calculation step; a texture calculation step of calculating texture information of the foreground region in the region of interest of each frame image of the video; a motion information correction step of correcting the motion information by using the texture information to obtain corrected motion information; and a passenger flow statistics step of calculating the displacement sum of the foreground pixels in the concerned area of all frame images according to the correction motion information for the video in a given period, and obtaining the passenger flow by counting the passenger flow based on the displacement sum. The above patent document 2 can perform people flow statistics without detection and tracking, and improve the accuracy of passenger flow statistics based on temporal and spatial characteristics. However, in the solution of patent document 2, it is necessary to calculate the velocity information of each pixel by the motion calculating unit, determine the foreground region formed by the motion foreground pixels by the foreground region determining unit, and calculate the texture information for the foreground region, but the whole calculation process is complicated without detection and tracking, and the problem of matching delay of the image stream due to the time sequence extension increases errors, and the calculation timeliness is affected.
Disclosure of Invention
Therefore, it is necessary to provide a public transportation passenger flow statistics device and method based on deep learning, which have more reasonable collected image samples, higher operation efficiency and higher detection accuracy, and can realize real-time online detection of passenger flow.
In order to achieve the above object, the present invention provides a public transportation passenger flow statistics device based on deep learning, which performs statistics on public transportation passenger flows according to videos shot by an external camera, including:
a reference image acquisition unit that acquires a reference video and an image of a passenger from the camera;
and a region of interest setting unit for setting an ROI region, which is one image region of interest, from the reference video and the image.
A video image decoding unit for acquiring the video and image of the passenger from the camera, decoding the video and image to generate a bitmap,
a subsampling unit for subsampling the bitmap according to the region of interest of the image;
an image recognition unit for performing image recognition calculation on the subsampled bitmap by using YOLOV3 algorithm to obtain a person region array PV in the ROI region 1 Then secondary filtering and position information conversion are carried out according to the similarity condition, and a secondary personnel area queue PV is obtained cur ，
Object tracking unit for aligning secondary personnel area array PV cur Filtering the regional object to obtain a personnel regional object tracking queue PV t The method comprises the steps of carrying out a first treatment on the surface of the If personnel area object tracking queue PV t Empty, then use secondary personnel area queue PV cur Joining to personnel area object tracking queue PV t In (a) and (b); if personnel area object tracking queue PV t If not, entering a data association part for processing;
a data association matching unit for queuing the secondary personnel area PV cur And personnel area object tracking queue PV t Performing data association, and if the association matching is successful, targeting the personnel areaTracking queue PV t The attributes of the medium object are used for the associated secondary personnel area queue PV cur The attribute of the target is updated and replaced, if the association matching fails, a secondary personnel area queue PV cur Joining as a new target to a personnel area object tracking queue PV t In the video image region, a virtual irregular closed polygon detection region R is set det And keep track of personnel area objects in the queue PV t Each element in the polygon sample is decomposed by geometric relationship, and R is calculated det And PV (photovoltaic) t The current geometry queue VG for each element of (1) cur ，
Geometric relationship comparison part, current geometric relationship queue VG cur Comparing with the old geometric relation queue VG, and ignoring the tracking object when the geometric relation between the tracking object and the detection area is consistent; when the geometric relationship of the tracking object and the detection area is inconsistent, the current state is further judged to enter or exit.
The invention also provides a public transport passenger flow statistical method based on deep learning, which comprises the following steps:
obtaining a reference image, namely obtaining a reference video and an image of a passenger from the camera;
video image decoding, capturing the video and images of the passenger from the camera, decoding, generating a bitmap,
setting a region of interest, namely, a region of interest (ROI) of an image is set from the reference video and the image,
sampling the bitmap according to the image interest area;
image recognition, namely performing image recognition calculation on the bit map after the secondary sampling to obtain a personnel area array PV in the ROI area 1 Then secondary filtering and position information conversion are carried out to obtain a secondary personnel area queue PV cur Determining a secondary personnel area queue PV cur If not, entering an object tracking step, and if not, ending statistics;
object tracking, if secondary personnel area queue PV cu Not empty, for secondary personnel area queues PV cur Filtering to obtain a personnel area object tracking queue PV t The method comprises the steps of carrying out a first treatment on the surface of the If personnel area object tracking queue PV t Empty, then use secondary personnel area queue PV cur Joining to personnel area object tracking queue PV t In (a) and (b); if personnel area object tracking queue PV t If not, entering a data association step;
data association matching, then the secondary personnel area queue PV cur And personnel area object tracking queue PV t Performing data association, and if the association matching is successful, tracking the personnel area object in the queue PV t The attributes of the medium object are used for the associated secondary personnel area queue PV cur The attribute of the target is updated and replaced, if the association matching fails, a secondary personnel area queue PV cur Joining as a new target to a personnel area object tracking queue PV t In the video image region, a virtual irregular closed polygon detection region R is set det And keep track of personnel area objects in the queue PV t Each element in the polygon model is subjected to geometric relationship decomposition, and R is calculated det And PV (photovoltaic) t The current geometry queue VG for each element of (1) cur ，
Geometric relationship comparison, current geometric relationship queue VG cur Comparing with the old geometric relation queue VG, and when the geometric relation between the tracked object and the detection area is consistent, ignoring the tracked object and ending statistics; when the geometric relationship of the tracking object and the detection area is inconsistent, the current state is further judged to enter or exit.
Compared with the prior art, the technical scheme has the following beneficial effects:
the method adopts the region of interest to divide the acquired image video, carries out secondary sampling on the image video, carries out image recognition, object tracking, data association matching, detection region setting and geometric relationship comparison, judges the people flow on line in real time, has reasonable flow design, high operation efficiency and high accuracy, and can also improve the accuracy by adopting secondary sampling.
Drawings
FIG. 1 is a schematic diagram of a bus of a public transportation passenger flow statistics device according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a public transportation passenger flow statistics device according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating the setting of the ROI area according to the embodiment of the present invention;
FIG. 4 is a first schematic diagram of a data correlation algorithm according to an embodiment of the present invention;
FIG. 5 is a second schematic diagram of a data correlation algorithm according to an embodiment of the present invention;
FIG. 6 is a third diagram of a data correlation algorithm according to an embodiment of the present invention;
FIG. 7 is a fourth diagram of a data correlation algorithm according to an embodiment of the present invention;
fig. 8 is a fifth schematic diagram of a data association algorithm according to an embodiment of the present invention.
Reference numerals illustrate:
101. the camera is used for the image processing of the image,
102. the camera is used for the image processing of the image,
200. a passenger car.
Detailed Description
In order to describe the technical content, constructional features, achieved objects and effects of the technical solution in detail, the following description is made in connection with the specific embodiments in conjunction with the accompanying drawings.
The present embodiment is exemplified by the case where the public transportation passenger flow statistics apparatus of the present invention is used for statistics of passenger flow of vehicles such as passenger cars, but the application of the passenger flow statistics apparatus of the present invention is not limited thereto, and may be used for traffic statistics in airports, stations, wharfs, banks, dangerous places, work areas.
Referring to fig. 1, a passenger car 200 is shown with the deep learning based mass transit flow statistics device 100 of the present invention installed. The inner top of the passenger car 200 is provided with 2 cameras 101, 102, the cameras are arranged at the top of a passenger car boarding area, a hemispherical ceiling mode is adopted, and a video area covers the passenger car boarding area, the periphery of the passenger boarding area and the door part. And cameras 101, 102 are respectively disposed above the front door and the rear face of the passenger car and are used for capturing traffic videos in a downward direction in a top view, in other embodiments, too many cameras may be disposed and may also be disposed at a position inside the passenger car near the door to capture traffic entering and exiting the door. The setting angle of the camera is not particularly limited, and shooting is preferably performed at an angle opposite to the traffic direction of the traffic at the entrance and exit of the vehicle door.
Fig. 2 is a schematic diagram of the device structure of the present embodiment, which is a public transportation passenger flow statistics device of the present embodiment, and statistics is performed on public transportation passenger flows according to videos shot by an external camera, including:
a reference image acquisition unit that acquires a reference video and an image of a passenger (getting on or off) from the camera; the camera mounted on the top of the passenger car is used for continuous snapshot, and representative grabs of each scene are selected from a large number of pictures according to different scenes aiming at different angles, different time periods, different climatic environments and different crowds to form a personnel sample file.
And a region of interest setting unit for setting a region of interest (ROI) which is one image region of interest from the reference video and image, with reference to fig. 3, which illustrates a schematic ROI region setting diagram. The ROI areas are trapezoidal areas in the graph, and occupy 60% -80% of the pixels in the whole image respectively. The ROI (region of interest) area is saved to the local system.
A video image decoding unit which acquires a video and an image of a passenger from a camera, decodes the video and the image, generates a bitmap, and subsamples the bitmap according to an image region of interest; the second time of adoption is to reduce the resolution of the existing image, adopt the lossy interlacing sampling, and finally the target resolution is 460x460. The video stream is obtained through an API interface or a communication protocol of the camera, the video stream is decoded, the decoded bitmap is subjected to secondary sampling in the bitmap according to the ROI area, the pixel values of the width and the height of the picture subjected to secondary sampling can be divided by 4, the pixels adjacent to the RIO area can be complemented by the insufficient division by 4, and the calculation formula of the width and the height of the secondary sampling is as follows:
Wc＝W roi +(W rio ％4)；
an image recognition unit for performing image recognition calculation on the subsampled bitmap using YOLOV3 algorithmAcquiring person region enqueues PV within a ROI region 1 Performing secondary filtering according to the similarity condition, and reserving when the similarity value identified by the object reaches a certain set value SIM; a value below this SIM discards the identification object. Then the position information is converted to obtain a secondary personnel area array PV cur ，
Object tracking unit for tracking the area of the secondary person in the queue PV cu Not empty, for secondary personnel area queues PV cur Performing Kalman filtering to obtain a personnel area object tracking queue PV t If personnel area object tracking queue PV t Empty, then use secondary personnel area queue PV cur Queue addition to personnel area object tracking queue PV t In the queue; if the secondary personnel area queue PV cu Not empty, for secondary personnel area queues PV cur And (3) predicting the subsequent area of the personnel detection area by using a Kalman filtering algorithm, optimally estimating the system state by inputting the personnel detection area (x, y, w, h), and observing the influence of noise and interference in the system. Obtaining personnel area object tracking queues PV t If personnel area object tracking queue PV t Empty, then use secondary personnel area queue PV cur Queue addition to personnel area object tracking queue PV t In the queue; by copy-by-copy pattern from the first member of the queue to the last member
A data association matching unit for tracking the queue PV of personnel area object t If not empty, the secondary personnel area array PV is provided cur And personnel area object tracking queue PV t Employing rectangular frame data (x, y, w, h) associative matching to borrow a hungarian algorithm, wherein a model of the algorithm associative matching is a value f of an area of an intersection divided by a total area of two matching elements as a reference model of the two matching elements iou ，f iou The larger the value of (2), the higher the matching degree of the two elements; the method of association matching is that the maximum matching number and the minimum coverage number of the bipartite graph are obtained by continuously searching the augmentation path which has already determined the original matching in the queues PVcur set member Xn (x, y, w, h) and PVt set member Yn (x, y, w, h) as bipartite graphs. If the association matching is successful, thenPersonnel area object tracking queue PV t The attributes of the medium object are used for the associated secondary personnel area queue PV cur Detecting the attribute update substitution of the target in the queue, then tracking the Xn (x, y) area object in the queue PV t The attribute of the target is replaced by the attribute update of the target in the associated detection queue, if the associated matching fails, xn (x, y) is added into the personnel area object tracking queue PV as a new target t ，
A detection region setting unit for setting a virtual irregular closed polygon detection region R in the video image region det And keep track of personnel area objects in the queue PV t Each element in the (1) is subjected to geometric relation (GPC algorithm) to decompose the polygon sample, and R is calculated det And PV (photovoltaic) t The current geometry queue VG for each element of (1) cur ，
Geometric relationship comparison part, current geometric relationship queue VG cur Comparing with the old geometric relation queue VG, copying queue elements in VGcur into the VG queue after the first queue VG is empty and VGcur is compared, and ignoring the tracking object when the geometric relation between the tracking object and the detection area is consistent; when the geometric relationship between the tracking object and the detection area is inconsistent, further judging the current state, and if the current state is entering, adding 1 to the personnel count; if the current state is exit, the personnel count is decremented by 1.
The invention also discloses a public transport passenger flow statistical method based on deep learning, which comprises the following steps:
step 1: the camera mounted on the top of the passenger car is used for continuous snapshot, and representative grabs of each scene are selected from a large number of pictures according to different scenes aiming at different angles, different time periods, different climatic environments and different crowds to form a personnel sample file.
Step 2: an ROI (region of interest) area is set in the video image of the camera and stored in a local system.
Step 3: the video stream is obtained through an API interface or a communication protocol of the camera, the video stream is decoded, the decoded bitmap is subjected to secondary sampling in the bitmap according to the ROI area, the pixel values of the width and the height of the picture subjected to secondary sampling can be divided by 4, the insufficient divided by 4 can be complemented by the adjacent pixels of the ROI area, and the calculation formula of the width and the height of the secondary sampling is as follows:
Wc＝Wroi+(Wroi％4)；
step 4: and (3) carrying out image recognition on the subsampled bitmap, adopting an algorithm which is Yolov3, obtaining a personnel area queue PV1 in the ROI area through recognition processing, carrying out secondary filtering and position information conversion according to conditions such as similarity and the like to obtain a secondary personnel area queue PVcur, and entering a step (5) when PVcur is not empty.
Step 5: carrying out Kalman filtering on the PVcur regional object to obtain a personnel regional object tracking queue PVt, and if the tracking queue is empty, adding the PVcur queue into the PVt queue; if the trace queue is not empty, go to step 6.
Step 6: and carrying out data association on the PVcur queue and the PVt queue through a Hungary algorithm, if the association matching is successful, replacing the attribute of the target in the tracking queue with the attribute of the target in the associated detection queue, if the association matching is failed, adding the target as a new target into the PVt of the tracking queue, distinguishing different tracking objects by each tracking object in the queue, generating a new ID number by the new tracking object, adding the ID number and corresponding object information into the PVt of the tracking queue, and deleting the ID number and the corresponding object in the tracking queue.
The specific calculation process comprises the following steps: PVcur queues are x1, x2, x3, x4, x5, x6 in FIG. 4, PVt queues are y1, y2, y3, y4, y5, y6 in FIG. 4, and the upper graph is a two-part graph, we discuss from the left of the upper graph, and our goal is to find the pairing for the most points in x as possible. The maximum matches are each other, and if we find the most corresponding points in Y for X, it is also impossible to have more points in Y matched.
An edge, (x 1, y 1) is arbitrarily chosen, and the initial match is constructed, with the result that the paired edges are marked with bold lines as in fig. 5. A match is added to x2 as the (x 2, y 2) edge of fig. 6. A match M is formed having (x 1, y 1), (x 2, y 2) two sides. An edge is matched for x3, as in fig. 7. A path P (x 3, y1, x1, y2, x2, y 5) is obtained, P being an augmented path of M. Splitting the paired points in M apart and recombining yields a larger match M1 with (x 3, y 1), (x 1, y 2), (x 2, y 5) three edges. Likewise, x4, x5 are added sequentially, eventually resulting in the maximum match M2 of FIG. 8.
By the method, all elements in the PVcur queue and the PVt queue can be associated.
Step 7: in the video image area, a virtual irregular closed polygon detection area Rdet is set. Performing GPC algorithm decomposition on the detection area and each element of the tracking queue PVt obtained in the step 6 to obtain a polygon sample, calculating a current geometric relationship queue VGcur of each element of Rdet and PVt, comparing the current geometric relationship queue with a old geometric relationship queue VG, and ignoring the tracking object when the geometric relationship of the tracking object-detection area is consistent; when the geometric relationship between the tracking object and the detection area is inconsistent, further judging the current state, and if the current state is entering, adding 1 to the personnel count; if the current state is exit, the personnel count is decremented by 1.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the statement "comprising … …" or "comprising … …" does not exclude the presence of additional elements in a process, method, article or terminal device comprising the element. Further, herein, "greater than," "less than," "exceeding," and the like are understood to not include the present number; "above", "below", "within" and the like are understood to include this number.
It will be appreciated by those skilled in the art that the various embodiments described above may be provided as methods, apparatus, or computer program products. These embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. All or part of the steps in the methods according to the above embodiments may be implemented by a program for instructing related hardware, and the program may be stored in a storage medium readable by a computer device, for performing all or part of the steps in the methods according to the above embodiments. The computer device includes, but is not limited to: personal computers, servers, general purpose computers, special purpose computers, network devices, embedded devices, programmable devices, intelligent mobile terminals, intelligent home devices, wearable intelligent devices, vehicle-mounted intelligent devices and the like; the storage medium includes, but is not limited to: RAM, ROM, magnetic disk, magnetic tape, optical disk, flash memory, usb disk, removable hard disk, memory card, memory stick, web server storage, web cloud storage, etc.
The embodiments described above are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a computer device to produce a machine, such that the instructions, which execute via the processor of the computer device, create means for implementing the functions specified in the flowchart block or blocks and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer device-readable memory that can direct a computer device to function in a particular manner, such that the instructions stored in the computer device-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer apparatus to cause a series of operational steps to be performed on the computer apparatus to produce a computer implemented process such that the instructions which execute on the computer apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the embodiments have been described above, other variations and modifications will occur to those skilled in the art once the basic inventive concepts are known, and it is therefore intended that the foregoing description and drawings illustrate only embodiments of the invention and not limit the scope of the invention, and it is therefore intended that the invention not be limited to the specific embodiments described, but that the invention may be practiced with their equivalent structures or with their equivalent processes or with their use directly or indirectly in other related fields.
Claims (10)
1. Public transportation passenger flow statistics device based on deep learning carries out statistics to public transportation passenger flow according to the video that outside camera took, its characterized in that: comprising the following steps:
a reference image acquisition unit that acquires a reference video and an image of a passenger from the camera;
a video image decoding unit for acquiring the video and image of the passenger from the camera, decoding the video and image to generate a bitmap,
a region of interest setting unit for setting a region of interest (ROI) which is an image region of interest from the bitmap,
a subsampling unit for subsampling the bitmap according to the region of interest of the image;
an image recognition unit for performing image recognition calculation on the subsampled bitmap to obtain a person region array PV in the ROI region 1 Then secondary filtering and position information conversion are carried out to obtain a secondary personnel area queue PV cur Determining a secondary personnel area queue PV cur If not, entering the object tracking part for processing, and if not, ending statistics;
an object tracking section for tracking the object of the object,for secondary personnel area queues PV cur Filtering the regional object to obtain a personnel regional object tracking queue PV t The method comprises the steps of carrying out a first treatment on the surface of the If personnel area object tracking queue PV t Empty, then use secondary personnel area queue PV cur Joining to personnel area object tracking queue PV t In (a) and (b); if personnel area object tracking queue PV t If not, entering a data association part for processing;
a data association matching unit for queuing the secondary personnel area PV cur And personnel area object tracking queue PV t Performing data association, and if the association matching is successful, tracking the personnel area object in the queue PV t The attributes of the medium object are used for the associated secondary personnel area queue PV cur The attribute of the target is updated and replaced, if the association matching fails, a secondary personnel area queue PV cur Joining as a new target to a personnel area object tracking queue PV t In the video image region, a virtual irregular closed polygon detection region R is set det And keep track of personnel area objects in the queue PV t Each element in the polygon model is subjected to geometric relationship decomposition, and R is calculated det And PV (photovoltaic) t The current geometry queue VG for each element of (1) cur ，
Geometric relationship comparison part, current geometric relationship queue VG cur Comparing with the old geometric relation queue VG, and when the geometric relation between the tracked object and the detection area is consistent, ignoring the tracked object and ending statistics; when the geometric relationship of the tracking object and the detection area is inconsistent, the current state is further judged to enter or exit.
2. A deep learning based mass transit passenger flow statistics device as claimed in claim 1, wherein: the image recognition part performs image recognition calculation on the bit map after the secondary sampling by adopting a YOLOV3 algorithm;
then secondary filtering and position information conversion are carried out according to the similarity condition, and a secondary personnel area queue PV is obtained cur 。
3. Root of Chinese characterA deep learning based mass transit passenger flow statistics device as claimed in claim 1, wherein: for secondary personnel area queues PV cur Kalman filtering is carried out on the regional object of the personnel regional object tracking queue PV t 。
4. A deep learning based mass transit passenger flow statistics device as claimed in claim 1, wherein: the secondary personnel area queue PV cur And personnel area object tracking queue PV t And adopting a Hungary algorithm to perform data association.
5. A deep learning based mass transit passenger flow statistics device as claimed in claim 1, wherein: current geometry queue VG cur Comparing with the old geometric relationship queue VG, if the geometric relationship between the tracking object and the detection area is inconsistent, further judging the current state, and if the current state is entered, adding 1 to the personnel count; if the current state is exit, the personnel count is decremented by 1.
6. A public transportation passenger flow statistical method based on deep learning is characterized in that: the method comprises the following steps:
acquiring a reference image, namely acquiring a reference video and an image of a passenger from a camera;
video image decoding, capturing the video and images of the passenger from the camera, decoding, generating a bitmap,
region of interest setting, setting an image region of interest, namely a ROI region, from the bitmap,
sampling the bitmap according to the image interest area;
image recognition, namely performing image recognition calculation on the bit map after the secondary sampling to obtain a personnel area array PV in the ROI area 1 Then secondary filtering and position information conversion are carried out to obtain a secondary personnel area queue PV cur Determining a secondary personnel area queue PV cur If not, entering an object tracking step, if notEnding statistics if the result is empty;
object tracking, if secondary personnel area queue PV cur Not empty, for secondary personnel area queues PV cur Filtering the regional object to obtain a personnel regional object tracking queue PV t The method comprises the steps of carrying out a first treatment on the surface of the If personnel area object tracking queue PV t Empty, then use secondary personnel area queue PV cur Joining to personnel area object tracking queue PV t In (a) and (b); if personnel area object tracking queue PV t If not, entering a data association step;
data association matching, then the secondary personnel area queue PV cur And personnel area object tracking queue PV t Performing data association, and if the association matching is successful, tracking the personnel area object in the queue PV t The attributes of the medium object are used for the associated secondary personnel area queue PV cur The attribute of the target is updated and replaced, if the association matching fails, a secondary personnel area queue PV cur Joining as a new target to a personnel area object tracking queue PV t In the video image region, a virtual irregular closed polygon detection region R is set det And keep track of personnel area objects in the queue PV t Each element in the polygon model is subjected to geometric relationship decomposition, and R is calculated det And PV (photovoltaic) t The current geometry queue VG for each element of (1) cur ，
Geometric relationship comparison, current geometric relationship queue VG cur Comparing with the old geometric relation queue VG, and when the geometric relation between the tracked object and the detection area is consistent, ignoring the tracked object and ending statistics; when the geometric relationship of the tracking object and the detection area is inconsistent, the current state is further judged to enter or exit.
7. The deep learning-based public transportation passenger flow statistics method as described in claim 6, wherein: the image recognition step adopts a YOLOV3 algorithm to perform image recognition calculation on the bit map after the secondary sampling;
then secondary filtering and position information conversion are carried out according to the similarity condition, and secondary personnel are obtainedZone queues PV cur 。
8. The deep learning-based public transportation passenger flow statistics method as described in claim 6, wherein:
for secondary personnel area queues PV cur Kalman filtering is carried out on the regional object of the personnel regional object tracking queue PV t 。
9. The deep learning-based public transportation passenger flow statistics method as described in claim 6, wherein:
the secondary personnel area queue PV cur And personnel area object tracking queue PV t And adopting a Hungary algorithm to perform data association.
10. The deep learning-based public transportation passenger flow statistics method as described in claim 6, wherein:
current geometry queue VG cur Comparing with the old geometric relation queue VG, if the geometric relation of the tracking object-detection area is inconsistent, further judging the current state, and if the current state is entering, adding 1 to the personnel count; if the current state is exit, the personnel count is decremented by 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011560622.1A CN112580633B (en) | 2020-12-25 | 2020-12-25 | Public transport passenger flow statistics device and method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011560622.1A CN112580633B (en) | 2020-12-25 | 2020-12-25 | Public transport passenger flow statistics device and method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112580633A CN112580633A (en) | 2021-03-30 |
| CN112580633B true CN112580633B (en) | 2024-03-01 |
Family
ID=75140529
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011560622.1A Active CN112580633B (en) | 2020-12-25 | 2020-12-25 | Public transport passenger flow statistics device and method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112580633B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114943955B (en) * | 2022-07-25 | 2022-11-01 | 山东广通汽车科技股份有限公司 | Automatic unloading control method for semitrailer |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008092393A1 (en) * | 2007-01-25 | 2008-08-07 | Shanghai Yao Wei Industry Co, Ltd | Method of moving target tracking and number accounting |
| EP2700054A1 (en) * | 2012-06-12 | 2014-02-26 | Institute of Electronics and Computer Science | System and method for video-based vehicle detection |
| CN103871082A (en) * | 2014-03-31 | 2014-06-18 | 百年金海科技有限公司 | Method for counting people stream based on security and protection video image |
| CN104156983A (en) * | 2014-08-05 | 2014-11-19 | 天津大学 | Public transport passenger flow statistical method based on video image processing |
| WO2017156772A1 (en) * | 2016-03-18 | 2017-09-21 | 深圳大学 | Method of computing passenger crowdedness and system applying same |
| CN107239762A (en) * | 2017-06-06 | 2017-10-10 | 电子科技大学 | Patronage statistical method in a kind of bus of view-based access control model |
| CN110633671A (en) * | 2019-09-16 | 2019-12-31 | 天津通卡智能网络科技股份有限公司 | Bus passenger flow real-time statistical method based on depth image |
-
2020
- 2020-12-25 CN CN202011560622.1A patent/CN112580633B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008092393A1 (en) * | 2007-01-25 | 2008-08-07 | Shanghai Yao Wei Industry Co, Ltd | Method of moving target tracking and number accounting |
| EP2700054A1 (en) * | 2012-06-12 | 2014-02-26 | Institute of Electronics and Computer Science | System and method for video-based vehicle detection |
| CN103871082A (en) * | 2014-03-31 | 2014-06-18 | 百年金海科技有限公司 | Method for counting people stream based on security and protection video image |
| CN104156983A (en) * | 2014-08-05 | 2014-11-19 | 天津大学 | Public transport passenger flow statistical method based on video image processing |
| WO2017156772A1 (en) * | 2016-03-18 | 2017-09-21 | 深圳大学 | Method of computing passenger crowdedness and system applying same |
| CN107239762A (en) * | 2017-06-06 | 2017-10-10 | 电子科技大学 | Patronage statistical method in a kind of bus of view-based access control model |
| CN110633671A (en) * | 2019-09-16 | 2019-12-31 | 天津通卡智能网络科技股份有限公司 | Bus passenger flow real-time statistical method based on depth image |
Non-Patent Citations (3)
| Title |
|---|
| An automatic statistical method of bus passenger flow based on video;Zhao Xiangmo，等;Computer Engineering;20151231;第41卷(第06期);第136-142页 * |
| 基于OpenCV的公交客流计数方法;张震，等;现代电子技术;20170115(第02期);第16-20页 * |
| 基于智能监控视频的人流量统计;牛秋月，等;电子技术与软件工程;20181231(第04期);第64-66页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112580633A (en) | 2021-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109819208B (en) | Intensive population security monitoring management method based on artificial intelligence dynamic monitoring | |
| WO2020173226A1 (en) | Spatial-temporal behavior detection method | |
| CN105844234B (en) | Method and equipment for counting people based on head and shoulder detection | |
| US10735694B2 (en) | System and method for activity monitoring using video data | |
| US9652851B2 (en) | Side window detection in near-infrared images utilizing machine learning | |
| CN107977646B (en) | Partition delivery detection method | |
| CN111144247A (en) | Escalator passenger reverse-running detection method based on deep learning | |
| CN103986910A (en) | Method and system for passenger flow statistics based on cameras with intelligent analysis function | |
| CN103824070A (en) | Rapid pedestrian detection method based on computer vision | |
| CN107483894B (en) | The high-speed rail station video monitoring system of realization passenger transportation management is judged based on scene | |
| CN104159088B (en) | A kind of long-distance intelligent vehicle monitoring system and method | |
| CN111008574A (en) | Key person track analysis method based on body shape recognition technology | |
| CN113743260A (en) | Pedestrian tracking method under dense pedestrian flow condition of subway platform | |
| Harville | Stereo person tracking with short and long term plan-view appearance models of shape and color | |
| CN112580633B (en) | Public transport passenger flow statistics device and method based on deep learning | |
| CN113920585A (en) | Behavior recognition method and device, equipment and storage medium | |
| Chen et al. | An intelligent virtual fence security system for the detection of people invading | |
| Liu et al. | A technology for automatically counting bus passenger based on YOLOv2 and MIL algorithm | |
| Yang et al. | SPDNet: a real-time passenger detection method based on attention mechanism in subway station scenes | |
| CN116311166A (en) | Traffic obstacle recognition method and device and electronic equipment | |
| CN109726750A (en) | A kind of passenger falls down detection device and its detection method and passenger conveying appliance | |
| CN111062294B (en) | Passenger flow queuing time detection method, device and system | |
| CN114677608A (en) | Identity feature generation method, device and storage medium | |
| CN113223081A (en) | High-altitude parabolic detection method and system based on background modeling and deep learning | |
| CN114612850A (en) | High altitude parabola detection method, electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |