CN111062337A - People flow direction detection method and device, storage medium and electronic equipment - Google Patents
People flow direction detection method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN111062337A CN111062337A CN201911318979.6A CN201911318979A CN111062337A CN 111062337 A CN111062337 A CN 111062337A CN 201911318979 A CN201911318979 A CN 201911318979A CN 111062337 A CN111062337 A CN 111062337A
- Authority
- CN
- China
- Prior art keywords
- frame image
- crowd density
- flow direction
- people flow
- current frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
- G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
Abstract
The disclosure provides a people flow direction detection method, a people flow direction detection device, a storage medium and electronic equipment, and relates to the technical field of image processing. The people flow direction detection method comprises the following steps: acquiring a video to be detected, and extracting a current frame image and at least one historical frame image of the video to be detected; respectively carrying out crowd density estimation on the current frame image and the historical frame image to obtain crowd density maps respectively corresponding to the current frame image and the historical frame image; performing optical flow detection by using crowd density graphs respectively corresponding to the current frame image and the historical frame image to determine crowd density change parameters; and determining the direction of the people flow based on the crowd density change parameter. This openly can improve crowd's flow direction detection's degree of accuracy.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to a people flow direction detection method, a people flow direction detection apparatus, a storage medium, and an electronic device.
Background
With the rapid development of image processing technology and the continuous expansion of the application range thereof, the image processing technology can be applied to the field of crowd statistical analysis. The detection of the crowd flowing direction is used as an important branch of crowd statistical analysis, and has important application value in multiple aspects of scenic spot management, current limiting control, safety prevention and the like.
At present, in the process of detecting the flow direction of people, due to the limitation of the shooting object and the shooting environment, the problem of low accuracy of the detection of the flow direction of people may occur.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The present disclosure aims to provide a people flow direction detection method, a people flow direction detection device, a storage medium, and an electronic apparatus, so as to overcome the problem of low accuracy in detecting the flow direction of people at least to a certain extent.
According to a first aspect of the present disclosure, there is provided a people flow direction detection method, including: acquiring a video to be detected, and extracting a current frame image and at least one historical frame image of the video to be detected; respectively carrying out crowd density estimation on the current frame image and the historical frame image to obtain crowd density maps respectively corresponding to the current frame image and the historical frame image; performing optical flow detection by using crowd density graphs respectively corresponding to the current frame image and the historical frame image to determine crowd density change parameters; and determining the direction of the people flow based on the crowd density change parameter.
According to a second aspect of the present disclosure, there is provided a human flow direction detecting apparatus including an image extracting module, a density estimating module, an optical flow detecting module, and a human flow direction determining module.
Specifically, the image extraction module is used for acquiring a video to be detected, and extracting a current frame image and at least one historical frame image of the video to be detected; the density estimation module is used for respectively carrying out crowd density estimation on the current frame image and the historical frame image to obtain crowd density maps respectively corresponding to the current frame image and the historical frame image; the optical flow detection module is used for carrying out optical flow detection by utilizing crowd density graphs respectively corresponding to the current frame image and the historical frame image to determine crowd density change parameters; the people flow direction determining module is used for determining the people flow direction based on the crowd density change parameters.
Optionally, the image extraction module may be configured to perform: and extracting a frame image which is a preset time away from the current frame image in the video to be detected as a historical frame image.
Optionally, the image extraction module may be further configured to perform: and extracting each frame image in a preset time period from the current frame image in the video to be detected as a historical frame image. In this case, the people flow direction determination module may be configured to perform: and determining the direction of the people flow by using the crowd density change parameter between every two adjacent frame images in a preset time period.
Optionally, the image extraction module may be configured to perform: performing image quality evaluation on each frame image in a preset time period from the current frame image in the video to be detected; and eliminating the images with the scores smaller than the score threshold after image quality evaluation, so that the people flow direction determining module determines the people flow direction by using the people density change parameters between every two adjacent frames of images after the images with the scores smaller than the score threshold are eliminated.
Optionally, the optical flow detection module may be configured to perform: determining the pixel value variation of each pixel point in the crowd density map by utilizing the crowd density map respectively corresponding to the current frame image and the historical frame image; and determining the crowd density change parameter of each pixel point according to the pixel value variable quantity of each pixel point in the crowd density graph.
Optionally, the people flow direction determining module may include a parameter clustering unit and a people flow direction determining unit.
Specifically, the parameter clustering unit is used for clustering each pixel point according to the crowd density variation parameter; and the people flow direction determining unit is used for determining the people flow direction according to the clustering result.
Optionally, the crowd density variation parameter comprises a crowd density variation size and a crowd density variation direction, in which case the parameter clustering unit may be configured to perform: determining pixel points with the crowd density change size larger than the density change threshold value as pixel points to be clustered; and clustering the pixels to be clustered according to the crowd density change direction of the pixels to be clustered and the distance between the pixels to be clustered.
Optionally, the people flow direction determination unit may be configured to perform: determining the number of pixels contained in each cluster after clustering; and eliminating clusters with the number of pixel points smaller than the preset number, and determining the direction of the people flow by using the rest clusters in the clustering result.
According to a third aspect of the present disclosure, there is provided a storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described people flow direction detection method.
According to a fourth aspect of the present disclosure, there is provided an electronic device comprising: a processor; and a memory for storing executable instructions for the processor; wherein the processor is configured to execute the people flow direction detection method via executing the executable instructions.
In the technical scheme provided by some embodiments of the present disclosure, a current frame image and at least one historical frame image of a video to be detected are extracted, crowd density estimation is performed on the extracted images to obtain crowd density maps of the images, optical flow detection is performed by using the obtained crowd density maps to determine crowd density change parameters, and a crowd direction is determined based on the crowd density change parameters. On one hand, the crowd flow direction is analyzed by utilizing the crowd density graph, so that the scheme disclosed by the invention can be applied to scenes with more people, and the problem of poor portrait tracking effect caused by directly analyzing the image can be avoided; on the other hand, compared with a scheme of performing feature point matching on an original image in some technologies, the scheme of calculating the optical flow based on the density map disclosed by the invention has the advantages that only human information exists on the density map when optical flow processing is performed, so that environmental interference such as background and illumination can be eliminated, and the accuracy and robustness of people flow direction detection are improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty. In the drawings:
fig. 1 schematically shows a flow chart of a people flow direction detection method according to an exemplary embodiment of the present disclosure;
FIG. 2 shows a schematic diagram of converting a captured image into a population density map;
FIG. 3 shows another schematic diagram of converting a captured image into a population density map;
FIG. 4 is a schematic diagram illustrating the change in crowd density determined by optical flow detection using a crowd density map according to the present disclosure;
FIG. 5 schematically illustrates a flow chart for determining a direction of flow of a person using a crowd density variation parameter according to an exemplary embodiment of the present disclosure;
fig. 6 schematically shows a block diagram of a people flow direction detection apparatus according to an exemplary embodiment of the present disclosure;
FIG. 7 schematically illustrates a block diagram of a people flow direction determination module, according to an exemplary embodiment of the present disclosure;
fig. 8 schematically shows a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the steps. For example, some steps may be decomposed, and some steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
In one embodiment of the present disclosure, the direction of people flow in a video may be determined by detecting and tracking people in the video. However, for scenes with a large number of people or long distance and small scene, the method may have the problems of unstable detection and poor tracking effect.
In another embodiment of the present disclosure, the direction of the stream of people in the video may be determined based on a feature point matching method of the video. However, the method is seriously affected by the shooting environment, and the interference of the video such as illumination, weather, background and the like can cause the instability of the feature points, so that the feature points cannot be well matched with people, and the accuracy is low.
In view of this, in the following exemplary embodiments, a method capable of improving the accuracy of detecting the flow direction of a crowd is provided.
The people flow direction detection method of the present disclosure may be applied in a monitoring scene, in which case, a device performing the following method may be a terminal device connected with a monitoring camera. Specifically, the terminal device and the monitoring camera may be in communication connection in a wired or wireless manner. The terminal devices may include, but are not limited to, personal computers, tablet computers, cell phones, and the like. In addition, the terminal equipment can also refer to a server, and the detection result of the crowd flowing direction is displayed through the display device.
In addition, it should be noted that the people flow direction detection method of the present disclosure may be applied to monitoring scenes such as scenic spot management, current limiting control, security prevention, and the like, and may also be applied to people flow analysis of an area to determine the possibility and the location of a shop, which is not limited by the present disclosure.
Fig. 1 schematically shows a flowchart of a people flow direction detection method of an exemplary embodiment of the present disclosure. Referring to fig. 1, the people flow direction detection method may include the steps of:
s12, acquiring a video to be detected, and extracting a current frame image and at least one historical frame image of the video to be detected.
In some embodiments of the present disclosure, the video to be detected may be a video obtained by shooting a scene in real time. That is, the video to be detected may be a video that is continuously generating images. In this case, the current frame image is an image corresponding to the current time acquired by the camera in real time, and the history frame image may be an image generated before the current time.
In other embodiments of the present disclosure, the video to be detected may also be a video that has been shot and is to be analyzed, in which case, the video to be detected may be a video generated by shooting in advance, or a video acquired from a network or other device, and the source of the video is not limited in the present disclosure. In addition, for the video to be detected which has been shot, the current frame image may be an image corresponding to a video time point at which the user desires to determine the direction of the people stream, and the historical frame image may be a video frame image whose time point is before the current frame image. For example, for a 30-second video, if the user desires to know the direction of the people stream in the video at 10 seconds, the video frame image corresponding to 10 seconds in the video may be determined as the current frame image, and correspondingly, the historical frame image may be the video frame image before 10 seconds.
In an exemplary embodiment of the present disclosure, for a video to be detected, in addition to extracting a current frame image, at least one history frame image may be extracted to perform a process of the following steps.
According to some embodiments of the present disclosure, the terminal device may extract one history frame image to implement the determination process of the direction of the people stream. Specifically, a frame image of the video to be detected, which is a preset time away from the current frame image, may be extracted as the historical frame image extracted in step S12, where the preset time may be predetermined, for example, 3 seconds, 5 seconds, 10 seconds, and the like, and it is easy to see that the determination of the preset time is related to the running duration of the video to be detected, which is not limited by the present disclosure.
For example, if the video to be detected runs for 30 seconds, the current frame image is the video frame image corresponding to the current 30 th second, and if the preset time is 10 seconds, the video frame image corresponding to the video to be detected running for 20 th second can be determined as the historical frame image.
In addition, on one hand, the preset time can be adjusted according to the running time of the video to be detected, for example, if the video to be detected runs for only 7 seconds, an image corresponding to 3 seconds can be used as a historical frame image. On the other hand, if the running time of the video to be detected is less than the minimum preset time, the first frame image of the video to be detected can be determined as the extracted historical frame image.
According to other embodiments of the disclosure, the terminal device may extract more than two historical frame images to implement the determination process of the direction of the people flow. Specifically, each frame image in the video to be detected within a preset time period from the current frame image may be extracted as the plurality of historical frame images extracted in step S12, where the preset time period may be predetermined, for example, 5 seconds, 10 seconds, 30 seconds, 1 minute, and the like.
For example, if the video to be detected runs for 30 seconds, the current frame image is the video frame image corresponding to the current 30 th second, and if the preset time period is 5 seconds, all the video frame images running for 25 th to 30 th seconds of the video to be detected can be determined as the historical frame images.
Similarly, if the preset time period is longer than the running time of the video to be detected, all video frame images of the video to be detected are used as historical frame images.
In addition, in consideration of the possible existence of signal interference, camera shake, and the like, an image of poor quality may exist in the history frame image. For embodiments in which more than two historical frame images are extracted, the terminal device may analyze these historical frame images to reject images of poor image quality.
Specifically, first, image quality evaluation may be performed on each frame image in the video to be detected within a preset time period from the current frame image, for example, the images may be respectively input into a trained image quality evaluation model to determine the score of each image.
Next, the scores of the images may be compared with a score threshold, respectively, and images with scores less than the score threshold may be rejected. That is, in the subsequent processing, the image having the score equal to or greater than the score threshold value is processed as the history frame image.
And S14, performing crowd density estimation on the current frame image and the historical frame image respectively to obtain crowd density maps corresponding to the current frame image and the historical frame image respectively.
After the current frame image is determined, the terminal device can perform crowd density estimation on the current frame image.
According to some embodiments of the present disclosure, it is considered that the higher the crowd density is, the larger the proportion of the crowd in the entire image is, and the more the number of the extracted edge pixel points of the background image is. In view of this, the pixel statistical characteristics of the current frame image may be calculated to determine the edges of the crowd in the image, and further determine the crowd density estimate of the current frame image. In addition, the texture characteristics of the current frame image representing the gray level change or the color change can be directly determined, and the crowd density estimation can be determined by utilizing the texture characteristics. It will be readily appreciated that the pixel statistics and texture information may also be combined to determine a population density estimate for the current frame image. After the estimation of the crowd density is determined, a corresponding crowd density graph can be generated according to the estimation result.
According to other embodiments of the present disclosure, a crowd density map may be generated by performing crowd density estimation on a current frame image based on a crowd density estimation method of a CNN (Convolutional Neural network). Specifically, any one of the MCNN, CP-CNN, CSRNet, ic-CNN and SANet can be adopted to determine the crowd density map of the current frame image. The present disclosure does not limit the parameters of the network nor the training process.
Taking the MCNN network as an example, three rows of convolutional neural networks may be adopted, the current frame image is input into the three rows of convolutional neural networks, and then the outputs of the three rows of convolutional neural networks are combined to obtain a crowd density map corresponding to the current frame image.
Similarly, the crowd density estimation is also performed on the historical frame image determined in step S12 to obtain a corresponding crowd density map, which is not described herein again.
Referring to fig. 2 and 3, corresponding diagrams of video frame images and crowd density maps are shown. Specifically, the crowd density estimation may be performed on the image 21 to obtain a crowd density map 22 corresponding to the image 21; the above population density estimation may be performed on the image 31 to obtain a population density map 32 corresponding to the image 31.
And S16, performing optical flow detection by using the crowd density images respectively corresponding to the current frame image and the historical frame image, and determining crowd density change parameters.
In an exemplary embodiment of the present disclosure, optical flow detection may be understood as a method of calculating motion information of an object (i.e., a person) from frame to frame. The motion information can be represented by the variation of the pixel value of the pixel point.
Specifically, the terminal device may determine a variation of a pixel value of each pixel in the crowd density map by using the crowd density map corresponding to the current frame image and the historical frame image, and determine a crowd density variation parameter of each pixel according to the variation of the pixel value.
The crowd density variation parameters described in the exemplary embodiments of the present disclosure include, but are not limited to, the size of the crowd density variation, which in turn can be understood as the intensity of the density variation, and the direction of the crowd density variation.
In the embodiment of determining the crowd density change parameter by using the current frame image and one historical frame image, the crowd density map of the current frame image can be directly compared with the crowd density map of the historical frame image, the crowd density estimation of the historical frame image is used as a starting point, and the crowd density estimation of the current frame image is used as an end point, so that the crowd density change parameter is obtained.
In the embodiment of determining the crowd density variation parameter by using the current frame image and the two or more historical frame images, the optical flow detection may be implemented by, for example, a gradient-based method, a neural network-based method, or the like, and the optical flow method is not limited by the present disclosure.
The gradient-based method, which may also be referred to as a differentiation method, uses the spatio-temporal differentiation (spatio-temporal gradient function) of the time-varying image gray scale (or a filtered version thereof) to calculate the velocity vector of a pixel. Specifically, the method may include the Horn-Schunck algorithm, the Lucas-Kanade (LK) algorithm, and the like.
It should be noted that, for the embodiment where there are a current frame image and more than two historical frame images, these images may be sequentially formed into an image sequence according to video time, in which case, the process of determining the crowd density variation parameter includes determining the crowd density variation parameter between every two adjacent frame images in the image sequence.
Referring to FIG. 4, a schematic diagram of the change in crowd density determined by optical flow detection using a crowd density map is shown. The crowd density parameter is obtained from the graph, i.e., the whiter the area, the larger the magnitude of the crowd density change, and the arrow indicates the crowd density change direction.
And S18, determining the direction of the people flow based on the crowd density change parameters.
According to some embodiments of the present disclosure, the crowd direction may be determined directly by using the crowd density variation parameter determined in step S16. It should be understood that, in a scene corresponding to a current frame image, there may be crowd density changes in multiple situations, that is, there may be multiple density change directions, and the density changes are different in magnitude, so that multiple people flow directions can be determined.
Noise may be present in view of the density map calculation and the video itself, resulting in a deviation of the determined direction of the flow of people from reality. According to other embodiments of the present disclosure, the crowd density variation parameter determined in step S16 may be denoised, and then the crowd direction may be determined by using the denoising result.
Specifically, the crowd density variation parameter may be denoised in a clustering manner, which is described below with reference to fig. 5.
Referring to fig. 5, in step S502, the terminal device may cluster the population density variation parameters, and may implement clustering of the density variation size and the density variation direction by using, for example, a K-means clustering (K-means clustering) algorithm, and it is easy to see that the clustering process based on the density variation size and the density variation direction is combined with the pixel position information of the image. After clustering, a plurality of clusters can be obtained.
According to some embodiments of the present disclosure, each pixel point can be used as a pixel point to be clustered, and clustering is performed by using a clustering algorithm, so as to obtain a clustering result of each pixel point.
According to other embodiments of the disclosure, the terminal device may determine the pixel points whose population density change size is greater than the density change threshold, and use the pixel points as the pixel points to be clustered. It should be understood that, under the condition that the crowd density change is less than or equal to the density change threshold, it is indicated that the crowd density change degree at the corresponding pixel point is small, and when the direction of the people flow is considered, the pixel points can be ignored, so that on one hand, the complexity of a subsequent clustering algorithm can be reduced, and on the other hand, the problem of attention deviation during analysis due to the fact that the pixel points may interfere with the main direction of the people flow is avoided.
After the pixels to be clustered are determined, clustering can be performed on the pixels to be clustered according to the crowd density direction of the pixels to be clustered and the distance between the pixels to be clustered. The distance described herein refers to an actual physical distance between pixels in an image, and may be represented by a specific length unit, or may be represented by the number of pixels spaced apart, which is not limited in this disclosure.
The following takes two pixel points to be clustered as an example, and explains the distance defined during clustering.
The two pixels to be clustered are respectively marked as a first pixel and a second pixel, and it should be understood that the first pixel and the second pixel are both arbitrarily selected pixels among the pixels to be clustered. Firstly, the density change direction of the first pixel point and the density change direction of the second pixel point are determined, and an included angle between the two directions is calculated. Next, an actual physical distance between the first pixel point and the second pixel point is determined. Subsequently, the weight corresponding to the included angle and the weight corresponding to the physical distance may be obtained, and the weighting result of the included angle and the physical distance is determined as the distance defined during clustering, that is, the clustering process of the pixel points to be clustered is performed by using the calculated distance. The weights may be preconfigured, and the specific values are not limited by the present disclosure.
In step S504, the terminal device may determine the number of pixels included in each cluster; in step S506, clusters with the number of pixels smaller than the predetermined number are determined, and the clusters are removed, where the predetermined number may be determined in advance by a developer based on image resolution and scene needs, and specific values of the present disclosure are not limited.
In step S508, the terminal device may determine the people flow direction by using the clusters remaining after the elimination.
That is, only the results of the large population density variations are retained, and for the determined cases of small population density variations, which may be due to noise or individual conditions of several people, are not considered when specifically determining the direction of the population flow.
After the people flow direction is determined by the method, the safety early warning effect can be achieved by combining the scene and the number of people. For example, in a scene of monitoring a suspension bridge, if a large number of visitors rushing to the suspension bridge are determined, a warning can be sent out through a loudspeaker and security personnel can be informed, so that the occurrence of safety accidents caused by trampling or damage to the suspension bridge can be reduced as much as possible.
It should be noted that although the various steps of the methods of the present disclosure are depicted in the drawings in a particular order, this does not require or imply that these steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.
Further, the present exemplary embodiment also provides a people flow direction detection apparatus.
Fig. 6 schematically shows a block diagram of a human flow direction detection apparatus according to an exemplary embodiment of the present disclosure. Referring to fig. 6, the human flow direction detecting apparatus 6 according to the exemplary embodiment of the present disclosure may include an image extracting module 61, a density estimating module 63, an optical flow detecting module 65, and a human flow direction determining module 67.
Specifically, the image extraction module 61 may be configured to obtain a video to be detected, and extract a current frame image and at least one historical frame image of the video to be detected; the density estimation module 63 may be configured to perform crowd density estimation on the current frame image and the historical frame image respectively to obtain crowd density maps corresponding to the current frame image and the historical frame image respectively; the optical flow detection module 65 may be configured to perform optical flow detection by using the crowd density maps corresponding to the current frame image and the historical frame image, respectively, to determine a crowd density change parameter; the people flow direction determination module 67 may be configured to determine the people flow direction based on the crowd density variation parameter.
According to an exemplary embodiment of the present disclosure, the image extraction module 61 may be configured to perform: and extracting a frame image which is a preset time away from the current frame image in the video to be detected as a historical frame image.
According to an exemplary embodiment of the present disclosure, the image extraction module 61 may be further configured to perform: and extracting each frame image in a preset time period from the current frame image in the video to be detected as a historical frame image. In this case, the people flow direction determining module 67 may be configured to perform: and determining the direction of the people flow by using the crowd density change parameter between every two adjacent frame images in a preset time period.
According to an exemplary embodiment of the present disclosure, the image extraction module 61 may be configured to perform: performing image quality evaluation on each frame image in a preset time period from the current frame image in the video to be detected; and eliminating the images with the scores smaller than the score threshold after image quality evaluation, so that the people flow direction determining module 67 determines the people flow direction by using the people density change parameters between every two adjacent frames of images after the images with the scores smaller than the score threshold are eliminated.
According to an exemplary embodiment of the present disclosure, the optical flow detection module 65 may be configured to perform: determining the pixel value variation of each pixel point in the crowd density map by utilizing the crowd density map respectively corresponding to the current frame image and the historical frame image; and determining the crowd density change parameter of each pixel point according to the pixel value variable quantity of each pixel point in the crowd density graph.
According to an exemplary embodiment of the present disclosure, referring to fig. 7, the people flow direction determining module 67 may include a parameter clustering unit 701 and a people flow direction determining unit 703.
Specifically, the parameter clustering unit 701 may be configured to cluster each pixel point according to the crowd density variation parameter; the people flow direction determining unit 703 may be configured to determine the people flow direction according to the result of the clustering.
According to an exemplary embodiment of the present disclosure, the crowd density variation parameter includes a crowd density variation size and a crowd density variation direction, in which case the parameter clustering unit 701 may be configured to perform: determining pixel points with the crowd density change size larger than the density change threshold value as pixel points to be clustered; and clustering the pixels to be clustered according to the crowd density change direction of the pixels to be clustered and the distance between the pixels to be clustered.
According to an exemplary embodiment of the present disclosure, the people flow direction determining unit 703 may be configured to perform: determining the number of pixels contained in each cluster after clustering; and eliminating clusters with the number of pixel points smaller than the preset number, and determining the direction of the people flow by using the rest clusters in the clustering result.
Since each functional module of the people flow direction detection device in the embodiment of the present invention is the same as that in the embodiment of the present invention, the description thereof is omitted here.
In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above section "exemplary methods" of the present description, when said program product is run on the terminal device.
The program product for implementing the above method according to an embodiment of the present invention may employ a portable compact disc read only memory (CD-ROM) and include program codes, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical disk, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device 800 according to this embodiment of the invention is described below with reference to fig. 8. The electronic device 800 shown in fig. 8 is only an example and should not bring any limitations to the function and scope of use of the embodiments of the present invention.
As shown in fig. 8, electronic device 800 is in the form of a general purpose computing device. The components of the electronic device 800 may include, but are not limited to: the at least one processing unit 810, the at least one memory unit 820, a bus 830 connecting different system components (including the memory unit 820 and the processing unit 810), and a display unit 840.
Wherein the storage unit stores program code that is executable by the processing unit 810 to cause the processing unit 810 to perform steps according to various exemplary embodiments of the present invention as described in the above section "exemplary methods" of the present specification. For example, the processing unit 810 may perform steps S12 through S18 as shown in fig. 1.
The storage unit 820 may include readable media in the form of volatile memory units such as a random access memory unit (RAM)8201 and/or a cache memory unit 8202, and may further include a read only memory unit (ROM) 8203.
The storage unit 820 may also include a program/utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 800 may also communicate with one or more external devices 900 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 800, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 800 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interfaces 850. Also, the electronic device 800 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 860. As shown, the network adapter 860 communicates with the other modules of the electronic device 800 via the bus 830. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.
Claims (11)
1. A people flow direction detection method is characterized by comprising the following steps:
acquiring a video to be detected, and extracting a current frame image and at least one historical frame image of the video to be detected;
respectively carrying out crowd density estimation on the current frame image and the historical frame image to obtain crowd density maps respectively corresponding to the current frame image and the historical frame image;
performing optical flow detection by using crowd density graphs respectively corresponding to the current frame image and the historical frame image to determine crowd density change parameters;
and determining the direction of the people flow based on the crowd density change parameter.
2. The people flow direction detection method according to claim 1, wherein extracting at least one historical frame image of the video to be detected comprises:
and extracting a frame image which is a preset time away from the current frame image in the video to be detected as a historical frame image.
3. The people flow direction detection method according to claim 1, wherein the historical frame image comprises each frame image in the video to be detected within a preset time period from the current frame image; wherein the content of the first and second substances,
determining a flow direction based on the crowd density variation parameter comprises:
and determining the direction of the people flow by using the crowd density change parameter between every two adjacent frame images in the preset time period.
4. The people flow direction detection method according to claim 3, characterized by further comprising:
performing image quality evaluation on each frame image in the video to be detected within a preset time period from the current frame image;
and eliminating the images with the scores smaller than the score threshold after image quality evaluation, so as to determine the people flow direction by using the crowd density change parameters between every two adjacent frames of images after the images with the scores smaller than the score threshold are eliminated.
5. The method of detecting a human flow direction according to claim 1, wherein the determining a crowd density variation parameter by performing optical flow detection using a crowd density map corresponding to each of the current frame image and the historical frame image includes:
determining the pixel value variation of each pixel point in the crowd density map by utilizing the crowd density map respectively corresponding to the current frame image and the historical frame image;
and determining the crowd density change parameter of each pixel point according to the pixel value variable quantity of each pixel point in the crowd density graph.
6. The people flow direction detection method according to any one of claims 1 to 5, wherein determining the people flow direction based on the crowd density variation parameter comprises:
clustering each pixel point according to the crowd density change parameter;
and determining the direction of the people flow according to the clustering result.
7. The method according to claim 6, wherein the crowd density variation parameters comprise a crowd density variation size and a crowd density variation direction; wherein, clustering each pixel point according to the crowd density variation parameter comprises:
determining pixel points with the crowd density change size larger than the density change threshold value as pixel points to be clustered;
and clustering the pixels to be clustered according to the crowd density change direction of the pixels to be clustered and the distance between the pixels to be clustered.
8. The people flow direction detection method according to claim 7, wherein determining the people flow direction according to the result of clustering comprises:
determining the number of pixels contained in each cluster after clustering;
and eliminating clusters with the number of pixel points smaller than the preset number, and determining the direction of the people flow by using the rest clusters in the clustering result.
9. A people flow direction detection device, comprising:
the image extraction module is used for acquiring a video to be detected and extracting a current frame image and at least one historical frame image of the video to be detected;
the density estimation module is used for respectively carrying out crowd density estimation on the current frame image and the historical frame image to obtain crowd density maps respectively corresponding to the current frame image and the historical frame image;
the optical flow detection module is used for carrying out optical flow detection by utilizing crowd density graphs respectively corresponding to the current frame image and the historical frame image to determine crowd density change parameters;
and the people flow direction determining module is used for determining the people flow direction based on the crowd density change parameter.
10. A storage medium on which a computer program is stored, the computer program, when executed by a processor, implementing the people flow direction detection method of any one of claims 1 to 8.
11. An electronic device, comprising:
a processor; and
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform the people flow direction detection method of any one of claims 1 to 8 via execution of the executable instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911318979.6A CN111062337B (en) | 2019-12-19 | 2019-12-19 | People stream direction detection method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911318979.6A CN111062337B (en) | 2019-12-19 | 2019-12-19 | People stream direction detection method and device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111062337A true CN111062337A (en) | 2020-04-24 |
| CN111062337B CN111062337B (en) | 2023-08-04 |
Family
ID=70302485
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911318979.6A Active CN111062337B (en) | 2019-12-19 | 2019-12-19 | People stream direction detection method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111062337B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU4807497A (en) * | 1996-10-31 | 1998-05-22 | Sensormatic Electronics Corporation | Intelligent video information management system |
| US20090222388A1 (en) * | 2007-11-16 | 2009-09-03 | Wei Hua | Method of and system for hierarchical human/crowd behavior detection |
| CN102324016A (en) * | 2011-05-27 | 2012-01-18 | 郝红卫 | Statistical method for high-density crowd flow |
| US20130243343A1 (en) * | 2012-03-16 | 2013-09-19 | Nec (China) Co., Ltd. | Method and device for people group detection |
| WO2015014882A1 (en) * | 2013-07-31 | 2015-02-05 | Connaught Electronics Ltd. | Method for detecting a target object by clustering of characteristic features of an image, camera system and motor vehicle |
| US20160078883A1 (en) * | 2013-04-26 | 2016-03-17 | Nec Corporation | Action analysis device, action analysis method, and action analysis program |
| CN107330372A (en) * | 2017-06-05 | 2017-11-07 | 四川大学 | A kind of crowd density based on video and the analysis method of unusual checking system |
| CN110296707A (en) * | 2019-07-31 | 2019-10-01 | 张家界航空工业职业技术学院 | Indoor service robot based on VLC technology |
| CN110390685A (en) * | 2019-07-24 | 2019-10-29 | 中国人民解放军国防科技大学 | Feature point tracking method based on event camera |
-
2019
- 2019-12-19 CN CN201911318979.6A patent/CN111062337B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU4807497A (en) * | 1996-10-31 | 1998-05-22 | Sensormatic Electronics Corporation | Intelligent video information management system |
| US20090222388A1 (en) * | 2007-11-16 | 2009-09-03 | Wei Hua | Method of and system for hierarchical human/crowd behavior detection |
| CN102324016A (en) * | 2011-05-27 | 2012-01-18 | 郝红卫 | Statistical method for high-density crowd flow |
| US20130243343A1 (en) * | 2012-03-16 | 2013-09-19 | Nec (China) Co., Ltd. | Method and device for people group detection |
| US20160078883A1 (en) * | 2013-04-26 | 2016-03-17 | Nec Corporation | Action analysis device, action analysis method, and action analysis program |
| WO2015014882A1 (en) * | 2013-07-31 | 2015-02-05 | Connaught Electronics Ltd. | Method for detecting a target object by clustering of characteristic features of an image, camera system and motor vehicle |
| CN107330372A (en) * | 2017-06-05 | 2017-11-07 | 四川大学 | A kind of crowd density based on video and the analysis method of unusual checking system |
| CN110390685A (en) * | 2019-07-24 | 2019-10-29 | 中国人民解放军国防科技大学 | Feature point tracking method based on event camera |
| CN110296707A (en) * | 2019-07-31 | 2019-10-01 | 张家界航空工业职业技术学院 | Indoor service robot based on VLC technology |
Non-Patent Citations (1)
| Title |
|---|
| 陈祺, no. 09 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111062337B (en) | 2023-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200167554A1 (en) | Gesture Recognition Method, Apparatus, And Device | |
| US10915741B2 (en) | Time domain action detecting methods and system, electronic devices, and computer storage medium | |
| CN111178183B (en) | Face detection method and related device | |
| US10009579B2 (en) | Method and system for counting people using depth sensor | |
| CN108876804B (en) | Matting model training and image matting method, device and system and storage medium | |
| CN108229418B (en) | Human body key point detection method and apparatus, electronic device, storage medium, and program | |
| CN112200081A (en) | Abnormal behavior identification method and device, electronic equipment and storage medium | |
| US20210201501A1 (en) | Motion-based object detection method, object detection apparatus and electronic device | |
| WO2014074959A1 (en) | Real-time face detection using pixel pairs | |
| CN110858316A (en) | Classifying time series image data | |
| CN112149615A (en) | Face living body detection method, device, medium and electronic equipment | |
| CN111027450A (en) | Bank card information identification method and device, computer equipment and storage medium | |
| CN112070052A (en) | Interval monitoring method, device and system and storage medium | |
| CN114549557A (en) | Portrait segmentation network training method, device, equipment and medium | |
| CN112561879A (en) | Ambiguity evaluation model training method, image ambiguity evaluation method and device | |
| US20220076005A1 (en) | Person authentication apparatus, control method, and non-transitory storage medium | |
| JPWO2018179119A1 (en) | Video analysis device, video analysis method, and program | |
| CN111062337B (en) | People stream direction detection method and device, storage medium and electronic equipment | |
| US20210287051A1 (en) | Methods and systems for recognizing object using machine learning model | |
| CN114612976A (en) | Key point detection method and device, computer readable medium and electronic equipment | |
| CN114445751A (en) | Method and device for extracting video key frame image contour features | |
| García et al. | A gaussian-median filter for moving objects segmentation applied for static scenarios | |
| CN111985423A (en) | Living body detection method, living body detection device, living body detection equipment and readable storage medium | |
| CN112579824A (en) | Video data classification method and device, electronic equipment and storage medium | |
| CN108694347B (en) | Image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |