CN111062337B - People stream direction detection method and device, storage medium and electronic equipment - Google Patents
People stream direction detection method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN111062337B CN111062337B CN201911318979.6A CN201911318979A CN111062337B CN 111062337 B CN111062337 B CN 111062337B CN 201911318979 A CN201911318979 A CN 201911318979A CN 111062337 B CN111062337 B CN 111062337B
- Authority
- CN
- China
- Prior art keywords
- crowd density
- frame image
- determining
- image
- people
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
- G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
Abstract
The disclosure provides a people flow direction detection method, a people flow direction detection device, a storage medium and electronic equipment, and relates to the technical field of image processing. The people flow direction detection method comprises the following steps: acquiring a video to be detected, and extracting a current frame image and at least one historical frame image of the video to be detected; crowd density estimation is respectively carried out on the current frame image and the historical frame image, and crowd density graphs respectively corresponding to the current frame image and the historical frame image are obtained; carrying out optical flow detection by using crowd density maps respectively corresponding to the current frame image and the historical frame image, and determining crowd density change parameters; and determining the people stream direction based on the crowd density variation parameter. The accuracy of crowd flow direction detection can be improved.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to a people stream direction detection method, a people stream direction detection device, a storage medium, and an electronic apparatus.
Background
With the rapid development of image processing technology and the continuous expansion of application range thereof, the image processing technology can be applied to the field of crowd statistical analysis. The detection of the crowd flow direction is taken as an important branch of crowd statistical analysis, and has important application value in aspects of scenic spot management, current limiting control, safety prevention and the like.
At present, in the process of detecting the crowd flow direction, due to the limitation of a shooting object and a shooting environment, the problem of low accuracy of crowd flow direction detection may occur.
It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The disclosure aims to provide a people flow direction detection method, a people flow direction detection device, a storage medium and electronic equipment, so as to overcome the problem of low detection accuracy of crowd flow directions at least to a certain extent.
According to a first aspect of the present disclosure, there is provided a people stream direction detection method, including: acquiring a video to be detected, and extracting a current frame image and at least one historical frame image of the video to be detected; crowd density estimation is respectively carried out on the current frame image and the historical frame image, and crowd density graphs respectively corresponding to the current frame image and the historical frame image are obtained; carrying out optical flow detection by using crowd density maps respectively corresponding to the current frame image and the historical frame image, and determining crowd density change parameters; and determining the people stream direction based on the crowd density variation parameter.
According to a second aspect of the present disclosure, there is provided a people stream direction detection apparatus including an image extraction module, a density estimation module, an optical flow detection module, and a people stream direction determination module.
Specifically, the image extraction module is used for acquiring a video to be detected and extracting a current frame image and at least one historical frame image of the video to be detected; the density estimation module is used for carrying out crowd density estimation on the current frame image and the historical frame image respectively to obtain crowd density images corresponding to the current frame image and the historical frame image respectively; the optical flow detection module is used for carrying out optical flow detection by utilizing crowd density maps respectively corresponding to the current frame image and the historical frame image, and determining crowd density change parameters; the people stream direction determining module is used for determining the people stream direction based on the crowd density change parameter.
Alternatively, the image extraction module may be configured to perform: and extracting a frame image which is a preset time from the current frame image in the video to be detected as a historical frame image.
Optionally, the image extraction module may be further configured to perform: and extracting each frame image within a preset time period from the current frame image in the video to be detected as a historical frame image. In this case, the people stream direction determination module may be configured to perform: and determining the people stream direction by using the crowd density change parameters between every two adjacent frame images in a preset time period.
Alternatively, the image extraction module may be configured to perform: evaluating the image quality of each frame of image within a preset time period from the current frame of image in the video to be detected; and eliminating the images with the scores smaller than the scoring threshold after the image quality evaluation is carried out, so that the people stream direction determining module determines the people stream direction by using the crowd density change parameters between every two adjacent frame images with the scores smaller than the scoring threshold after eliminating the images.
Alternatively, the optical flow detection module may be configured to perform: determining the pixel value variation of each pixel point in the crowd density map by using the crowd density map respectively corresponding to the current frame image and the historical frame image; and determining crowd density change parameters of each pixel point according to the pixel value change quantity of each pixel point in the crowd density map.
Alternatively, the people stream direction determination module may include a parameter clustering unit and a people stream direction determination unit.
Specifically, the parameter clustering unit is used for clustering each pixel point according to the crowd density change parameters; the people stream direction determining unit is used for determining the people stream direction according to the clustering result.
Optionally, the crowd density variation parameter includes a crowd density variation size and a crowd density variation direction, in which case the parameter clustering unit may be configured to perform: determining pixel points with crowd density change larger than a density change threshold value as pixel points to be clustered; and clustering the pixel points to be clustered according to the crowd density change direction of the pixel points to be clustered and the distance between the pixel points to be clustered.
Alternatively, the people stream direction determination unit may be configured to perform: determining the number of pixel points contained in each cluster after clustering; clusters with the number of pixels smaller than the preset number are removed, and the people flow direction is determined by using the rest clusters in the clustering result.
According to a third aspect of the present disclosure, there is provided a storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described people stream direction detection method.
According to a fourth aspect of the present disclosure, there is provided an electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-described people stream direction detection method via execution of the executable instructions.
In some embodiments of the present disclosure, a current frame image and at least one historical frame image of a video to be detected are extracted, crowd density estimation is performed on the extracted images to obtain crowd density maps of the images, optical flow detection is performed by using the obtained crowd density maps, crowd density variation parameters are determined, and a people flow direction is determined based on the crowd density variation parameters. On one hand, the crowd flow direction is analyzed by using the crowd density map, so that the scheme disclosed by the invention can be applied to scenes with more people, and the problem of poor portrait tracking effect caused by directly analyzing images is avoided; on the other hand, compared with the scheme of performing feature point matching on the original image in some technologies, the scheme of calculating the optical flow based on the density map disclosed by the invention has the advantage that only human information exists on the density map when optical flow processing is performed, so that environmental interference such as background, illumination and the like can be eliminated, and the accuracy and the robustness of crowd flow direction detection are improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. It will be apparent to those of ordinary skill in the art that the drawings in the following description are merely examples of the disclosure and that other drawings may be derived from them without undue effort. In the drawings:
fig. 1 schematically illustrates a flow chart of a people stream direction detection method according to an exemplary embodiment of the present disclosure;
FIG. 2 shows a schematic diagram of converting a captured image into a population density map;
FIG. 3 shows another schematic diagram of converting a captured image into a population density map;
FIG. 4 is a schematic diagram of the crowd density change determined by optical flow detection using crowd density maps according to the present disclosure;
FIG. 5 schematically illustrates a flow chart for determining a direction of people flow using crowd density variation parameters according to an exemplary embodiment of the present disclosure;
fig. 6 schematically illustrates a block diagram of a people stream direction detection device according to an exemplary embodiment of the present disclosure;
FIG. 7 schematically illustrates a block diagram of a people stream direction determination module according to an exemplary embodiment of the present disclosure;
fig. 8 schematically illustrates a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the present disclosure. One skilled in the relevant art will recognize, however, that the aspects of the disclosure may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.
The flow diagrams depicted in the figures are exemplary only and not necessarily all steps are included. For example, some steps may be decomposed, and some steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.
In one embodiment of the present disclosure, people flow direction in a video may be determined by detecting and tracking people in the video. However, this method may have problems of unstable detection and poor tracking effect for scenes with a large number of people or with a large number of people and a small scene.
In another embodiment of the present disclosure, the people stream direction in the video may be determined based on a feature point matching method of the video. However, the method is seriously affected by shooting environment, the characteristic points are unstable due to interference of illumination, weather, background and the like of the video, the characteristic points cannot be well matched with people, and the accuracy is low.
In view of this, in the following exemplary embodiments, a method capable of improving the accuracy of crowd flow direction detection is provided.
The people stream direction detection method of the present disclosure may be applied to a monitoring scene, in which case the device performing the method described below may be a terminal device connected to a monitoring camera. Specifically, the terminal device and the monitoring camera can be in communication connection in a wired or wireless mode. The terminal device may include, but is not limited to, a personal computer, tablet, cell phone, etc. In addition, the terminal device may also refer to a server, and display the detection result of the crowd flowing direction through the display device.
In addition, it should be noted that, the people flow direction detection method of the present disclosure may be applied to, for example, monitoring scenes such as scenic spot management, current limiting control, security prevention, etc., and also may be applied to people flow analysis on an area to determine the possibility and the location of setting up a shop, which is not limited in this disclosure.
Fig. 1 schematically shows a flowchart of a people stream direction detection method according to an exemplary embodiment of the present disclosure. Referring to fig. 1, the people stream direction detection method may include the steps of:
s12, acquiring a video to be detected, and extracting a current frame image and at least one historical frame image of the video to be detected.
In some embodiments of the present disclosure, the video to be detected may be a video obtained by capturing a scene in real time. That is, the video to be detected may be a video in which an image is being continuously generated. In this case, the current frame image is an image corresponding to the current time acquired by the camera in real time, and the history frame image may be an image generated before the current time.
In other embodiments of the present disclosure, the video to be detected may also be a video that has been shot and analyzed, in which case the video to be detected may be a video generated by pre-shooting, or may be a video obtained from a network or other device, and the source of the video is not limited in this disclosure. In addition, for a video to be detected that has been completed, the current frame image may be an image corresponding to a video time point at which the user desires to determine the direction of the human flow, and the history frame image may be a video frame image that is prior to the current frame image at the time point. For example, for a 30 second video, the user desires to know the people flow direction in the video at 10 seconds, and then the video frame image corresponding to the 10 th second in the video can be determined as the current frame image, and the corresponding historical frame image can be the video frame image before the 10 th second.
In an exemplary embodiment of the present disclosure, for a video to be detected, at least one history frame image may be extracted in addition to the current frame image to perform the process of the following steps.
According to some embodiments of the present disclosure, the terminal device may extract a history frame image to implement the process of determining the people stream direction. Specifically, an image of the video to be detected, which is a frame of the video to be detected and is a preset time from the current frame of the image, may be extracted as the historical frame of the image extracted in step S12, where the preset time may be predetermined, for example, 3 seconds, 5 seconds, 10 seconds, etc., and it is easy to see that the determination of the preset time is related to the running duration of the video to be detected, which is not limited in the present disclosure.
For example, if the video to be detected has been operated for 30 seconds, the current frame image is a video frame image corresponding to the current 30 th second, and if the preset time is 10 seconds, the video frame image corresponding to the 20 th second when the video to be detected is operated may be determined as the history frame image.
In addition, on the one hand, the preset time may be adjusted according to the time length that the video to be detected has been run, for example, if the video to be detected has been run for only 7 seconds, an image corresponding to 3 seconds may be used as the history frame image. On the other hand, if the duration in which the video to be detected has been run is less than the minimum preset time, the first frame image of the video to be detected may be determined as the extracted history frame image.
According to other embodiments of the present disclosure, the terminal device may extract more than two history frame images to implement the process of determining the people stream direction. Specifically, each frame image within a preset period of time from the current frame image in the video to be detected may be extracted as the plurality of history frame images extracted in step S12, where the preset period of time may be predetermined, for example, 5 seconds, 10 seconds, 30 seconds, 1 minute, and the like.
For example, if the video to be detected has been operated for 30 seconds, the current frame image is the video frame image corresponding to the current 30 th second, and if the preset time period is 5 seconds and minutes, all the video frame images within the 25 th to 30 th seconds of the video to be detected operation can be determined as the history frame images.
Similarly, if the preset time period is longer than the running time of the video to be detected, taking all video frame images of the video to be detected as history frame images.
In addition, in consideration of the possible existence of signal interference, camera shake, and the like, there may be poor quality images in the history frame image. For embodiments that extract more than two historical frame images, the terminal device may analyze these historical frame images to reject images of poor image quality.
Specifically, first, image quality evaluation may be performed on each frame of image in the video to be detected within a preset period of time from the current frame of image, for example, the images may be respectively input into a trained image quality evaluation model to determine the score of each image.
The scores of the images may then be compared to a score threshold, respectively, and images having scores less than the score threshold may be rejected. That is, in the subsequent processing, an image having a score equal to or greater than the score threshold is used as the history frame image for processing.
S14, crowd density estimation is carried out on the current frame image and the historical frame image respectively, and crowd density images corresponding to the current frame image and the historical frame image respectively are obtained.
After determining the current frame image, the terminal device may perform crowd density estimation on the current frame image.
According to some embodiments of the present disclosure, the greater the population density, the greater the proportion of the population to the entire image, and the greater the number of extracted background image edge pixels. In view of this, the pixel statistics of the current frame image can be calculated to determine the edges of the population in the image, and thus determine the population density estimate of the current frame image. In addition, the texture characteristics of the current frame image representing gray level change or color change can be directly determined, and crowd density estimation is determined by utilizing the texture characteristics. It will be readily appreciated that the pixel statistics and texture information may also be combined to determine a population density estimate for the current frame image. After the crowd density estimation is determined, a corresponding crowd density map can be generated according to the estimation result.
According to other embodiments of the present disclosure, crowd density estimation of a current frame image may be performed based on a crowd density estimation method of CNN (Convolutional Neural Networks, convolutional neural network), generating a crowd density map. Specifically, any one network of MCNN, CP-CNN, CSRNet, ic-CNN and SANet can be adopted to determine the crowd density map of the current frame image. The present disclosure does not limit the parameters of the network and the training process.
Taking the MCNN network as an example, three-column convolutional neural networks can be adopted, the current frame image is respectively input into the three-column convolutional neural networks, and then the outputs of the three-column convolutional neural networks are combined to obtain a crowd density map corresponding to the current frame image.
Similarly, crowd density estimation is also performed on the history frame image determined in step S12 to obtain a corresponding crowd density map, which is not described herein.
Referring to fig. 2 and 3, corresponding schematic diagrams of video frame images and crowd density maps are shown. Specifically, the crowd density estimation may be performed on the image 21 to obtain a crowd density map 22 corresponding to the image 21; the crowd density estimation described above may be performed on the image 31 to obtain a crowd density map 32 corresponding to the image 31.
S16, carrying out optical flow detection by using crowd density maps respectively corresponding to the current frame image and the historical frame image, and determining crowd density change parameters.
In exemplary embodiments of the present disclosure, optical flow detection may be understood as a method of calculating motion information of an object (i.e., a person) from frame to frame. Wherein the motion information can be characterized by the pixel value variation of the pixel point.
Specifically, the terminal device may determine a pixel value variation of each pixel point in the crowd density map by using the crowd density maps corresponding to the current frame image and the historical frame image respectively, and determine a crowd density variation parameter of each pixel point according to the pixel value variation.
The crowd density change parameters according to the exemplary embodiments of the present disclosure include, but are not limited to, a crowd density change size and a crowd density change direction, and the crowd density change size can be understood as the intensity of the density change.
In the embodiment of determining the crowd density variation parameter by using the current frame image and one historical frame image, the crowd density map of the current frame image and the crowd density map of the historical frame image can be directly compared, the crowd density estimation of the historical frame image is used as a starting point, and the crowd density estimation of the current frame image is used as an end point, so that the crowd density variation parameter is obtained.
In embodiments where the current frame image and two or more historical frame images are utilized to determine the crowd density variation parameters, optical flow detection may be implemented, for example, using a gradient-based method, a neural network-based method, and the like, as the optical flow method is not limited by the present disclosure.
Gradient-based methods, which may also be referred to as differentiation methods, utilize the spatiotemporal differentiation (spatiotemporal gradient function) of the gray scale (or filtered version thereof) of a time-varying image to calculate the velocity vector of a pixel. Specifically, the method can comprise a Horn-Schunck algorithm, a Lucas-Kanade (LK) algorithm and the like.
It should be noted that, for embodiments in which there are a current frame image and two or more historical frame images, the images may form an image sequence sequentially according to video time, and in this case, the process of determining the crowd density variation parameter includes determining the crowd density variation parameter between every two adjacent frame images in the image sequence.
Referring to fig. 4, a schematic diagram of a crowd density change determined by optical flow detection using a crowd density map is shown. The crowd density parameter can be obtained through the graph, namely, the whiter the area is, the larger the crowd density change is, and the arrow indicates the crowd density change direction.
S18, determining the people stream direction based on the crowd density change parameters.
According to some embodiments of the present disclosure, the people stream direction may be determined directly by using the crowd density variation parameter determined in step S16. It should be understood that, in the scene corresponding to the current frame image, there may be a plurality of crowd density changes, that is, there may be a plurality of density change directions, where the density change directions are different, so that a plurality of people flow directions may be determined.
Noise may be present in view of the density map calculation and the video itself, resulting in a deviation of the determined people stream direction from the actual. According to other embodiments of the present disclosure, the crowd density variation parameter determined in step S16 may be denoised, and then the direction of the people stream may be determined using the denoising result.
Specifically, the crowd density variation parameters may be denoised in a clustering manner, which is described below in conjunction with fig. 5.
Referring to fig. 5, in step S502, the terminal device may cluster the crowd density variation parameters, and may implement clustering of the density variation size and the density variation direction by using, for example, a K-means clustering (K-means clustering) algorithm, and it is easy to see that the clustering process based on the density variation size and the density variation direction is combined with the pixel point position information of the image. After clustering, a plurality of clusters can be obtained.
According to some embodiments of the present disclosure, each pixel point may be used as a pixel point to be clustered, and clustering is performed by using a clustering algorithm, so as to obtain a clustering result of each pixel point.
According to other embodiments of the present disclosure, the terminal device may determine pixels with a crowd density variation greater than a density variation threshold, and use the pixels as pixels to be clustered. It should be appreciated that in the case where the population density change is less than or equal to the density change threshold, it is indicated that the population density change degree at the corresponding pixels is small, and the pixels can be ignored when considering the traffic direction, so that on one hand, the complexity of the subsequent clustering algorithm can be reduced, and on the other hand, the problem that the pixels may interfere with the main traffic direction and cause the attention shift during analysis is avoided.
After the pixel points to be clustered are determined, the pixel points to be clustered can be clustered according to the crowd density direction of the pixel points to be clustered and the distance between the pixel points to be clustered. The distance herein refers to an actual physical distance between pixels in an image, which may be represented by a specific length unit, or may be represented by a number of pixels separated from each other, which is not limited in this disclosure.
The distance defined during clustering is described below by taking two pixel points to be clustered as an example.
The two pixels to be clustered are respectively marked as a first pixel and a second pixel, and it is understood that the first pixel and the second pixel are arbitrarily selected pixels in the pixels to be clustered. First, determining a density change direction of a first pixel point and a density change direction of a second pixel point, and calculating an included angle between the two directions. Next, an actual physical distance between the first pixel point and the second pixel point is determined. Then, a weight corresponding to the included angle and a weight corresponding to the physical distance may be obtained, and a weighted result of the included angle and the physical distance may be determined as a distance defined when clustering is performed, that is, a clustering process of the pixel points to be clustered is performed by using the calculated distance. The weights may be preconfigured, and specific values thereof are not limited in the disclosure.
In step S504, the terminal device may determine the number of pixel points included in each cluster; in step S506, clusters with the number of pixels smaller than the predetermined number are determined, and these clusters are eliminated, where the predetermined number may be determined in advance by a developer based on the image resolution and the scene requirement, and the specific value of the present disclosure is not limited.
In step S508, the terminal device may determine the people stream direction by using the clusters remaining after the removal.
That is, only the result of the large change in crowd density is retained, and for the case where the change in crowd density is determined to be small, it may be due to noise or individual cases of several persons, which are not considered in determining the crowd flow direction specifically.
After the people flow direction is determined by adopting the method, the safety precaution effect can be achieved by combining the scene and the number of people. For example, in the scene of monitoring the hanging bridge, if a large number of tourists are determined to be rushed to the hanging bridge, an alarm can be sent out through a loudspeaker and security personnel can be notified, so that the occurrence of safety accidents caused by trampling or damage to the hanging bridge is reduced as much as possible.
It should be noted that although the steps of the methods in the present disclosure are depicted in the accompanying drawings in a particular order, this does not require or imply that the steps must be performed in that particular order, or that all illustrated steps be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform, etc.
Further, in this example embodiment, a people flow direction detection device is also provided.
Fig. 6 schematically shows a block diagram of a people stream direction detection device according to an exemplary embodiment of the present disclosure. Referring to fig. 6, the people stream direction detection apparatus 6 according to an exemplary embodiment of the present disclosure may include an image extraction module 61, a density estimation module 63, an optical flow detection module 65, and a people stream direction determination module 67.
Specifically, the image extraction module 61 may be configured to obtain a video to be detected, and extract a current frame image and at least one historical frame image of the video to be detected; the density estimation module 63 may be configured to perform crowd density estimation on the current frame image and the historical frame image respectively, so as to obtain crowd density maps corresponding to the current frame image and the historical frame image respectively; the optical flow detection module 65 may be configured to perform optical flow detection by using crowd density maps corresponding to the current frame image and the historical frame image, and determine crowd density variation parameters; the people stream direction determination module 67 may be used to determine people stream direction based on the crowd density variation parameters.
According to an exemplary embodiment of the present disclosure, the image extraction module 61 may be configured to perform: and extracting a frame image which is a preset time from the current frame image in the video to be detected as a historical frame image.
According to an exemplary embodiment of the present disclosure, the image extraction module 61 may be further configured to perform: and extracting each frame image within a preset time period from the current frame image in the video to be detected as a historical frame image. In this case, the people stream direction determination module 67 may be configured to perform: and determining the people stream direction by using the crowd density change parameters between every two adjacent frame images in a preset time period.
According to an exemplary embodiment of the present disclosure, the image extraction module 61 may be configured to perform: evaluating the image quality of each frame of image within a preset time period from the current frame of image in the video to be detected; images with scores smaller than the score threshold after image quality evaluation are removed, so that the people stream direction determining module 67 determines people stream directions by using crowd density change parameters between every two adjacent frame images after images with scores smaller than the score threshold are removed.
According to an example embodiment of the disclosure, the optical flow detection module 65 may be configured to perform: determining the pixel value variation of each pixel point in the crowd density map by using the crowd density map respectively corresponding to the current frame image and the historical frame image; and determining crowd density change parameters of each pixel point according to the pixel value change quantity of each pixel point in the crowd density map.
According to an exemplary embodiment of the present disclosure, referring to fig. 7, the people stream direction determining module 67 may include a parameter clustering unit 701 and a people stream direction determining unit 703.
Specifically, the parameter clustering unit 701 may be configured to cluster each pixel point according to a crowd density variation parameter; the people stream direction determination unit 703 may be used for determining the people stream direction from the result of the clustering.
According to an exemplary embodiment of the present disclosure, the crowd density variation parameter includes a crowd density variation size and a crowd density variation direction, in which case the parameter clustering unit 701 may be configured to perform: determining pixel points with crowd density change larger than a density change threshold value as pixel points to be clustered; and clustering the pixel points to be clustered according to the crowd density change direction of the pixel points to be clustered and the distance between the pixel points to be clustered.
According to an exemplary embodiment of the present disclosure, the people stream direction determination unit 703 may be configured to perform: determining the number of pixel points contained in each cluster after clustering; clusters with the number of pixels smaller than the preset number are removed, and the people flow direction is determined by using the rest clusters in the clustering result.
Because each functional module of the people flow direction detection device in the embodiment of the present invention is the same as that in the embodiment of the present invention, the description thereof is omitted herein.
In an exemplary embodiment of the present disclosure, a computer-readable storage medium having stored thereon a program product capable of implementing the method described above in the present specification is also provided. In some possible embodiments, the various aspects of the invention may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the invention as described in the "exemplary methods" section of this specification, when said program product is run on the terminal device.
The program product for implementing the above-described method according to an embodiment of the present invention may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical disk, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
Those skilled in the art will appreciate that the various aspects of the invention may be implemented as a system, method, or program product. Accordingly, aspects of the invention may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.
An electronic device 800 according to such an embodiment of the invention is described below with reference to fig. 8. The electronic device 800 shown in fig. 8 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 8, the electronic device 800 is embodied in the form of a general purpose computing device. Components of electronic device 800 may include, but are not limited to: the at least one processing unit 810, the at least one storage unit 820, a bus 830 connecting the different system components (including the storage unit 820 and the processing unit 810), and a display unit 840.
Wherein the storage unit stores program code that is executable by the processing unit 810 such that the processing unit 810 performs steps according to various exemplary embodiments of the present invention described in the above section of the "exemplary method" of the present specification. For example, the processing unit 810 may perform steps S12 to S18 as shown in fig. 1.
The storage unit 820 may include readable media in the form of volatile storage units, such as Random Access Memory (RAM) 8201 and/or cache memory 8202, and may further include Read Only Memory (ROM) 8203.
Storage unit 820 may also include a program/utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 830 may be one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device 800 may also communicate with one or more external devices 900 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 800, and/or any device (e.g., router, modem, etc.) that enables the electronic device 800 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 850. Also, electronic device 800 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through network adapter 860. As shown, network adapter 860 communicates with other modules of electronic device 800 over bus 830. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 800, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, including several instructions to cause a computing device (may be a personal computer, a server, a terminal device, or a network device, etc.) to perform the method according to the embodiments of the present disclosure.
Furthermore, the above-described drawings are only schematic illustrations of processes included in the method according to the exemplary embodiment of the present invention, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.
It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (9)
1. A method for detecting a direction of a flow of people, comprising:
acquiring a video to be detected, and extracting a current frame image and at least one historical frame image of the video to be detected;
crowd density estimation is respectively carried out on the current frame image and the historical frame image, so that crowd density images respectively corresponding to the current frame image and the historical frame image are obtained;
determining the pixel value variation of each pixel point in the crowd density map by using the crowd density map corresponding to the current frame image and the historical frame image respectively, and determining the crowd density variation parameter of each pixel point according to the pixel value variation of each pixel point in the crowd density map;
determining a people stream direction based on the crowd density variation parameter;
wherein determining the people stream direction based on the crowd density variation parameter comprises:
and clustering the pixel points according to the crowd density change parameters, and determining the people stream direction according to the clustering result.
2. The people stream direction detection method according to claim 1, characterized in that extracting at least one history frame image of the video to be detected comprises:
and extracting a frame of image which is in the video to be detected and is at a preset time from the current frame of image as a historical frame of image.
3. The people stream direction detection method according to claim 1, characterized in that the history frame image includes each frame image in the video to be detected within a preset period of time from the current frame image; wherein, the liquid crystal display device comprises a liquid crystal display device,
determining the people stream direction based on the crowd density variation parameter comprises:
and determining the people stream direction by using the crowd density change parameters between every two adjacent frame images in the preset time period.
4. The people stream direction detection method according to claim 3, characterized in that the people stream direction detection method further comprises:
performing image quality evaluation on each frame of image in the video to be detected within a preset time period from the current frame of image;
and eliminating the images with the scores smaller than the scoring threshold after the image quality evaluation so as to determine the people stream direction by using the crowd density change parameters between every two adjacent frame images after eliminating the images with the scores smaller than the scoring threshold.
5. The people stream direction detection method according to claim 1, characterized in that the crowd density variation parameter includes a crowd density variation size and a crowd density variation direction; wherein, clustering each pixel point according to the crowd density variation parameter comprises:
determining pixel points with crowd density change larger than a density change threshold value as pixel points to be clustered;
and clustering the pixel points to be clustered according to the crowd density change direction of the pixel points to be clustered and the distance between the pixel points to be clustered.
6. The people stream direction detection method of claim 5, wherein determining the people stream direction based on the result of the clustering comprises:
determining the number of pixel points contained in each cluster after clustering;
clusters with the number of pixels smaller than the preset number are removed, and the people flow direction is determined by using the rest clusters in the clustering result.
7. A people flow direction detection device, characterized by comprising:
the image extraction module is used for acquiring a video to be detected and extracting a current frame image and at least one historical frame image of the video to be detected;
the density estimation module is used for carrying out crowd density estimation on the current frame image and the historical frame image respectively to obtain crowd density images corresponding to the current frame image and the historical frame image respectively;
the optical flow detection module is used for determining the pixel value variation of each pixel point in the crowd density map by utilizing the crowd density map corresponding to the current frame image and the historical frame image respectively, and determining the crowd density variation parameter of each pixel point according to the pixel value variation of each pixel point in the crowd density map;
the people stream direction determining module is used for determining the people stream direction based on the crowd density change parameter;
wherein, the people flow direction determining module includes:
the parameter clustering unit is used for clustering each pixel point according to the crowd density change parameters;
and the people flow direction determining unit is used for determining the people flow direction according to the clustering result.
8. A storage medium having stored thereon a computer program, which when executed by a processor implements the people stream direction detection method of any of claims 1 to 6.
9. An electronic device, comprising:
a processor; and
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform the people stream direction detection method of any of claims 1 to 6 via execution of the executable instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911318979.6A CN111062337B (en) | 2019-12-19 | 2019-12-19 | People stream direction detection method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911318979.6A CN111062337B (en) | 2019-12-19 | 2019-12-19 | People stream direction detection method and device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111062337A CN111062337A (en) | 2020-04-24 |
| CN111062337B true CN111062337B (en) | 2023-08-04 |
Family
ID=70302485
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911318979.6A Active CN111062337B (en) | 2019-12-19 | 2019-12-19 | People stream direction detection method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111062337B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU4807497A (en) * | 1996-10-31 | 1998-05-22 | Sensormatic Electronics Corporation | Intelligent video information management system |
| CN102324016A (en) * | 2011-05-27 | 2012-01-18 | 郝红卫 | Statistical method for high-density crowd flow |
| WO2015014882A1 (en) * | 2013-07-31 | 2015-02-05 | Connaught Electronics Ltd. | Method for detecting a target object by clustering of characteristic features of an image, camera system and motor vehicle |
| CN107330372A (en) * | 2017-06-05 | 2017-11-07 | 四川大学 | A kind of crowd density based on video and the analysis method of unusual checking system |
| CN110296707A (en) * | 2019-07-31 | 2019-10-01 | 张家界航空工业职业技术学院 | Indoor service robot based on VLC technology |
| CN110390685A (en) * | 2019-07-24 | 2019-10-29 | 中国人民解放军国防科技大学 | Feature point tracking method based on event camera |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8195598B2 (en) * | 2007-11-16 | 2012-06-05 | Agilence, Inc. | Method of and system for hierarchical human/crowd behavior detection |
| CN103310183B (en) * | 2012-03-16 | 2016-12-14 | 日电（中国）有限公司 | The method and apparatus of crowd massing detection |
| US9761248B2 (en) * | 2013-04-26 | 2017-09-12 | Nec Corporation | Action analysis device, action analysis method, and action analysis program |
-
2019
- 2019-12-19 CN CN201911318979.6A patent/CN111062337B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU4807497A (en) * | 1996-10-31 | 1998-05-22 | Sensormatic Electronics Corporation | Intelligent video information management system |
| CN102324016A (en) * | 2011-05-27 | 2012-01-18 | 郝红卫 | Statistical method for high-density crowd flow |
| WO2015014882A1 (en) * | 2013-07-31 | 2015-02-05 | Connaught Electronics Ltd. | Method for detecting a target object by clustering of characteristic features of an image, camera system and motor vehicle |
| CN107330372A (en) * | 2017-06-05 | 2017-11-07 | 四川大学 | A kind of crowd density based on video and the analysis method of unusual checking system |
| CN110390685A (en) * | 2019-07-24 | 2019-10-29 | 中国人民解放军国防科技大学 | Feature point tracking method based on event camera |
| CN110296707A (en) * | 2019-07-31 | 2019-10-01 | 张家界航空工业职业技术学院 | Indoor service robot based on VLC technology |
Non-Patent Citations (1)
| Title |
|---|
| 陈祺.基于改进光流法的运动目标检测与跟踪.2019,(第09期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111062337A (en) | 2020-04-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200167554A1 (en) | Gesture Recognition Method, Apparatus, And Device | |
| CN107862270B (en) | Face classifier training method, face detection method and device and electronic equipment | |
| WO2017151241A2 (en) | Video processing | |
| US10445590B2 (en) | Image processing apparatus and method and monitoring system | |
| CN113012176B (en) | Sample image processing method and device, electronic equipment and storage medium | |
| CN112381104A (en) | Image identification method and device, computer equipment and storage medium | |
| US20210201501A1 (en) | Motion-based object detection method, object detection apparatus and electronic device | |
| CN110660102B (en) | Speaker recognition method, device and system based on artificial intelligence | |
| CN112200081A (en) | Abnormal behavior identification method and device, electronic equipment and storage medium | |
| Parveen et al. | A motion detection system in python and opencv | |
| WO2014074959A1 (en) | Real-time face detection using pixel pairs | |
| CN112149615A (en) | Face living body detection method, device, medium and electronic equipment | |
| CN112488071A (en) | Method, device, electronic equipment and storage medium for extracting pedestrian features | |
| CN112070052A (en) | Interval monitoring method, device and system and storage medium | |
| CN114898416A (en) | Face recognition method and device, electronic equipment and readable storage medium | |
| CN114549557A (en) | Portrait segmentation network training method, device, equipment and medium | |
| CN111191556A (en) | Face recognition method and device and electronic equipment | |
| CN111310595B (en) | Method and device for generating information | |
| WO2023231479A1 (en) | Pupil detection method and apparatus, and storage medium and electronic device | |
| JPWO2018179119A1 (en) | Video analysis device, video analysis method, and program | |
| CN111062337B (en) | People stream direction detection method and device, storage medium and electronic equipment | |
| CN111488887A (en) | Image processing method and device based on artificial intelligence | |
| CN108334811B (en) | Face image processing method and device | |
| CN111091056A (en) | Method and device for identifying sunglasses in image, electronic equipment and storage medium | |
| US20210287051A1 (en) | Methods and systems for recognizing object using machine learning model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |