CN115661758A - Public place crowd density monitoring method and system based on artificial intelligence - Google Patents
Public place crowd density monitoring method and system based on artificial intelligence Download PDFInfo
- Publication number
- CN115661758A CN115661758A CN202211427991.2A CN202211427991A CN115661758A CN 115661758 A CN115661758 A CN 115661758A CN 202211427991 A CN202211427991 A CN 202211427991A CN 115661758 A CN115661758 A CN 115661758A
- Authority
- CN
- China
- Prior art keywords
- feature map
- characteristic diagram
- convolution
- crowd
- difference
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The application discloses a public place crowd density monitoring method and system based on artificial intelligence, wherein a crowd monitoring image collected by a camera deployed in a public place is coded through a first convolution neural network using a first convolution kernel to obtain a first scale characteristic diagram, the crowd monitoring image is coded through a second convolution kernel using a second convolution kernel to obtain a second scale characteristic diagram, then, the difference between the first scale characteristic diagram and the second scale characteristic diagram is calculated to obtain a difference characteristic diagram, then, characteristic values of all positions in the difference characteristic diagram are corrected to obtain a corrected difference characteristic diagram, and finally, the corrected difference characteristic diagram passes through a classifier to obtain a classification result which is used for indicating whether the crowd density of the public place to be monitored exceeds a safety standard or not. Therefore, the accuracy of monitoring the crowd density in public places is improved.
Description
Technical Field
The application relates to the technical field of crowd monitoring, in particular to a public place crowd density monitoring method and system based on artificial intelligence.
Background
With the shortage of international situation and the occurrence of accidents in places where a plurality of public places in China are gathered, the monitoring of the crowd density in the public places becomes important. In recent years, artificial intelligence has made a major breakthrough in various fields such as big data analysis, computer vision, semantic analysis and the like, and the convolutional neural network has made a new breakthrough in population density estimation research, but the population density estimation still faces a series of challenges in research under the influence of problems such as spatial perspective, serious shielding, light change and the like.
Therefore, an optimized public space population intensity monitoring scheme is desired.
Disclosure of Invention
The present application is proposed to solve the above-mentioned technical problems. The embodiment of the application provides a public place crowd density monitoring system, a public place crowd density monitoring method and electronic equipment based on artificial intelligence, wherein a crowd monitoring image collected by a camera deployed in a public place is coded through a first convolution neural network using a first convolution kernel to obtain a first scale characteristic diagram, the crowd monitoring image is coded through a second convolution kernel using a second convolution kernel to obtain a second scale characteristic diagram, then, the difference between the first scale characteristic diagram and the second scale characteristic diagram is calculated to obtain a difference characteristic diagram, then, characteristic values of all positions in the difference characteristic diagram are corrected to obtain a corrected difference characteristic diagram, and finally, the corrected difference characteristic diagram is passed through a classifier to obtain a classification result which is used for indicating whether the crowd density of the public place to be monitored exceeds a safety standard or not. Therefore, the accuracy of monitoring the crowd density in public places is improved.
According to one aspect of the present application, there is provided an artificial intelligence based public space population intensity monitoring system, comprising:
the monitoring image acquisition module is used for acquiring a crowd monitoring image through a camera deployed in a public place;
the first scale coding module is used for enabling the crowd monitoring image to pass through a first convolution neural network using a first convolution kernel so as to obtain a first scale feature map;
the second scale coding module is used for enabling the crowd monitoring image to pass through a second convolution neural network using a second convolution kernel so as to obtain a second scale characteristic map;
the difference module is used for calculating the difference between the first scale feature map and the second scale feature map to obtain a difference feature map;
the correction module is used for correcting the characteristic values of all positions in the differential characteristic diagram to obtain a corrected differential characteristic diagram; and
and the crowd density monitoring result generating module is used for enabling the corrected differential characteristic diagram to pass through a classifier so as to obtain a classification result, and the classification result is used for indicating whether the crowd density of the public place to be monitored exceeds a safety standard or not.
On the other hand, the application provides a public place crowd intensity monitoring method based on artificial intelligence, which comprises the following steps:
collecting a crowd monitoring image through a camera deployed in a public place;
passing the crowd monitoring image through a first convolution neural network using a first convolution kernel to obtain a first scale feature map;
passing the crowd monitoring image through a second convolutional neural network using a second convolutional kernel to obtain a second scale feature map;
calculating the difference between the first scale feature map and the second scale feature map to obtain a difference feature map;
correcting the characteristic value of each position in the differential characteristic diagram to obtain a corrected differential characteristic diagram; and
and passing the corrected differential characteristic diagram through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the crowd density of the public place to be monitored exceeds a safety standard or not.
According to still another aspect of the present application, there is provided an electronic apparatus including: a processor; and a memory having stored therein computer program instructions that, when executed by the processor, cause the processor to perform the artificial intelligence based public space population intensity monitoring method as described above.
According to yet another aspect of the present application, there is provided a computer readable medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the artificial intelligence based public space crowd intensity monitoring method as described above.
Compared with the prior art, the public place crowd density monitoring system and method based on artificial intelligence provided by the application encode the crowd monitoring image collected by the camera deployed in the public place through the first convolution neural network using the first convolution kernel to obtain the first scale characteristic diagram, encode the crowd monitoring image through the second convolution kernel using the second convolution kernel to obtain the second scale characteristic diagram, then calculate the difference between the first scale characteristic diagram and the second scale characteristic diagram to obtain the difference characteristic diagram, then correct the characteristic value of each position in the difference characteristic diagram to obtain the corrected difference characteristic diagram, and finally pass the corrected difference characteristic diagram through the classifier to obtain the classification result which is used for indicating whether the crowd density of the public place to be monitored exceeds the safety standard. Therefore, the accuracy of monitoring the crowd density in public places is improved.
Drawings
The above and other objects, features and advantages of the present application will become more apparent by describing in more detail embodiments of the present application with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings, like reference numbers generally indicate like parts or steps.
FIG. 1 illustrates an application scenario diagram of a public space crowd density monitoring method based on artificial intelligence according to an embodiment of the application.
FIG. 2 illustrates a block diagram of an artificial intelligence based public space population intensity monitoring system according to an embodiment of the present application;
FIG. 3 illustrates an architectural diagram of an artificial intelligence based public space population intensity monitoring system in accordance with an embodiment of the present application;
FIG. 4 illustrates a flow chart of a method of artificial intelligence based public space population intensity monitoring in accordance with an embodiment of the present application;
FIG. 5 illustrates a block diagram of an electronic device in accordance with an embodiment of the present application.
Detailed Description
Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only a few embodiments of the present application, and not all embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.
Application overview:
through research, the inventor of the application finds that the majority of existing schemes are along the idea of target detection and target quantity measurement in crowd density estimation, that is, how many human objects are in an image is identified through target detection, whether the distribution density of the human objects in a preset public place exceeds a preset standard is determined through measurement, and if the distribution density exceeds the preset standard, an early warning that the crowd density is too dense is generated. However, in the actual operation process of the above detection scheme, the detection accuracy is difficult to meet the application requirements, because in the actual application scene, problems such as serious shielding and light change exist at the image acquisition end, and at the feature extraction end, because human objects belong to small-sized objects in the image, other objects are easy to be confused.
Based on this, the present inventors tried to construct a population density monitoring scheme from relative indicators of population density. That is, if the population density of the public place is high, the feature distributions under different receptive fields are more uniform, and if the population density of the public place is low, the feature distributions under different receptive fields are more different. Therefore, a population density monitoring scheme can be constructed based on the difference of the feature distribution of the population monitoring image in different receptive fields.
Specifically, a crowd monitoring image is collected through a camera deployed in a public place. Then, the crowd monitoring image is passed through a first convolution neural network and a second convolution neural network using different convolution kernels (defined as a first convolution kernel and a second convolution kernel for convenience of explanation) to obtain a first feature map and a second feature map. It should be understood that for the convolutional neural network model, different convolution kernels represent different characteristic receptive fields, and therefore, the first feature map and the second feature map are used to represent feature representations of population density distribution under different receptive fields.
In an embodiment of the application, the first convolution kernel and the second convolution kernel may be solid convolution kernels of different sizes, for example, the size of the first convolution kernel is smaller than the size of the second convolution kernel. Of course, in other examples of the present application, the first convolution kernel and the second convolution kernel are hole convolution kernels, and a hole rate of the first convolution kernel is smaller than a hole rate of the second convolution kernel, that is, the first convolution kernel and the second convolution kernel have the same size but have different hole rates so that the first convolution neural network and the second convolution neural network have different receptive fields.
Then, calculating the difference between the first characteristic diagram and the second characteristic diagram and enabling the obtained difference characteristic diagram to pass through a classifier to obtain a classification result for indicating whether the crowd density of the public place to be monitored exceeds a safety standard or not.
In particular, in the embodiment of the present application, in the case where the first scale feature map and the second scale feature map are obtained using convolution kernels of different sizes, the feature value set spatial position distributions of the first scale feature map and the second scale feature map may have a deviation, thereby easily causing sparsity of feature distribution of the differential feature map when the differential feature map is calculated.
Therefore, preferably, the difference profile is sparsely implicitly limited factor corrected, i.e.:
f is the characteristic value of the differential characteristic diagram, and
and the global characteristic value mean value of the differential characteristic diagram is obtained.
The sparsity implicit limiting factor correction sparsity-restricts implicit expressions of features through KL-like divergence forms to sparsity-restrict parameter spaces of the first convolutional neural network and the second convolutional neural network so as to improve average activity of an activation unit which infers expected characteristics during training of model parameters, thereby improving group optimization capability of the first scale feature map and the second scale feature map relative to each other and improving deviation of feature value set space position distribution of the first scale feature map and the second scale feature map. Therefore, the accuracy of monitoring the crowd density in public places is improved.
Based on the above, the application provides an artificial intelligence-based public place crowd density monitoring system and method, the system is characterized in that a crowd monitoring image collected by a camera deployed in a public place is coded through a first convolution neural network using a first convolution kernel to obtain a first scale feature map, the crowd monitoring image is coded through a second convolution kernel using a second convolution kernel to obtain a second scale feature map, then, a difference between the first scale feature map and the second scale feature map is calculated to obtain a difference feature map, then, feature values of all positions in the difference feature map are corrected to obtain a corrected difference feature map, and finally, the corrected difference feature map is passed through a classifier to obtain a classification result, and the classification result is used for indicating whether the crowd density of the public place to be monitored exceeds a safety standard. Therefore, the accuracy of monitoring the crowd density in public places is improved.
FIG. 1 illustrates a scene diagram of an artificial intelligence based public space population intensity monitoring method according to an embodiment of the application. As shown in fig. 1, in an application scenario of the present application, first, a crowd monitoring image collected by a camera (e.g., C in fig. 1) deployed in a public place (e.g., B in fig. 1) is obtained, and then, the crowd monitoring image is input into a server (e.g., S in fig. 1) deployed with an artificial intelligence-based public place crowd density monitoring algorithm, wherein the server can process the crowd monitoring image through the artificial intelligence-based public place crowd density monitoring algorithm to output a classification result indicating whether the crowd density of the public place to be monitored exceeds a safety standard.
Having described the general principles of the present application, various non-limiting embodiments of the present application will now be described with reference to the accompanying drawings.
Exemplary System
FIG. 2 illustrates a block diagram of an artificial intelligence based public space population intensity monitoring system according to an embodiment of the application.
As shown in fig. 2, an artificial intelligence based public space population intensity monitoring system 100 provided in the embodiment of the present application includes: the monitoring image acquisition module 110 is used for acquiring a crowd monitoring image through a camera deployed in a public place; a first scale encoding module 120, configured to pass the crowd monitoring image through a first convolution neural network using a first convolution kernel to obtain a first scale feature map; a second scale encoding module 130, configured to pass the crowd monitoring image through a second convolutional neural network using a second convolutional kernel to obtain a second scale feature map; a difference module 140, configured to calculate a difference between the first scale feature map and the second scale feature map to obtain a difference feature map; a correction module 150, configured to correct feature values at various positions in the difference feature map to obtain a corrected difference feature map; and the crowd density monitoring result generating module 160 is configured to pass the corrected differential feature map through a classifier to obtain a classification result, where the classification result is used to indicate whether the crowd density of the public place to be monitored exceeds a safety standard.
FIG. 3 illustrates an architectural diagram of an artificial intelligence based public space population intensity monitoring system according to an embodiment of the application. As shown in fig. 3, in the network architecture, firstly, a crowd monitoring image collected by a camera deployed in a public place is passed through a first convolution neural network using a first convolution kernel to obtain a first scale feature map, then the crowd monitoring image is passed through the first convolution neural network using the first convolution kernel to obtain a first scale feature map, and the crowd monitoring image is passed through a second convolution neural network using a second convolution kernel to obtain a second scale feature map; then, calculating the difference between the first scale feature map and the second scale feature map to obtain a difference feature map; then, correcting the characteristic value of each position in the differential characteristic diagram to obtain a corrected differential characteristic diagram; and finally, passing the corrected differential characteristic diagram through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the crowd density of the public place to be monitored exceeds a safety standard or not.
The following describes, by way of example, an artificial intelligence based public space population density monitoring system 100 for outputting a classification result indicating whether the population density of a public space to be monitored exceeds a safety standard.
Through research, the inventor of the application finds that most of the existing schemes are along the thinking of target detection and target quantity measurement in crowd density estimation, namely, how many human objects exist in an image are identified through target detection, whether the distribution density of the human objects in a preset public place exceeds a preset standard is determined through measurement, and if the distribution density exceeds the preset standard, an early warning that the crowd density is too dense is generated. However, in the actual operation process of the above detection scheme, the detection accuracy is difficult to meet the application requirement, because in the actual application scene, problems such as serious shielding and light change exist at the image acquisition end, and at the feature extraction end, because a human object belongs to a small-sized object in the image, confusion is easily generated by other objects.
Based on this, the present inventors tried to construct a population density monitoring scheme from relative indicators of population density. That is, if the population density of the public place is high, the feature distributions under different receptive fields are more uniform, and if the population density of the public place is low, the feature distributions under different receptive fields are more different. Therefore, a population density monitoring scheme can be constructed based on the difference of the feature distribution of the population monitoring image in different receptive fields.
Specifically, the monitoring image acquisition module 110 and the first scale encoding module 120 are configured to acquire a crowd monitoring image through a camera deployed in a public place, obtain a first scale feature map through a first convolutional neural network using a first convolutional kernel, and obtain a second scale feature map through a second convolutional neural network using a second convolutional kernel, where the crowd monitoring image is acquired through the second scale encoding module 130. Specifically, a crowd monitoring image is collected through a camera deployed in a public place. Then, the crowd monitoring image is passed through a first convolution neural network and a second convolution neural network using different convolution kernels (defined as a first convolution kernel and a second convolution kernel for convenience of explanation) to obtain a first feature map and a second feature map. It should be understood that for the convolutional neural network model, different convolution kernels represent different characteristic receptive fields, and therefore, the first feature map and the second feature map are used to represent feature representations of population density distribution under different receptive fields.
In an embodiment of the application, the first convolution kernel and the second convolution kernel may be solid convolution kernels of different sizes, for example, the size of the first convolution kernel is smaller than the size of the second convolution kernel. Of course, in other examples of the present application, the first convolution kernel and the second convolution kernel are hole convolution kernels, and a hole rate of the first convolution kernel is smaller than a hole rate of the second convolution kernel, that is, the first convolution kernel and the second convolution kernel have the same size but have different hole rates so that the first convolution neural network and the second convolution neural network have different receptive fields.
Specifically, the first scale encoding module 120 is further configured to: using the layers of the first convolutional neural network to perform, in forward passes of the layers, on input data:
performing convolution processing on the input data based on the first convolution kernel to obtain a convolution characteristic diagram;
performing global pooling processing based on a feature matrix on the convolution feature map to obtain a pooled feature vector; and
carrying out nonlinear activation on the characteristic value of each position in the pooled characteristic vector to obtain an activated characteristic vector;
wherein, the output of the last layer of the first convolutional neural network is the first scale feature map, and the input of the first layer of the first convolutional neural network is the crowd monitoring image.
That is, the first scale encoding module 120 is further configured to perform convolution processing, pooling processing and nonlinear activation processing based on a first convolution kernel on input data in forward direction transfer of layers using layers of the first convolutional neural network to output the first scale feature map from the last layer of the first convolutional neural network, wherein the first layer input of the first convolutional neural network is the monitored image from the crowd. In this way, the high-dimensional local behavior feature of the smaller-scale object in the crowd, namely the first-scale feature map, is extracted from the crowd monitoring image through the first convolution neural network based on the first convolution kernel.
Specifically, the second scale encoding module 130 is further configured to: performing, using layers of a second convolutional neural network, input data in forward passes of layers:
performing convolution processing on the input data based on the second convolution kernel to obtain a convolution characteristic diagram;
performing global pooling processing based on a feature matrix on the convolution feature map to obtain a pooled feature vector; and
carrying out nonlinear activation on the characteristic value of each position in the pooled characteristic vector to obtain an activated characteristic vector;
and outputting the last layer of the second convolutional neural network as the second scale feature map, and inputting the first layer of the second convolutional neural network as the crowd monitoring image.
That is, the second scale encoding module 130 is further configured to perform convolution processing, pooling processing and nonlinear activation processing based on a second convolution kernel on the input data in forward direction transfer of layers using layers of the second convolutional neural network respectively to output the second scale feature map from the last layer of the second convolutional neural network, where the first layer input of the second convolutional neural network is the monitored image from the crowd. In this way, the high-dimensional local behavior feature of the object with larger size in the crowd density, i.e. the second scale feature map, is extracted from the crowd monitoring image through the second convolution neural network based on the second convolution kernel.
The difference module 140 is configured to calculate a difference between the first scale feature map and the second scale feature map to obtain a difference feature map. Then, calculating the difference between the first characteristic diagram and the second characteristic diagram and enabling the obtained difference characteristic diagram to pass through a classifier to obtain a classification result for indicating whether the crowd density of the public place to be monitored exceeds a safety standard or not.
Specifically, the difference module 140 is further configured to calculate a difference by position between the first scale feature map and the second scale feature map to obtain the difference feature map;
wherein the formula is:
wherein, F d For the difference feature map, F 1 Is the first featureSymbolizing a picture, and F 2 Is the second characteristic diagram.
In particular, in the embodiment of the present application, in the case where the first scale feature map and the second scale feature map are obtained using convolution kernels of different sizes, the spatial position distribution of the feature value sets of the first scale feature map and the second scale feature map may have a deviation, thereby easily causing sparsity of the feature distribution of the differential feature map when the differential feature map is calculated.
Therefore, the correction module 150 is configured to correct the feature values of the positions in the differential feature map to obtain a corrected differential feature map. That is, the difference profile is sparsely implicitly restricted by factor correction.
Specifically, the correcting module 150 is further configured to correct the feature value of each position in the differential feature map to obtain a corrected differential feature map according to the following formula:
wherein f' is the corrected difference characteristic diagram, f is the characteristic value of the difference characteristic diagram, and
and the global characteristic value mean value of the differential characteristic diagram is obtained.
The sparsity implicit limiting factor correction sparsity-constrains implicit expressions of features through a KL-like divergence form to sparsity-limit parameter spaces of the first convolutional neural network and the second convolutional neural network so as to improve the average activity of an activation unit of a model parameter for deducing expected characteristics during training, thereby improving the group optimization capability of the first scale feature map and the second scale feature map relative to each other and improving the deviation of the position distribution of the feature value set spaces of the first scale feature map and the second scale feature map. Therefore, the accuracy of monitoring the crowd density in the public place is improved.
The crowd density monitoring result generating module 160 is configured to pass the corrected differential feature map through a classifier to obtain a classification result, where the classification result is used to indicate whether the crowd density of the public place to be monitored exceeds a safety standard.
In some embodiments of the present application, the process of classifying the crowd density monitoring result generating module 160 comprises: fully connecting and coding the corrected differential feature map by using a plurality of fully-connected layers of the classifier to convert the corrected differential feature map into classified feature vectors, and inputting the classified feature vectors into a Softmax classification function to obtain probability values of whether the population densities of the classified feature vectors respectively belonging to the public places to be monitored exceed safety standards; if the probability value that the crowd concentration of the public places to be monitored exceeds the safety standard is larger than or equal to the probability value that the crowd concentration of the public places to be monitored does not exceed the safety standard, outputting the classification result that the crowd concentration of the public places to be monitored exceeds the safety standard, and if the probability value that the crowd concentration of the public places to be monitored exceeds the safety standard is smaller than the probability value that the crowd concentration of the public places to be monitored does not exceed the safety standard, outputting the classification result that the crowd concentration of the public places to be monitored does not exceed the safety standard.
The population intensity monitoring result generating module 160 is further configured to: processing the corrected differential feature map using the classifier with the following formula to generate a classification result, wherein the formula is: softmax { (W) n ,B n 0:…:(W 1 ,B 1 ) L Project (F), where Project (F) represents the projection of the corrected differential feature map as a vector, W 1 To W n As a weight matrix for each fully connected layer, B 1 To B n A bias matrix representing the layers of the fully connected layer.
In summary, the artificial intelligence based public space population density monitoring system is illustrated in the embodiment of the present application, which encodes a population monitoring image collected by a camera deployed in a public space through a first convolution neural network using a first convolution kernel to obtain a first scale feature map, encodes the population monitoring image through a second convolution kernel using a second convolution kernel to obtain a second scale feature map, then calculates a difference between the first scale feature map and the second scale feature map to obtain a difference feature map, then corrects a feature value of each position in the difference feature map to obtain a corrected difference feature map, and finally passes the corrected difference feature map through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the population density of the public space to be monitored exceeds a safety standard. Therefore, the accuracy of monitoring the crowd density in public places is improved.
As described above, the artificial intelligence based public space population density monitoring system 100 according to the embodiment of the present application can be implemented in various terminal devices, such as a server for artificial intelligence based public space population density monitoring and the like. In one example, the artificial intelligence based public space population intensity monitoring system 100 according to embodiments of the present application may be integrated into a terminal device as one software module and/or hardware module. For example, the artificial intelligence based public space population intensity monitoring system 100 may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the artificial intelligence based public space population intensity monitoring system 100 could also be one of many hardware modules of the terminal device.
Alternatively, in another example, the artificial intelligence based public space population intensity monitoring system 100 and the terminal device may also be separate devices, and the artificial intelligence based public space population intensity monitoring system 100 may be connected to the terminal device via a wired and/or wireless network and transmit the interaction information in an agreed data format.
Exemplary method
FIG. 4 illustrates a flow chart of a method for artificial intelligence based public space population intensity monitoring in accordance with an embodiment of the application. As shown in fig. 4, the method for monitoring the crowd density in the public places based on artificial intelligence according to the embodiment of the application includes:
s101, collecting crowd monitoring images through a camera deployed in a public place;
s102, obtaining a first scale characteristic map by the crowd monitoring image through a first convolution neural network using a first convolution kernel;
s103, obtaining a second scale characteristic map by the crowd monitoring image through a second convolution neural network using a second convolution kernel;
s104, calculating the difference between the first scale feature map and the second scale feature map to obtain a difference feature map;
s105, correcting the characteristic value of each position in the differential characteristic diagram to obtain a corrected differential characteristic diagram; and
and S106, passing the corrected differential characteristic diagram through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the crowd density of the public place to be monitored exceeds a safety standard or not.
In one example, in the above artificial intelligence based public space population intensity monitoring method, the size of the first convolution kernel is smaller than the size of the second convolution kernel.
In one example, in the above artificial intelligence-based public space crowd density monitoring method, the first convolution kernel and the second convolution kernel are hole convolution kernels, and a hole rate of the first convolution kernel is smaller than a hole rate of the second convolution kernel.
In one example, in the above artificial intelligence based public space population density monitoring method, the passing the population monitoring image through a first convolution neural network using a first convolution kernel to obtain a first scale feature map includes: using the layers of the first convolutional neural network to perform, in forward passes of the layers, on input data:
performing convolution processing on the input data based on the first convolution kernel to obtain a convolution characteristic diagram;
performing global pooling processing based on a feature matrix on the convolution feature map to obtain a pooled feature vector; and
carrying out nonlinear activation on the characteristic value of each position in the pooled characteristic vector to obtain an activated characteristic vector;
wherein, the output of the last layer of the first convolutional neural network is the first scale feature map, and the input of the first layer of the first convolutional neural network is the crowd monitoring image.
In one example, in the above artificial intelligence based public space population density monitoring method, the passing the population monitoring image through a second convolutional neural network using a second convolutional kernel to obtain a second scale feature map includes: performing, using layers of a second convolutional neural network, on the input data in forward passes of the layers:
performing convolution processing on the input data based on the second convolution kernel to obtain a convolution characteristic diagram;
performing global pooling processing based on a feature matrix on the convolution feature map to obtain a pooled feature vector; and
carrying out nonlinear activation on the characteristic value of each position in the pooled characteristic vector to obtain an activated characteristic vector;
and outputting the last layer of the second convolutional neural network as the second scale feature map, and inputting the first layer of the second convolutional neural network as the crowd monitoring image.
In one example, in the above artificial intelligence-based public place population density monitoring method, the difference between the first scale feature map and the second scale feature map by position is calculated by the following formula to obtain the difference feature map;
wherein the formula is:
wherein, F d For the difference feature map, F 1 Is the first characteristic diagram, and F 2 Is a stand forThe second characteristic diagram.
In one example, in the above artificial intelligence-based public place population density monitoring method, the feature values of the positions in the differential feature map are corrected by the following formula to obtain a corrected differential feature map, where the formula is:
wherein f' is the corrected difference characteristic diagram, f is the characteristic value of the difference characteristic diagram, and
and the global characteristic value mean value of the differential characteristic diagram is obtained.
In one example, in the above artificial intelligence-based public space population density monitoring method, the passing the corrected differential feature map through a classifier to obtain a classification result includes:
processing the corrected differential feature map using the classifier with the following formula to generate a classification result, wherein the formula is: softmax { (W) n ,B n ):…:(W 1 ,B 1 ) L Project (F), where Project (F) represents the projection of the corrected differential feature map as a vector, W 1 To W n As a weight matrix for each fully connected layer, B 1 To B n A bias matrix representing the fully connected layers of each layer.
Here, it will be understood by those skilled in the art that the specific functions and steps in the above-described artificial intelligence based public space population density monitoring method have been described in detail in the above description of the artificial intelligence based public space population density monitoring system with reference to fig. 2 to 3, and thus, a repetitive description thereof will be omitted.
It is to be understood that some or all of the steps or operations in the above-described embodiments are merely examples, and other operations or variations of various operations may be performed by the embodiments of the present application. Further, the various steps may be performed in a different order presented in the above embodiments, and not all of the operations in the above embodiments may be performed.
Exemplary electronic device
Next, an electronic apparatus according to an embodiment of the present application is described with reference to fig. 5.
FIG. 5 illustrates a block diagram of an electronic device in accordance with an embodiment of the present application.
As shown in fig. 5, the electronic device 10 includes one or more processors 11 and memory 12.
The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
The input device 13 may include, for example, a keyboard, a mouse, and the like.
The output device 14 can output various information including classification results or warning prompts to the outside. The output devices 14 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for simplicity, only some of the components of the electronic device 10 relevant to the present application are shown in fig. 5, and components such as buses, input/output interfaces, and the like are omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.
Exemplary computer program product and computer-readable storage Medium
In addition to the methods and apparatus described above, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in an artificial intelligence based public space population intensity monitoring method according to various embodiments of the present application described in the "exemplary methods" section above of this specification.
The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present application may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps in the artificial intelligence based public space population intensity monitoring method according to various embodiments of the present application described in the "exemplary methods" section above in this specification.
The computer readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The basic principles of the present application have been described above with reference to specific embodiments, but it should be noted that advantages, effects, etc. mentioned in the present application are only examples and are not limiting, and the advantages, effects, etc. must not be considered to be possessed by various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is provided for purposes of illustration and understanding only, and is not intended to limit the application to the details which are set forth in order to provide a thorough understanding of the present application.
The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by one skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably herein. As used herein, the words "or" and "refer to, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
It should also be noted that in the devices, apparatuses, and methods of the present application, each component or step can be decomposed and/or re-combined. These decompositions and/or recombinations are to be considered as equivalents of the present application.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
Claims (10)
1. A public place crowd density monitoring system based on artificial intelligence, comprising:
the monitoring image acquisition module is used for acquiring a crowd monitoring image through a camera deployed in a public place;
the first scale coding module is used for enabling the crowd monitoring image to pass through a first convolution neural network using a first convolution kernel so as to obtain a first scale feature map;
the second scale coding module is used for enabling the crowd monitoring image to pass through a second convolution neural network using a second convolution kernel so as to obtain a second scale feature map;
the difference module is used for calculating the difference between the first scale feature map and the second scale feature map to obtain a difference feature map;
the correction module is used for correcting the characteristic values of all positions in the differential characteristic diagram to obtain a corrected differential characteristic diagram; and
and the crowd density monitoring result generating module is used for enabling the corrected differential feature map to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether the crowd density of the public place to be monitored exceeds a safety standard or not.
2. The artificial intelligence based public space population intensity monitoring system of claim 1, wherein a size of the first convolution kernel is smaller than a size of the second convolution kernel.
3. The artificial intelligence based public space population intensity monitoring system of claim 2, wherein the first convolution kernel and the second convolution kernel are hole convolution kernels, and a hole rate of the first convolution kernel is less than a hole rate of the second convolution kernel.
4. The artificial intelligence based public space population intensity monitoring system of claim 3, wherein the first scale encoding module is further configured to: using layers of a first convolutional neural network to perform, on input data, in forward passes of the layers:
performing convolution processing on the input data based on the first convolution core to obtain a convolution characteristic diagram;
performing global pooling processing based on a feature matrix on the convolution feature map to obtain a pooled feature vector; and
carrying out nonlinear activation on the characteristic value of each position in the pooled characteristic vector to obtain an activated characteristic vector;
wherein, the output of the last layer of the first convolutional neural network is the first scale feature map, and the input of the first layer of the first convolutional neural network is the crowd monitoring image.
5. The artificial intelligence based public space population intensity monitoring system of claim 4, wherein the second scale encoding module is further configured to: performing, using layers of a second convolutional neural network, on the input data in forward passes of the layers:
performing convolution processing on the input data based on the second convolution kernel to obtain a convolution characteristic diagram;
performing global pooling processing on the convolution feature map based on a feature matrix to obtain a pooled feature vector; and
carrying out nonlinear activation on the characteristic value of each position in the pooled characteristic vector to obtain an activated characteristic vector;
and outputting the last layer of the second convolutional neural network as the second scale feature map, and inputting the first layer of the second convolutional neural network as the crowd monitoring image.
6. The artificial intelligence based public space population intensity monitoring system of claim 5, wherein the difference module is further configured to calculate a difference by location between the first scale feature map and the second scale feature map to obtain the difference feature map with the following formula;
wherein the formula is:
wherein, F d For the difference feature map, F 1 Is the first characteristic diagram, and F 2 Is the second characteristic diagram.
7. The artificial intelligence based public space population intensity monitoring system of claim 6, wherein the correction module is further configured to correct feature values of each position in the differential feature map to obtain a corrected differential feature map according to the following formula:
wherein f' is the corrected difference characteristic diagram, f is the characteristic value of the difference characteristic diagram, and
and the global characteristic value mean value of the differential characteristic diagram is obtained.
8. The artificial intelligence based public space population intensity monitoring system of claim 7, wherein the population intensity monitoring result generation module is further configured to:
processing the corrected differential feature map using the classifier with the following formula to generate a classification result, wherein the formula is: softmax { (W) n ,B n ):…:(W 1 ,B 1 ) L Project (F), where Project (F) represents the projection of the corrected differential feature map as a vector, W 1 To W n As a weight matrix for all connected layers of each layer, B 1 To B n A bias matrix representing the fully connected layers of each layer.
9. A public place crowd density monitoring method based on artificial intelligence is characterized by comprising the following steps:
collecting a crowd monitoring image through a camera deployed in a public place;
passing the crowd monitoring image through a first convolution neural network using a first convolution kernel to obtain a first scale feature map;
passing the crowd monitoring image through a second convolutional neural network using a second convolutional kernel to obtain a second scale feature map;
calculating the difference between the first scale feature map and the second scale feature map to obtain a difference feature map;
correcting the characteristic value of each position in the differential characteristic diagram to obtain a corrected differential characteristic diagram; and
and passing the corrected differential characteristic diagram through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the crowd density of the public place to be monitored exceeds a safety standard or not.
10. The artificial intelligence based public space population intensity monitoring method of claim 9, wherein the feature values of each position in the differential feature map are corrected to obtain a corrected differential feature map by the following formula:
wherein f' is the corrected difference characteristic diagram, f is the characteristic value of the difference characteristic diagram, and
and the global characteristic value mean value of the differential characteristic diagram is obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211427991.2A CN115661758A (en) | 2022-11-15 | 2022-11-15 | Public place crowd density monitoring method and system based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211427991.2A CN115661758A (en) | 2022-11-15 | 2022-11-15 | Public place crowd density monitoring method and system based on artificial intelligence |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115661758A true CN115661758A (en) | 2023-01-31 |
Family
ID=85021375
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211427991.2A Pending CN115661758A (en) | 2022-11-15 | 2022-11-15 | Public place crowd density monitoring method and system based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115661758A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117011264A (en) * | 2023-08-03 | 2023-11-07 | 杭州高腾机电科技有限公司 | Underwear machine cluster production monitoring system and control method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734171A (en) * | 2017-04-14 | 2018-11-02 | 国家海洋环境监测中心 | A kind of SAR remote sensing image ocean floating raft recognition methods of depth collaboration sparse coding network |
| CN111985349A (en) * | 2020-07-30 | 2020-11-24 | 河海大学 | Radar received signal type classification and identification method and system |
| CN113392779A (en) * | 2021-06-17 | 2021-09-14 | 中国工商银行股份有限公司 | Crowd monitoring method, device, equipment and medium based on generation of confrontation network |
| CN114365180A (en) * | 2019-11-29 | 2022-04-15 | 奥林巴斯株式会社 | Image processing method, learning device, and image processing device |
| CN115063396A (en) * | 2022-07-11 | 2022-09-16 | 浙江金汇华特种耐火材料有限公司 | Preparation system and preparation method of long-life refractory brick |
-
2022
- 2022-11-15 CN CN202211427991.2A patent/CN115661758A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734171A (en) * | 2017-04-14 | 2018-11-02 | 国家海洋环境监测中心 | A kind of SAR remote sensing image ocean floating raft recognition methods of depth collaboration sparse coding network |
| CN114365180A (en) * | 2019-11-29 | 2022-04-15 | 奥林巴斯株式会社 | Image processing method, learning device, and image processing device |
| CN111985349A (en) * | 2020-07-30 | 2020-11-24 | 河海大学 | Radar received signal type classification and identification method and system |
| CN113392779A (en) * | 2021-06-17 | 2021-09-14 | 中国工商银行股份有限公司 | Crowd monitoring method, device, equipment and medium based on generation of confrontation network |
| CN115063396A (en) * | 2022-07-11 | 2022-09-16 | 浙江金汇华特种耐火材料有限公司 | Preparation system and preparation method of long-life refractory brick |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117011264A (en) * | 2023-08-03 | 2023-11-07 | 杭州高腾机电科技有限公司 | Underwear machine cluster production monitoring system and control method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115203380B (en) | Text processing system and method based on multi-mode data fusion | |
| CN115796173A (en) | Data processing method and system for supervision submission requirements | |
| CN111414987A (en) | Training method and training device for neural network and electronic equipment | |
| CN114782882B (en) | Video target behavior anomaly detection method and system based on multi-modal feature fusion | |
| CN115099684B (en) | Enterprise safety production management system and management method thereof | |
| CN115171317A (en) | Internet of things smart home method and system and electronic equipment | |
| CN115759658B (en) | Enterprise energy consumption data management system suitable for smart city | |
| CN115392369A (en) | Indoor air quality prediction system based on Internet of things and prediction method thereof | |
| CN116015837A (en) | Intrusion detection method and system for computer network information security | |
| CN115471216B (en) | Data management method of intelligent laboratory management platform | |
| CN115512420A (en) | Visitor management system and method thereof | |
| CN114548246A (en) | Control method and system of intelligent device and electronic device | |
| CN115410275A (en) | Office place personnel state detection method and system based on image recognition | |
| CN115144312A (en) | Indoor air fine particle measuring method and system based on Internet of things | |
| CN116247824A (en) | Control method and system for power equipment | |
| CN115661758A (en) | Public place crowd density monitoring method and system based on artificial intelligence | |
| CN114610488A (en) | Data acquisition method and system based on load balancing and electronic equipment | |
| CN116091414A (en) | Cardiovascular image recognition method and system based on deep learning | |
| CN116759053A (en) | Medical system prevention and control method and system based on Internet of things system | |
| CN115205788A (en) | Food material quality monitoring system | |
| CN115374822A (en) | Fault diagnosis method and system based on multi-level feature fusion | |
| CN114745168A (en) | Cloud platform inlet real-time flow copying method and system and electronic equipment | |
| CN113408674B (en) | Model training method and device, and image recognition method and device | |
| CN114884772A (en) | Bare computer vxlan deployment method, system and electronic equipment | |
| CN114612501A (en) | Neural network model training method and cryoelectron microscope density map resolution estimation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |