CN112668375B - Tourist distribution analysis system and method in scenic spot - Google Patents
Tourist distribution analysis system and method in scenic spot Download PDFInfo
- Publication number
- CN112668375B CN112668375B CN201910985165.1A CN201910985165A CN112668375B CN 112668375 B CN112668375 B CN 112668375B CN 201910985165 A CN201910985165 A CN 201910985165A CN 112668375 B CN112668375 B CN 112668375B
- Authority
- CN
- China
- Prior art keywords
- scenic spot
- information
- image
- tourist
- tourists
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention provides a tourist distribution analysis system and a tourist distribution analysis method in a scenic spot, which acquire images in a monitoring range through video equipment distributed in the scenic spot; processing the image information by using a method combining texture analysis of LBP and a high-efficiency classification mechanism of a convolutional neural network to obtain tourist density data; the tourist density is transmitted to a scenic spot map displayed by the cloud platform, and tourists in the scenic spot can learn the people flow distribution in the scenic spot through the mobile terminal by accessing the cloud platform, so that the tourists can select a route suitable for playing by themselves or through scenic spot navigation, and the occurrence of events affecting the tourist quality and safety of the tourists such as congestion, trampling and the like is reduced. According to the method and the system for analyzing the distribution of tourists in the scenic spot, the density of the tourists in the video monitoring area can be accurately obtained in real time based on the information collected by the video monitoring equipment in the scenic spot, and the scenic spot management is facilitated.
Description
Technical Field
The invention relates to the field of intelligent tourism research, in particular to a tourist distribution analysis system and method in scenic spots.
Background
By 2018, according to the domestic travel sampling investigation result, the domestic travel number of 55.39 hundred million people is increased by 10.8% compared with the same period of the last year. Wherein, the urban residents are increased by 12.0 percent for 41.19 hundred million times; the rural residents are increased by 7.3 percent for 14.20 hundred million times. Domestic travel income is 5.13 trillion yuan, and the annual contemporaneous growth is 12.3 percent. Wherein, urban residents spend 4.26 trillion yuan, and the increment is 13.1 percent; rural residents spend 0.87 trillion yuan, and the increase of the rural residents is 8.8 percent. The number of foreign tourists and the number of tourists in the night are respectively increased by 4.7 percent and 5.2 percent. The prosperity of the tourist market promotes the development of the tourist industry, and simultaneously, higher requirements are put forward on scenic spots and tourist cities, and how to effectively obtain tourist flow data prediction by utilizing data, predictively conduct guiding regulation and control of tourists, so that the service quality of the scenic spots is improved, and the tourist experience of the tourists is a major concern of developing intelligent tourism.
At present, when traveling in a busy season, the phenomenon that tourists are exploded and the passenger capacity of scenic spots is exceeded often occurs, so that the traveling experience of the tourists is poor, the distribution of the tourists in each area of the scenic spots is not clear when the tourists play in the scenic spots, the tourists are often piled up and travel, and then safety accidents are easily caused. Some public address devices are also arranged in the existing scenic spot, the public address devices are used for playing and notifying tourists in the scenic spot to keep away from danger when an emergency comes, and staff enter the scenic spot to conduct on-site command, but the two modes have poor emergency effects due to low familiarity of the tourists to the scenic spot, and accidents such as mutual trampling of the tourists occur again due to high-density panic passenger flow even in the process of guiding the tourists.
In order to solve the above problems, the national tourism authorities have issued notification requirements to establish a robust tourist flow control mechanism throughout the country, but the tourist areas still lack a perfect passenger flow prediction and peak regulation mechanism from the current development situation of the tourism industry throughout the country. Although the tourist flow prediction can start from ticket sales, the tourist flow prediction early warning data support is provided for tourists going to the scenic spot, but the tourist flow prediction early warning data support has no great guiding significance for tourists entering the scenic spot, so that the dynamic monitoring of the people flow distribution information of each area of the scenic spot is necessary to be implemented inside the scenic spot, the early warning and the dispersion of the passenger flow are carried out, the experience of the tourists is improved, and the public opinion of the scenic spot is reduced.
Disclosure of Invention
In order to overcome the above problems, the present inventors have made intensive studies to acquire images within a monitoring range by video recording apparatuses distributed in a scenic spot; processing the image information by using a method combining texture analysis of LBP and a high-efficiency classification mechanism of a convolutional neural network to obtain tourist density data; the tourist density is transmitted to a scenic spot map displayed by a cloud platform, and tourists in the scenic spot can learn the people flow distribution in the scenic spot through the mobile terminal by accessing the cloud platform, and then select a route suitable for playing by oneself or select a route suitable for playing by scenic spot navigation, so that the occurrence of events affecting tourist quality and safety of the tourists, such as congestion, trampling and the like, is reduced, and scenic spot management is facilitated, thereby completing the invention.
The invention provides a tourist distribution analysis system and method in scenic spots, and the technical scheme mainly comprises the following aspects:
in a first aspect, a system for analyzing distribution of guests within a attraction, the system comprising: cloud platform: the system is connected with a scenic spot scheduling system, realizes interaction with internal information of a scenic spot, accepts access of a mobile terminal, and feeds back a request result to the mobile terminal based on a relevant instruction requested by the mobile terminal;
scenic spot scheduling system: the system comprises an information acquisition subsystem, an information processing subsystem and an information storage subsystem; the information acquisition subsystem comprises image acquisition modules distributed in scenic spots, and the image acquisition modules transmit the acquired image information of tourists in the scenic spots to the information processing subsystem; the information processing subsystem obtains tourist density data through processing the original information; the information storage subsystem stores the image information obtained by the image acquisition module and the operation result of the information processing subsystem, records the geographic space data of all facilities in the scenic spot, provides information including scenic spot maps for the cloud platform, and is the basis of inquiry of the mobile terminal;
a mobile terminal: the information released by the scenic spot is obtained by accessing the cloud platform, and the position of the information in the scenic spot map is obtained by sending corresponding coordinate information to the cloud platform.
In a second aspect, a method for analyzing distribution of tourists in a scenic spot, the method comprising the steps of:
acquiring images in a monitoring range through image acquisition modules distributed in scenic spots;
the image acquisition module transmits the acquired image information of tourists in the scenic spot to the information processing subsystem;
the information processing subsystem is used for processing the image information to obtain tourist density data;
the information storage subsystem stores the image information obtained by the image acquisition module and the operation result of the information processing subsystem, records the geographic space data of all facilities in the scenic spot, and provides information including scenic spot map and tourist density for the cloud platform.
According to the method and the system for analyzing the distribution of tourists in the scenic spot, provided by the invention, the method and the system have the following beneficial effects:
according to the tourist distribution analysis method and system in the scenic spot, the image information is processed by combining the texture analysis of LBP and the efficient classification mechanism of the convolutional neural network, so that the tourist density data is obtained; the tourist density is transmitted to a scenic spot map displayed by the cloud platform, and tourists in the scenic spot can learn the people flow distribution in the scenic spot through the mobile terminal by accessing the cloud platform, so that the tourists can select a route suitable for playing by themselves or through scenic spot navigation. The method and the system can accurately obtain the density of tourists in the video monitoring area in real time, are in line with the development concept of intelligent tourism in China, improve the experience of the tourists during the tourism, and are beneficial to scenic spot management.
Drawings
FIG. 1 is a schematic diagram showing a tourist flow distribution analysis system in a scenic spot according to a preferred embodiment of the invention;
fig. 2 shows a structural diagram of an improved convolutional neural network in a preferred embodiment of the present invention.
Detailed Description
The invention is further described in detail below with reference to the accompanying drawings. The features and advantages of the present invention will become more apparent from the description.
As shown in fig. 1, according to a first aspect of the present invention, there is provided a tourist traffic distribution analysis system in a scenic spot, the analysis system comprising:
cloud platform: the system is connected with a scenic spot scheduling system, realizes interaction with internal information of a scenic spot, accepts access of a mobile terminal, and feeds back a request result to the mobile terminal based on a relevant instruction requested by the mobile terminal;
scenic spot scheduling system: the system comprises an information acquisition subsystem, an information processing subsystem and an information storage subsystem; the information acquisition subsystem comprises image acquisition modules distributed in scenic spots, and the image acquisition modules transmit the acquired image information of tourists in the scenic spots to the information processing subsystem; the information processing subsystem obtains tourist density data through processing the original information; the information storage subsystem stores the image information obtained by the image acquisition module and the operation result of the information processing subsystem, records the geographic space data of all facilities in the scenic spot, provides information including scenic spot map and tourist density for the cloud platform, and is the basis of inquiry of the mobile terminal; the scenic spot map is realized by a GIS technology.
A mobile terminal: the information released by the scenic spot is obtained by accessing the cloud platform, and the position of the information in the scenic spot map is obtained by sending corresponding coordinate information to the cloud platform.
The cloud platform comprises a scenic spot scheduling system link module, a mobile terminal link module, a scenic spot navigation module, a scenic spot introduction module and a third party service module, wherein the scenic spot scheduling system link module is used for establishing connection with a scenic spot scheduling system and realizing information interaction with the scenic spot scheduling system; the scenic spot scheduling system link module can be a communication protocol conversion interface meeting the information transmission of the scenic spot scheduling system;
the mobile terminal link module is used for establishing connection with the mobile terminal and realizing interaction with the mobile terminal; the mobile terminal link module can be a communication protocol conversion interface meeting the access of the mobile terminal;
the scenic spot navigation module is used for providing searching of tourist routes based on scenic spot maps and sequencing the tourist routes based on tourist needs; for example, the tourist location is taken as a radiation center, the scenic spots adjacent to the tourist are taken as a line starting point for line sorting, the sorting mode can be descending order by taking the heat index of the scenic spot nearest to the tourist as a parameter, or ascending order by taking the crowd concentration of the scenic spot nearest to the tourist as a parameter, and the like;
the scenic spot introduction module is used for providing introduction of scenic spots along the way of the line and comprises text, pictures, voice or video information;
and the third party service module provides an interface for information release and transaction for travel agencies, hotels, transportation service departments, payment application service departments and the like.
In the present invention, the mobile terminal may be any electronic product that can perform man-machine interaction with a guest through a keyboard, a touch screen, or a voice control device, for example: the mobile terminal has the advantages that the mobile terminals are various, and the interactivity and the experience are enhanced.
In the invention, analysis of tourist traffic in a scenic spot is implemented by a scenic spot scheduling system. The image acquisition module may be a hardware device for acquiring an image, such as a camera device or a video device, and is preferably a video device in view of convenience of long-term monitoring.
In the invention, the information processing subsystem identifies tourists through a classification algorithm, and then obtains the tourist volume, namely the density of the tourists, in the shooting range of the image acquisition module. It is known that the tourists can wear in different shapes, and the density of the tourists is measured through the overall human body recognition and is more difficult to train the model than the recognition of the local human body characteristics. Considering that the camera is generally arranged at a position higher than the head of the tourist, a more visual way to measure the density of the tourist is to measure the head number of the person in the image, however, the head of the tourist may not be shielded, and may be shielded by a hat or an umbrella, and head shielding has adverse effect on the accuracy of identifying the tourist in the image. According to the invention, through research, the convolutional neural network is adopted to identify tourists, the tourists of various colors are identified through the classification mechanism, and the similarity of the same target can be found through the propagation mechanism, so that the network can learn finer target information, and the accuracy of tourist detection is improved.
In the present invention, a method for an information processing subsystem to obtain guest density includes the steps of:
step 1), adding LBP (Local Binary Patterns, local binary pattern) feature extraction module branches on a convolutional neural network, deleting a pooling layer and all full-connection layers after the last layer of convolutional layer of the convolutional neural network, adding a convolutional layer with the channel number being the total number of tourist categories after the last layer of convolutional layer, and connecting a global average pooling layer and a softmax regression layer to obtain an improved convolutional neural network;
step 2), inputting pictures in a training sample set into an improved convolutional neural network, and calculating a target response relation based on original image textures by an LBP (location based point) feature extraction module; training to obtain a convolutional neural network for positioning through a target response relationship strengthening feature map based on textures;
step 3), inputting video images into the trained convolutional neural network, and extracting features on the feature map of the last convolutional layer by using an activating method to obtain a category response map;
and 4) up-sampling the category response map to the original image size, binarizing, detecting connected domains of the binarized map, wherein the connected domains with the actual physical quantity larger than the set value corresponding to the connected domains are a positioning result, and determining the density of tourists according to the number of the positioning results.
In step 1), the convolutional neural network comprises a plurality of convolutional layers, pooling layers which are arranged alternately with the convolutional layers after the convolutional layers, and full-connection layers which are positioned at the deepest layer of the model, and picture classification prediction results, namely whether tourists exist in pictures and the probability of all types of tourists, are obtained through full-connection layer analysis. The types of tourists comprise at least three types of head portraits of which the head portraits of tourists are unoccluded, head portraits with various caps or umbrella-shaped occlusion, and the like.
An important distinguishing feature of the head and the background environment of a person is texture information, however, the convolutional neural network ignores local structural features of an image when extracting image features, particularly when the image features are large in background condition, the image of a tourist is small, the convolutional neural network possibly confuses the head features of the person with objects in the background due to factors such as illumination, and more convolutional layer extraction features are needed to improve the recognition capability. As shown in fig. 2, the LBP feature extraction module is used for calculating and obtaining the fusion of the target response relation based on the texture and the feature of the convolutional neural network, so as to form the final feature representation of the tourist head image, which is beneficial to improving the sensitivity of the convolutional neural network to the local feature.
In the invention, the LBP characteristic extraction module can be combined into various convolutional neural networks and can be upgraded, the convolutional neural networks comprise VGG16, googLeNet and ResNet, and the convolutional neural networks adopt VGG16 or GoogLeNet with fewer convolutional layers due to the improvement of the classification capability by inserting the LBP characteristic extraction module.
In the invention, the LBP feature extraction module calculates the probability that each position may have a target based on the pixel values in the neighborhood in the original picture. In the training process of the convolutional neural network, the module can inhibit the interference of the background on the convolutional neural network learning by guiding the convolutional neural network to focus on the preselected position with higher occurrence probability of the learning target.
The full-connection layer calculates the frequency of each number in the weighted average number of the characteristic values output by each layer of convolution layer in the convolution neural network through the weight matrix to form a complete characteristic diagram, so that the classification function is realized. However, the full connection layer is to combine all the position information in the feature map generated by the convolution layer and output the combined position information, which is not direct information related to the number of visitors. Convolutional layers have excellent positioning capability, but positioning capability is lost after adding fully connected layers for classification, so we discard fully connected layers.
In the invention, a convolution layer with the channel number being the total number of tourist categories is added after the final convolution layer, and the characteristic map corresponding to the specific category is generated in the forward propagation process, so that the step of obtaining the category response map by carrying out characteristic map weighted sum through backward propagation is not required to be additionally added after the forward propagation, and the calculation cost is greatly reduced.
In the present invention, step 2) comprises the sub-steps of:
step 2.1), obtaining image data of tourists, determining image types, forming a training sample set, and inputting images in the training sample set into the improved convolutional neural network.
The image categories are the categories of pedestrians in the images, and the head portraits comprise head portraits without shielding and are provided with various hat or umbrella-shaped shielding head portraits. Considering that a large number of images are required to be used in model training, the position of each target is accurately marked in the large number of images, and the target identification and positioning information can be learned by means of a large amount of marking information, but the data marking has very high requirements. This process of precisely labeling each target in the image dataset is time-consuming and labor-consuming, which greatly affects the expansion of the algorithm on the massive data, limiting the utilization of the deep learning algorithm on the large dataset. Therefore, in the invention, images in a sample set are trained, and whether tourists and the identification of the category of the tourists exist in the given images is not noted, so that the positions of the tourists are not noted.
The training sample set image comprises a positive sample and a negative sample, the negative sample is an image without tourists, the positive sample is an image with tourists, and the positive sample comprises an image sample with only any single tourist type on the image. For example, the positive example includes a head portrait with no shade on the image, a head portrait with various caps, and a head portrait with umbrella shade.
In a preferred embodiment, the training sample set is preprocessed before it is input into the convolutional neural network, and the image is Gaussian smoothed to reduce image noise, and the image brightness is equalized by using a limited contrast adaptive histogram equalization algorithm. The method has the advantages that noise in the image can be effectively prevented from being enhanced in the process of brightness equalization, the noise removing difficulty is increased, and then the image quality is influenced.
And 2.2), the LBP characteristic extraction module calculates and obtains a target response relation based on the original image texture.
The LBP feature extraction module compares the gray values of the adjacent 8 pixels with the window center pixel as a threshold value in the window of 3*3, and if the surrounding pixel values are greater than the center pixel value, the position of the pixel is marked as 1, otherwise, the position is marked as 0. Thus, 8 points within the 3*3 field can generate an 8-bit unsigned number, i.e., the LBP value for the window is obtained and used to reflect the texture information for that region.
The LBP feature extraction module obtains a texture-based target response relationship by the following formulas (1) and (2):
wherein P is n For the central pixel value, P c N is 0,1,2, … …, which is the pixel value on the center neighborhood.
And 2.3), fusing the target response relation based on the texture with the feature map, continuing forward propagation, and training to obtain the target convolutional neural network.
To fuse the texture-based target response relationship with the feature map, the first convolutional layer of the convolutional neural network has a kernel 1*1.
During the forward propagation of the convolutional neural network, the texture-based target response relationship will act on the feature map (multiplied by the corresponding pixels on the feature map), suppressing background noise so that the neural network is focused on learning the target region.
In the present invention, the step 2) further includes an optimization process for the target convolutional neural network, which includes the steps of:
step 2.4) designing a loss function of multi-class cross entropy, calculating error of a classification result relative to the labeling according to the loss function, performing back transmission on the error in a gradient form, and updating the network parameters (convolution kernel weight parameters and additivity bias) of the convolution layer;
step 2.5) judging whether the network converges or not according to the classification error obtained by the loss function calculation, or judging whether the maximum iteration number is reached, stopping model training if the network converges or the maximum iteration number is reached, otherwise, jumping to step 2.1).
The method updates the model parameters of the convolutional neural network through gradient back-propagation, wherein the learning rate of the network is 0.005. The depth frame can calculate corresponding gradient values of different layers of the whole network according to the gradient of the loss function, and update parameters of the network according to the learning rate.
And 3) inputting video images into the trained convolutional neural network, and extracting features on the feature map of the last convolutional layer by using an activating method to obtain a category response map.
For category b (category b represents any category of guests), its category response diagram is defined as
M k Representing a feature map corresponding to the kth channel; the weight matrix corresponding to the last convolution layer is signed +.>
And (3) representing. The activation class response map is simply a linearly weighted sum of the different channel feature maps. And up-sampling the class response diagram to the original diagram size to obtain an image area which is meaningful for head portrait identification.
Because the tourist moves relative to the background in the camera, in order to further eliminate the influence of the background on target recognition, a background difference method is adopted to obtain candidate positions of the head region of the person under the image to be detected before the target convolutional neural network is input.
Specifically, the method for obtaining the candidate position of the head portrait area of the human head by adopting the background difference method comprises the following substeps:
sub-step 3.1), firstly extracting n images from the front end of the video, and extracting the background image of the image by using a time median filtering method.
In the invention, the value of n is between 30 and 50.
In the invention, the time median filtering method is to take the recording time as an axis and take the median value of the gray values of pixel points at the same coordinates in the extracted n images in sequence, and the obtained image is the background image.
In a preferred embodiment, the operation of extracting the background image of the image is performed once again at set intervals to reduce the influence of weather and light.
And 3.2), differentiating the currently extracted image with the background image to find a region of change (or movement) in the image, and obtaining a differential image. The following is equivalent to carrying out tourist identification based on the differential image.
First, n images f (x, y, t) are extracted from the video i ) I=1, 2, …, n, and a background image B (x, y, t) is obtained by temporal median filtering i ). The current image and the background image are differentiated to obtain a differential image D (x, y, t) i ) I.e.
D(x,y,t i )＝|f(x,y,t i )－D(x,y,t i ) I, i=1, 2, …, n. formula (3)
It is noted that when the difference processing is performed on the images in the video, the extraction of the images is sequentially performed from front to back according to the recording time. Since the time taken by a single video frame is extremely short, the extraction of n images from the video takes only a few seconds, which has little effect on the real-time computation of the density of tourists.
And 4) up-sampling the category response map to the original image size, binarizing, detecting connected domains of the binarized map, wherein the connected domains with the actual physical quantity larger than the set value corresponding to the connected domains are a positioning result, and determining the density of tourists according to the number of the positioning results.
In a preferred embodiment, a local maximum point and a region with higher score exist in the activated class response graph, and the local maximum point represents that the response value of the class at the position is stronger than that of the neighborhood, so that the class response graph is a potential target position; the region with the higher score is the most discernable region, i.e., the region where the target is located. Points in the category response graph that are greater than a certain threshold may form a connected domain for target guest counting.
The connected domain is calculated by using a connected domain search algorithm, which is a technology disclosed in the prior art and will not be described herein.
The head portrait of the person is known to have a certain area on the image, and the physical quantity corresponding to the area, namely the head size of the person, can be obtained through the corresponding relation between the pixels and the physical distance. Accordingly, an appropriate threshold is given to the head size of a person who sets the image capturing apparatus, the size of the area corresponding thereto in the image is determined based on the threshold, and the connected region smaller than the area corresponding thereto is regarded as the non-head region.
In the present invention, the guest density may be expressed in terms of the number of downstream guests per unit area.
In the invention, the density of tourists can be presented on the cloud platform in the form of numeric data, or the density of tourists is divided into four grades of sparse density, moderate density, saturated density and excessive density, and the density is presented on the cloud platform in the form of grade identification. Density sparseness means that the density of tourists under continuous multiframes or under specified interval frames is less than a set minimum value of tourists; the moderate density means that the density of tourists under continuous multiframes or at specified intervals is between a set minimum value and a set maximum allowable value; the density saturation means that the guest density is between the highest allowable value and the warning value under continuous multi-frames or under the frame of a specified interval; the density excess means that the guest density is greater than the alert value under the continuous multiframe or the specified interval frame. Wherein, the lowest value, the highest allowable value and the warning value are set according to the condition of the specific scenic spot.
Tourists in the scenic spot can learn the people flow distribution in the scenic spot through the mobile terminal by accessing the cloud platform, and then select the route suitable for playing by themselves or through the scenic spot navigation module. Workers in the scenic spot can learn the density of tourists in the scenic spot through a scenic spot scheduling system, issue information to the cloud platform, and timely implement people flow scheduling at the position where the tourists are concentrated, so that the occurrence of events affecting the tourist quality and safety of the tourists, such as congestion, trampling and the like, is reduced.
According to a second aspect of the present invention, there is provided a method of analyzing distribution of tourists in a scenic spot, the method comprising the steps of:
acquiring images in a monitoring range through image acquisition modules distributed in scenic spots;
the image acquisition module transmits the acquired image information of tourists in the scenic spot to the information processing subsystem;
the information processing subsystem is used for processing the image information to obtain tourist density data;
the information storage subsystem stores the image information obtained by the image acquisition module and the operation result of the information processing subsystem, records the geographic space data of all facilities in the scenic spot and provides information including scenic spot map and tourist density for the cloud platform;
staff in the scenic spot can learn the density of tourists in the scenic spot through the data obtained by the information processing subsystem, issue information to the cloud platform, and implement people flow scheduling in the tourist concentration place in time;
the tourists access the cloud platform to obtain tourist density information including the tourist density information issued by the scenic spot, and the position of the tourists in the scenic spot map can be obtained by sending corresponding coordinate information to the cloud platform.
Wherein, cloud platform exercise following function: establishing connection with a scenic spot scheduling system to realize information interaction with the scenic spot scheduling system;
establishing connection with the mobile terminal to realize interaction with the mobile terminal;
providing a search for travel routes based on the scenic spot map, and ordering the travel routes based on guest demand; for example, the tourist location is taken as a radiation center, the scenic spots adjacent to the tourist are taken as a line starting point for line sorting, the sorting mode can be descending order by taking the heat index of the scenic spot nearest to the tourist as a parameter, or ascending order by taking the crowd concentration of the scenic spot nearest to the tourist as a parameter, and the like;
providing introduction of the scenic spots along the route, wherein the introduction comprises text, pictures, voice or video information;
interfaces for implementing information release and transactions for travel agencies, hotels, transportation service departments, payment application service departments, and the like.
In the invention, the information processing subsystem identifies people through a classification algorithm, so as to obtain the tourist volume, namely the tourist density, in the shooting range of the image acquisition module. Specifically, the method for obtaining the guest density comprises the following steps:
step 1), adding LBP (Local Binary Patterns, local binary pattern) feature extraction module branches on a convolutional neural network, deleting a pooling layer and all full-connection layers after the last layer of convolutional layer of the convolutional neural network, adding a convolutional layer with the channel number being the total number of tourist categories after the last layer of convolutional layer, and connecting a global average pooling layer and a softmax regression layer to obtain an improved convolutional neural network;
step 2), inputting pictures in a training sample set into an improved convolutional neural network, and calculating a target response relation based on original image textures by an LBP (location based point) feature extraction module; training to obtain a convolutional neural network for positioning through a target response relationship strengthening feature map based on textures;
step 3), inputting video images into the trained convolutional neural network, and extracting features on the feature map of the last convolutional layer by using an activating method to obtain a category response map;
and 4) up-sampling the category response map to the original image size, binarizing, detecting connected domains of the binarized map, wherein the connected domains with the actual physical quantity larger than the set value corresponding to the connected domains are a positioning result, and determining the density of tourists according to the number of the positioning results.
Examples
Since there are no guest concentration data sets available at present, in this embodiment, two guest optical image data sets are made. Wherein, the data is from the video of the image of the Yihe garden 2017, and the screenshot tool is used for obtaining the optical image of the tourist. The two data sets are a training sample set and a test set, wherein the training sample set contains positive sample 5639 of tourists (the types of tourists cover head images without shielding, head images with various caps and head images with umbrella-shaped shielding), and the training sample set contains no negative sample 6766 of tourists; test set 1276, sample 953 with guest and sample 323 without guest. Training images in a sample set, and determining whether tourists and the identification of the category of the tourists exist in the given images without paying attention to the positions of the tourists.
Building a target convolutional neural network: adding LBP feature extraction module branches on a VGG16 and a GoogleNet of a convolutional neural network, deleting a pooling layer and all full-connection layers after the last convolutional layer of the convolutional neural network, adding a convolutional layer with the channel number of the total number of tourist categories (3) after the last convolutional layer, and connecting a global average pooling layer and a softmax regression layer to obtain an improved convolutional neural network;
inputting the pictures in the training sample set into the improved convolutional neural network, and calculating by an LBP (location based p) feature extraction module to obtain a target response relation based on original image textures; and (3) continuing forward propagation to obtain an image classification prediction result through a target response relation strengthening feature map based on texture, calculating an error between the prediction result and an image class label, and reversely propagating the error in a gradient form until convergence to obtain the target convolutional neural network.
Inputting a video image into a target convolutional neural network, and extracting features on a feature map of the last convolutional layer by using an activating method to obtain a category response map; and (3) up-sampling the class response graph to the original image size, performing binarization (the threshold value is the gray average value of the background image), then performing connected domain detection on the binarized graph, wherein the connected domain, corresponding to the connected domain, of which the actual physical quantity is larger than a set value is a positioning result, and determining the density of tourists according to the number of the positioning results.
When the test samples are detected, the identification is considered to be accurate when the number of tourists is 95% -105% of the number of accurate values, the detection accuracy of the Google Net convolutional neural network is 98.3%, and the detection accuracy of the VGG16 convolutional neural network is 97.1%, wherein the detection accuracy of the samples without tourists is 100% and 99.4% respectively.
The invention has been described above in connection with preferred embodiments, which are, however, exemplary only and for illustrative purposes. On this basis, the invention can be subjected to various substitutions and improvements, and all fall within the protection scope of the invention.
Claims (9)
1. A tourist distribution analysis system in a scenic spot, the system comprising:
cloud platform: the system is connected with a scenic spot scheduling system, realizes interaction with internal information of a scenic spot, accepts access of a mobile terminal, and feeds back a request result to the mobile terminal based on a relevant instruction requested by the mobile terminal;
scenic spot scheduling system: the system comprises an information acquisition subsystem, an information processing subsystem and an information storage subsystem; the information acquisition subsystem comprises image acquisition modules distributed in scenic spots, and the image acquisition modules transmit the acquired image information of tourists in the scenic spots to the information processing subsystem; the information processing subsystem obtains tourist density data through processing the original information; the information storage subsystem stores the image information obtained by the image acquisition module and the operation result of the information processing subsystem, records the geographic space data of all facilities in the scenic spot, provides information including scenic spot map and tourist density for the cloud platform, and is the basis of inquiry of the mobile terminal;
a mobile terminal: the method comprises the steps of accessing a cloud platform to obtain information released by scenic spots, and sending corresponding coordinate information to the cloud platform to obtain the positions of the information in a scenic spot map;
the method for obtaining the guest density by the information processing subsystem comprises the following steps:
step 1), adding LBP feature extraction module branches on a convolutional neural network, deleting a pooling layer and all full-connection layers after the last layer of convolutional layer of the convolutional neural network, adding a convolutional layer with the number of channels being the total number of tourist categories after the last layer of convolutional layer, and connecting a global average pooling layer and a softmax regression layer to obtain an improved convolutional neural network;
step 2), inputting pictures in a training sample set into an improved convolutional neural network, and calculating a target response relation based on original image textures by an LBP (location based point) feature extraction module; training to obtain a convolutional neural network for positioning through a target response relationship strengthening feature map based on textures;
the LBP characteristic extraction module calculates a texture-based target response relation through the following formula (1) and formula (2):
wherein P is n For the central pixel value, P c N is 0,1,2, … …, for pixel values on the central neighborhood;
step 3), inputting video images into the trained convolutional neural network, and extracting features on the feature map of the last convolutional layer by using an activating method to obtain a category response map;
and 4) up-sampling the category response map to the original image size, binarizing, detecting connected domains of the binarized map, wherein the connected domains with the actual physical quantity larger than the set value corresponding to the connected domains are a positioning result, and determining the density of tourists according to the number of the positioning results.
2. The system of claim 1, wherein the cloud platform comprises a scenic spot scheduling system linking module, a mobile terminal linking module, a scenic spot navigation module, a scenic spot introduction module, a third party service module, wherein,
the scenic spot scheduling system link module is used for establishing connection with the scenic spot scheduling system and realizing information interaction with the scenic spot scheduling system;
the mobile terminal link module is used for establishing connection with the mobile terminal and realizing interaction with the mobile terminal;
the scenic spot navigation module is used for providing searching of tourist routes based on scenic spot maps and sequencing the tourist routes based on tourist needs;
the scenic spot introduction module is used for providing introduction of scenic spots along the way of the line and comprises text, pictures, voice or video information;
and the third party service module provides an interface for information release and transaction for travel agencies, hotels, transportation service departments and payment application service departments.
3. The system of claim 1, wherein the mobile terminal is an electronic product capable of human-machine interaction with the guest through a keyboard, a touch screen or a voice control device;
the image acquisition module is a photographic device or a video device.
4. The system of claim 1, wherein in step 1), the convolutional neural network is selected from VGG16 or GoogLeNet.
5. The system according to claim 1, wherein step 2) comprises the sub-steps of:
step 2.1), obtaining image data of tourists, determining image types, forming a training sample set, and inputting images in the training sample set into an improved convolutional neural network;
step 2.2), the LBP characteristic extraction module calculates and obtains a target response relation based on original image textures;
and 2.3), fusing the target response relation based on the texture with the feature map, continuing forward propagation, and training to obtain the target convolutional neural network.
6. The system of claim 5, wherein step 2) further comprises an optimization process for the target convolutional neural network, the process comprising the steps of:
step 2.4) designing a loss function of multi-class cross entropy, calculating errors of classification results relative to labels according to the loss function, and updating the network parameters of the convolution layer by back-transmitting the errors in a gradient mode;
step 2.5) judging whether the network converges or not according to the classification error obtained by the loss function calculation, or judging whether the maximum iteration number is reached, stopping model training if the network converges or the maximum iteration number is reached, otherwise, jumping to step 2.1).
7. The system according to claim 1, wherein in step 3), the video image is preprocessed before being input into the target convolutional neural network, and the subsequent convolutional processing is performed with a differential image, and specifically comprising the steps of:
extracting n images from the front end of the video, and extracting background images of the images by using a time median filtering method, wherein the value of n is between 30 and 50;
and differentiating the currently extracted image with the background image to find a region of change or movement in the image, thereby obtaining a differential image.
8. The system according to claim 1, wherein in step 4), the connected domain is calculated using a connected domain search algorithm.
9. A method of analyzing distribution of tourists in a scenic spot, implemented by a system according to one of claims 1 to 8, comprising the steps of:
acquiring images in a monitoring range through image acquisition modules distributed in scenic spots;
the image acquisition module transmits the acquired image information of tourists in the scenic spot to the information processing subsystem;
the information processing subsystem is used for processing the image information to obtain tourist density data;
the information storage subsystem stores the image information obtained by the image acquisition module and the operation result of the information processing subsystem, records the geographic space data of all facilities in the scenic spot, and provides information including scenic spot map and tourist density for the cloud platform.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910985165.1A CN112668375B (en) | 2019-10-16 | 2019-10-16 | Tourist distribution analysis system and method in scenic spot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910985165.1A CN112668375B (en) | 2019-10-16 | 2019-10-16 | Tourist distribution analysis system and method in scenic spot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112668375A CN112668375A (en) | 2021-04-16 |
| CN112668375B true CN112668375B (en) | 2023-07-07 |
Family
ID=75400406
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910985165.1A Active CN112668375B (en) | 2019-10-16 | 2019-10-16 | Tourist distribution analysis system and method in scenic spot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112668375B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114125722B (en) * | 2021-11-21 | 2022-07-19 | 特斯联科技集团有限公司 | Intelligent management method and system for forest and grassland tourists |
| CN116029864A (en) * | 2022-12-29 | 2023-04-28 | 广州松麓圣方电子科技有限公司 | 5G+ cloud fusion global travel data management system |
| CN116542509A (en) * | 2023-06-21 | 2023-08-04 | 广东致盛技术有限公司 | Campus logistics task management method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105718947A (en) * | 2016-01-21 | 2016-06-29 | 吉林大学 | Lung cancer image fine classification method based on fusion of LBP and wavelet moment features |
| CN108629323A (en) * | 2018-05-11 | 2018-10-09 | 哈尔滨工业大学 | A kind of integrated providing method of scenic spot tourist chain type trip |
| CN109858388A (en) * | 2019-01-09 | 2019-06-07 | 武汉中联智诚科技有限公司 | A kind of intelligent tourism management system |
| CN109859170A (en) * | 2019-01-04 | 2019-06-07 | 中国矿业大学 | A kind of steel wire rope surface damage intelligent monitoring method and system based on LBP feature |
-
2019
- 2019-10-16 CN CN201910985165.1A patent/CN112668375B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105718947A (en) * | 2016-01-21 | 2016-06-29 | 吉林大学 | Lung cancer image fine classification method based on fusion of LBP and wavelet moment features |
| CN108629323A (en) * | 2018-05-11 | 2018-10-09 | 哈尔滨工业大学 | A kind of integrated providing method of scenic spot tourist chain type trip |
| CN109859170A (en) * | 2019-01-04 | 2019-06-07 | 中国矿业大学 | A kind of steel wire rope surface damage intelligent monitoring method and system based on LBP feature |
| CN109858388A (en) * | 2019-01-09 | 2019-06-07 | 武汉中联智诚科技有限公司 | A kind of intelligent tourism management system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112668375A (en) | 2021-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109858388A (en) | A kind of intelligent tourism management system | |
| CN112668375B (en) | Tourist distribution analysis system and method in scenic spot | |
| CN112561146A (en) | Large-scale real-time traffic flow prediction method based on fuzzy logic and depth LSTM | |
| CN109559302A (en) | Pipe video defect inspection method based on convolutional neural networks | |
| CN108334847A (en) | A kind of face identification method based on deep learning under real scene | |
| CN109766769A (en) | A kind of road target detection recognition method based on monocular vision and deep learning | |
| CN109214280A (en) | Shop recognition methods, device, electronic equipment and storage medium based on streetscape | |
| CN112287827A (en) | Complex environment pedestrian mask wearing detection method and system based on intelligent lamp pole | |
| CN109935080A (en) | The monitoring system and method that a kind of vehicle flowrate on traffic route calculates in real time | |
| CN109086803A (en) | A kind of haze visibility detection system and method based on deep learning and the personalized factor | |
| CN115775085A (en) | Smart city management method and system based on digital twin | |
| Xu et al. | Building height calculation for an urban area based on street view images and deep learning | |
| CN110473428A (en) | A kind of intelligent parking method, apparatus and system | |
| CN112738725B (en) | Real-time identification method, device, equipment and medium for target crowd in semi-closed area | |
| CN113256978A (en) | Method and system for diagnosing urban congestion area and storage medium | |
| CN110765900B (en) | Automatic detection illegal building method and system based on DSSD | |
| CN112836590A (en) | Flood disaster monitoring method and device, electronic equipment and storage medium | |
| CN111897993A (en) | Efficient target person track generation method based on pedestrian re-recognition | |
| CN111726535A (en) | Smart city CIM video big data image quality control method based on vehicle perception | |
| CN110163081A (en) | Regional invasion real-time detection method, system and storage medium based on SSD | |
| Sun et al. | Automatic building age prediction from street view images | |
| CN115439933A (en) | Garbage classification release site detection method based on multiple model recognition strategies | |
| CN114708485A (en) | Method for acquiring flood disaster information from social media | |
| CN114529721A (en) | Urban remote sensing image vegetation coverage identification method based on deep learning | |
| CN113128559A (en) | Remote sensing image target detection method based on cross-scale feature fusion pyramid network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |