CN112668375A - System and method for analyzing tourist distribution in scenic spot - Google Patents
System and method for analyzing tourist distribution in scenic spot Download PDFInfo
- Publication number
- CN112668375A CN112668375A CN201910985165.1A CN201910985165A CN112668375A CN 112668375 A CN112668375 A CN 112668375A CN 201910985165 A CN201910985165 A CN 201910985165A CN 112668375 A CN112668375 A CN 112668375A
- Authority
- CN
- China
- Prior art keywords
- scenic spot
- image
- information
- tourist
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention provides a tourist distribution analysis system and method in a scenic spot, wherein images in a monitoring range are obtained through video equipment distributed in the scenic spot; processing the image information by using a method of combining texture analysis of LBP (local binary pattern) and an efficient classification mechanism of a convolutional neural network to obtain tourist density data; the tourist density is transmitted to a scenic spot map displayed by a cloud platform, and tourists in the scenic spot can learn the pedestrian flow distribution in the scenic spot by accessing the cloud platform through a mobile terminal, and then select routes suitable for playing by themselves or through scenic spot navigation, so that the occurrence of events which affect the tourism quality and safety of the tourists, such as congestion, treading and the like, is reduced. By the method and the system for analyzing the tourist distribution in the scenic spot, the density of the tourists in the video monitoring area can be accurately obtained in real time based on the information acquired by the video monitoring equipment in the scenic spot, and the scenic spot management is facilitated.
Description
Technical Field
The invention relates to the field of intelligent tourism research, in particular to a tourist distribution analysis system and method in a scenic spot.
Background
By the end of 2018, the number of domestic tourists is 55.39 hundred million people according to the domestic tourism sampling survey result, and is increased by 10.8 percent compared with the current year. Wherein, urban residents have 41.19 hundred million people times, and the number is increased by 12.0 percent; rural residents have 14.20 hundred million people times and the growth is 7.3 percent. The domestic travel income is 5.13 trillion yuan, and the annual contemporaneous increase is 12.3%. Wherein, urban residents spend 4.26 trillion yuan, which increases 13.1%; rural residents spend 0.87 trillion yuan, which increases 8.8%. The number of the foreign people entering the border tour and the number of the people entering the border tour overnight are respectively increased by 4.7 percent and 5.2 percent. The prosperity of the tourism market promotes the development of the tourism industry, simultaneously, higher requirements are provided for scenic spots and tourist cities, the data is utilized to effectively obtain tourist flow data prediction, and the tourist is guided and controlled in a predictive manner, so that the service quality of the scenic spots is improved, and the tourism experience of the tourists is improved to develop the problem of important attention of smart tourism.
At present when the tourism season, often take place the visitor and explode full, surpass the phenomenon of the capacity of the scenic spot, lead to visitor's tourism to experience relatively poorly, also not clear the visitor distribution in each region in the scenic spot when the visitor plays in the scenic spot moreover, often tie up and advance, and then lead to taking place the incident easily. Some public address equipment is arranged in some current scenic spots, and is used for playing and informing tourists in the guided scenic spots to get away from danger when an emergency comes, and workers also enter the scenic spots to conduct on-site commanding, but because the familiarity of the tourists with the scenic spots is not high, the emergency effect of the two modes is not good enough, and accidents such as trampling of the tourists and the like occur again due to high-density panic passenger flow sometimes even in the process of guiding the tourists.
In order to solve the problems, the national tourist bureau sends out a notice to require that a robust scenic spot tourist flow control mechanism is established everywhere, but from the development situation of the tourism industry all over the country at present, a scenic spot lacks a perfect passenger flow prediction and peak value regulation mechanism. Although the tourist flow prediction can start from ticket sales, and the tourist about to go to the scenic spot is provided with tourist flow prediction early warning data support, but the tourist has little guiding significance to the tourist who has entered the scenic spot, therefore inside the scenic spot, it is necessary to implement the dynamic monitoring of each regional people flow distribution information of the scenic spot, and early warning and leading are carried out to the tourist flow in advance, thereby promoting the experience of the tourist and reducing the public sentiments which are not beneficial to the scenic spot.
Disclosure of Invention
In order to overcome the above problems, the present inventors have conducted intensive studies to acquire images within a monitoring range by video recording devices distributed in a scenic spot; processing the image information by using a method of combining texture analysis of LBP (local binary pattern) and an efficient classification mechanism of a convolutional neural network to obtain tourist density data; the density of the tourists is transmitted to a scenic spot map displayed by a cloud platform, the tourists in the scenic spot can obtain the pedestrian flow distribution in the scenic spot by accessing the cloud platform through a mobile terminal, and then select routes suitable for playing by self or through scenic spot navigation, so that the occurrence of events which influence the tourism quality and safety of the tourists, such as congestion, treading and the like, is reduced, the scenic spot management is facilitated, and the invention is completed.
The invention provides a tourist distribution analysis system and a method in a scenic spot, and the technical scheme mainly comprises the following aspects:
in a first aspect, a system for analyzing tourist distribution in a scenic spot, the system comprising: cloud platform: the system is connected with a scenic spot scheduling system, realizes interaction with information in the scenic spot, receives access of the mobile terminal, and feeds back a request result to the mobile terminal based on a related instruction requested by the mobile terminal;
scenic spot scheduling system: the system comprises an information acquisition subsystem, an information processing subsystem and an information storage subsystem; the information acquisition subsystem comprises image acquisition modules distributed in the scenic spot, and the image acquisition modules transmit the acquired image information of the tourists in the scenic spot to the information processing subsystem; the information processing subsystem obtains tourist density data by processing the original information; the information storage subsystem stores the image information obtained by the image acquisition module and the operation result of the information processing subsystem, records the geographic space data of each facility in the scenic spot, provides information including scenic spot maps for the cloud platform, and is the basis for the query of the mobile terminal;
a mobile terminal: the method comprises the steps of obtaining information published by a scenic spot by accessing a cloud platform, and obtaining the position of the information in a scenic spot map by sending corresponding coordinate information to the cloud platform.
In a second aspect, a method for analyzing tourist distribution in a scenic spot, the method comprises the following steps:
acquiring images in a monitoring range through image acquisition modules distributed in a scenic spot;
the image acquisition module transmits the acquired image information of the tourists in the scenic spot to the information processing subsystem;
the information processing subsystem obtains tourist density data by processing the image information;
the information storage subsystem stores the image information obtained by the image acquisition module and the operation result of the information processing subsystem, records the geographic space data of each facility in the scenic spot, and provides information including scenic spot maps and tourist density for the cloud platform.
According to the method and the system for analyzing the tourist distribution in the scenic spot, provided by the invention, the following beneficial effects are achieved:
according to the method and the system for analyzing the tourist distribution in the scenic spot, image information is processed by combining texture analysis of LBP (local binary pattern) and an efficient classification mechanism of a convolutional neural network, so that tourist density data is obtained; the density of the tourists is transmitted to a scenic spot map displayed by a cloud platform, and the tourists in the scenic spot can learn the pedestrian flow distribution in the scenic spot by accessing the cloud platform through a mobile terminal, and then select routes suitable for playing by themselves or through scenic spot navigation. The method and the system can accurately obtain the density of tourists in the video monitoring area in real time, conform to the development concept of intelligent tourism in China, improve the experience of the tourists during the tourism and are beneficial to scenic spot management.
Drawings
FIG. 1 is a schematic diagram of a system for analyzing the distribution of tourist traffic in a scenic spot according to a preferred embodiment of the present invention;
fig. 2 shows a structure diagram of an improved convolutional neural network in a preferred embodiment of the present invention.
Detailed Description
The invention is explained in further detail below with reference to the drawing. The features and advantages of the present invention will become more apparent from the description.
As shown in fig. 1, according to a first aspect of the present invention, there is provided an in-scenic spot tourist traffic distribution analysis system including:
cloud platform: the system is connected with a scenic spot scheduling system, realizes interaction with information in the scenic spot, receives access of the mobile terminal, and feeds back a request result to the mobile terminal based on a related instruction requested by the mobile terminal;
scenic spot scheduling system: the system comprises an information acquisition subsystem, an information processing subsystem and an information storage subsystem; the information acquisition subsystem comprises image acquisition modules distributed in the scenic spot, and the image acquisition modules transmit the acquired image information of the tourists in the scenic spot to the information processing subsystem; the information processing subsystem obtains tourist density data by processing the original information; the information storage subsystem stores the image information obtained by the image acquisition module and the operation result of the information processing subsystem, records the geographic space data of each facility in the scenic spot, provides information including scenic spot maps and tourist density for the cloud platform, and is the basis for inquiring the mobile terminal; wherein, the scenic spot map is realized by a GIS technology.
A mobile terminal: the method comprises the steps of obtaining information published by a scenic spot by accessing a cloud platform, and obtaining the position of the information in a scenic spot map by sending corresponding coordinate information to the cloud platform.
The cloud platform comprises a scenic spot scheduling system link module, a mobile terminal link module, a scenic spot navigation module, a scenic spot introduction module and a third-party service module, wherein the scenic spot scheduling system link module is used for establishing connection with a scenic spot scheduling system and realizing information interaction with the scenic spot scheduling system; the scenic spot scheduling system link module can be a communication protocol conversion interface meeting information transmission of the scenic spot scheduling system;
the mobile terminal linking module is used for establishing connection with the mobile terminal and realizing interaction with the mobile terminal; the mobile terminal linking module can be a communication protocol conversion interface meeting the access of the mobile terminal;
the scenic spot navigation module is used for providing tourist routes for searching based on a scenic spot map and sequencing the tourist routes based on the requirements of tourists; for example, the positions of the tourists are used as radiation centers, the scenic spots adjacent to the tourists are used as starting points of routes for route sorting, and the sorting mode can be that the hot indexes of the scenic spots most adjacent to the tourists are used as parameters for descending sorting, or the crowd density of the scenic spots most adjacent to the tourists is used as parameters for ascending sorting, and the like;
the scenic spot introduction module is used for providing introduction of scenic spots along the route, wherein the introduction comprises text, pictures, voice or video information;
and the third-party service module provides an interface for information release and transaction for a travel agency, a hotel, a transportation service department, a payment application service department and the like.
In the present invention, the mobile terminal may be any electronic product capable of performing human-computer interaction with a guest through a keyboard, a touch screen, or a voice control device, for example: the mobile terminal has the advantages that the mobile terminal is diversified, and interactivity and experience are enhanced.
In the invention, the scenic spot dispatching system is used for analyzing the tourist flow in the scenic spot. The image acquisition module may be a hardware device for acquiring images, such as a camera device or a video device, and is preferably a video device in view of convenience of long-term monitoring.
In the invention, the information processing subsystem identifies the tourists through a classification algorithm, and then obtains the amount of the tourists in the shooting range of the image acquisition module, namely the density of the tourists. It is known that the physical and physical wearing of tourists is different, and the difficulty of training a model is higher when the density of the tourists is measured through the whole human body recognition compared with the model obtained through the recognition of the local human body features. Considering that the camera is generally arranged higher than the head of the tourist, the more intuitive way to measure the density of the tourist is to measure the number of the head of the person in the image, however, the head of the tourist may not be shielded, and may be shielded by various caps or umbrellas, and the head shielding also has adverse effect on the accuracy of the tourist identification in the image. According to the invention, through research and discovery, the convolutional neural network is adopted to identify the tourists, the tourists of various colors are identified through the classification mechanism, and the similarity of the same target can be found through the propagation mechanism, so that the network can learn more precise target information, and the accuracy of the detection of the tourists is improved.
In the invention, the method for acquiring the density of the tourists by the information processing subsystem comprises the following steps:
step 1), adding an LBP (Local Binary pattern) feature extraction module branch on a convolutional neural network, deleting a pooling layer and all full-connection layers after the last layer of a convolutional layer of the convolutional neural network, adding a convolutional layer with the channel number being the total number of tourist classes after the last layer of the convolutional layer, and then connecting a global average pooling layer and a softmax regression layer to obtain an improved convolutional neural network;
step 2), inputting the images in the training sample set into an improved convolutional neural network, and calculating by an LBP (local binary pattern) feature extraction module to obtain a target response relation based on the original image texture; training to obtain a convolutional neural network for positioning through a target response relation strengthening feature map based on textures;
step 3), inputting a video image into the trained convolutional neural network, and performing feature extraction on the feature map of the last convolutional layer by using an activation method to obtain a category response map;
and 4), up-sampling the category response image to the size of the original image, then carrying out binarization, and then carrying out connected domain detection on the binarized image, wherein the connected domain with the actual physical quantity greater than a set value corresponding to the connected domain is a positioning result, and the density of the tourists is determined according to the number of the positioning results.
In the step 1), the convolutional neural network comprises a plurality of convolutional layers, pooling layers which are arranged at intervals with the convolutional layers after the convolutional layers, and fully-connected layers which are arranged at the deepest layer of the model, and the picture classification prediction result, namely the probability of whether the picture contains tourists and various types of tourists, is obtained through analysis of the fully-connected layers. The types of the tourists comprise at least three types, namely head portraits without shelters, head portraits with various caps or umbrella shelters, and the like.
An important distinguishing feature of the human head and the background environment is texture information, however, the convolutional neural network ignores the local structural feature of the image when extracting the image feature, particularly, when under a larger background condition, the image of the tourist is small, the convolutional neural network may confuse the human head feature with the object in the background due to factors such as illumination, and more convolutional layers are needed to extract the feature to improve the recognition capability. As shown in fig. 2, the LBP feature extraction module calculates a texture-based target response relationship to be fused with the features of the convolutional neural network to form a final feature representation of the guest head image, which is beneficial to improving the sensitivity of the convolutional neural network to local features.
In the invention, the LBP feature extraction module can be combined into various convolutional neural networks and can upgrade the convolutional neural networks, wherein the convolutional neural networks comprise VGG16, GoogLeNet and ResNet, and the convolutional neural networks only adopt VGG16 or GoogLeNet with fewer convolution layers because the insertion of the LBP feature extraction module improves the classification capability.
In the invention, the LBP feature extraction module calculates and obtains the probability that each position may have the target based on the pixel value in the neighborhood in the original picture. In the training process of the convolutional neural network, the module can inhibit the interference of the background on the learning of the convolutional neural network by guiding the convolutional neural network to focus on the preselected position with higher occurrence probability of the learning target.
The full-connection layer calculates the frequency of each number in the weighted average number through the weight matrix according to the eigenvalue output by each layer of convolutional layer in the convolutional neural network to form a complete characteristic diagram, and the classification function is realized. However, the full link layer is not direct information on the number of guests, because all the position information in the feature map generated by the convolutional layer is combined and output. The convolutional layer has excellent positioning ability, but the positioning ability is lost after the addition of the fully-connected layer for classification, and thus, we discard the fully-connected layer.
In the invention, a convolutional layer with the channel number being the total number of the guest classes is added after the last convolutional layer, and the feature map corresponding to the specific class is generated in the forward propagation process, so that the step of performing feature map weighted sum through backward propagation to obtain the class response map is not required to be additionally added after the forward propagation, and the calculation cost is greatly reduced.
In the present invention, step 2) comprises the following substeps:
and 2.1) acquiring image data of the tourist, determining image types, forming a training sample set, and inputting the images in the training sample set into the improved convolutional neural network.
The image category is the category of the pedestrians in the image, and the head portrait comprises an unshaded head portrait and head portraits with various caps or umbrella-shaped shades. Considering that a large number of images are required in model training, and the position of each target is accurately labeled in the large number of images, although the target identification and positioning information can be learned by means of a large number of labeled information, the labeling of data is very demanding. The process of accurately labeling each target in the image data set is time-consuming and labor-consuming, so that the expansion of the algorithm on massive data is influenced to a great extent, and the utilization of a large data set by a deep learning algorithm is limited. For this reason, in the present invention, images in the sample set are trained, given whether there are guests and the identity of the class of guests in the images, without noting the location of the guests.
The images in the training sample set comprise positive examples and negative examples, wherein the negative examples are images without tourists, the positive examples are images with tourists, and the positive examples comprise image samples with images only containing any single type of tourists. For example, the positive examples include only an unobstructed head portrait, only a head portrait with various caps, and only an umbrella-shaped obstructed head portrait on an image.
In a preferred embodiment, before the training sample set is input into the convolutional neural network, the picture is preprocessed, the image is gaussian smoothed to reduce the image noise, and the image brightness is equalized by using a contrast-limited adaptive histogram equalization algorithm. The advantages of denoising and then brightness equalization are that the enhancement of noise existing in the image in the process of brightness equalization can be effectively prevented, the denoising difficulty is increased, and further the image quality is influenced.
And 2.2), calculating by an LBP (local binary pattern) feature extraction module to obtain an original image texture-based target response relation.
The LBP characteristic extraction module takes a window center pixel as a threshold value in a window of 3 x 3, the gray values of 8 adjacent pixels are compared with the threshold value, if the values of the surrounding pixels are larger than the value of the center pixel, the position of the pixel point is marked as 1, and if not, the position is 0. Thus, 8 points in the 3 x 3 domain can generate 8bit unsigned numbers, i.e., the LBP value of the window, and use this value to reflect the texture information of the region.
The LBP feature extraction module obtains a target response relation based on the texture through the following formulas (1) and (2):
wherein, PnIs the central pixel value, PcN is 0,1,2, … … 7 for pixel values on the center neighborhood.
And 2.3) fusing the target response relation based on the texture with the feature map, continuing forward propagation, and training to obtain the target convolutional neural network.
To fuse the texture-based target response relationship with the feature map, the kernel of the first convolutional layer of the convolutional neural network is 1 × 1.
In the forward propagation process of the convolutional neural network, the target response relation based on the texture acts on the feature map (is multiplied by corresponding pixels on the feature map), and background noise is suppressed, so that the neural network focuses on learning the target region.
In the present invention, the step 2) further includes an optimization process of the target convolutional neural network, the process includes the following steps:
step 2.4) designing a loss function of the multi-class cross entropy, calculating the error of the classification result relative to the label according to the loss function, reversely transmitting the error in a gradient form, and updating the network parameters (convolution kernel weight parameters and additive bias) of the convolution layer;
and 2.5) judging whether the network is converged or not according to the classification error obtained by calculating the loss function, or judging whether the maximum iteration times are reached or not, stopping model training if the network is converged or the maximum iteration times are reached, and otherwise, skipping to the step 2.1).
The method updates the model parameters of the convolutional neural network through gradient back transmission, wherein the learning rate of the network is 0.005. The depth framework can calculate corresponding gradient values of different layers of the whole network according to the gradient of the loss function, and update parameters of the network according to the learning rate.
And 3) inputting a video image into the trained convolutional neural network, and performing feature extraction on the feature map of the last convolutional layer by using an activation method to obtain a class response map.
For category b (category b represents any one of the categories of guests), the category response graph is defined as
MkRepresenting a characteristic diagram corresponding to the kth channel; sign for weight matrix corresponding to last convolutional layer
And (4) showing. The activation category response map is simply a linear weighted sum of the different channel feature maps. The category response image is up-sampled to the size of the original image, so that an image area which is meaningful for head image recognition can be obtained.
Because the tourist moves relative to the background in the image pickup, in order to further eliminate the influence of the background on target recognition, the candidate position of the human head region under the image to be detected is obtained by adopting a background difference method before the target convolutional neural network is input.
Specifically, the method for obtaining the candidate position of the head portrait region of the person by adopting the background subtraction method comprises the following sub-steps:
substep 3.1), firstly, extracting n images from the front end of the video, and extracting a background image of the images by using a time median filtering method.
In the invention, the value of n is between 30 and 50.
In the invention, the time median filtering method is to take the median of gray values of pixel points at the same coordinate in the extracted n images in sequence by taking the recording time as an axis, and the obtained image is the background image.
In a preferred embodiment, the operation of extracting the background image of the image is performed again at set time intervals to reduce the influence of weather and light.
Substep 3.2) differentiating the currently extracted image and the background image to find a changed (or moving) region in the image to obtain a differential image. The subsequent process is equivalent to guest identification based on the difference image.
First, n images f (x, y, t) are extracted from the videoi) And i is 1,2, …, n, and a background image B (x, y, t) is obtained by adopting time median filteringi). Differentiating the current image and the background image to obtain a differential image D (x, y, t)i) I.e. by
D(x,y,ti)＝|f(x,y,ti)－D(x,y,ti) 1,2, …, n. formula (3)
It should be noted that, when the difference processing is performed on the images in the video, the images are extracted sequentially from front to back according to the recording time. Because the time occupied by a single video frame is extremely short, only a few seconds of time are occupied for extracting n images from the video, and the real-time calculation of the tourist density is hardly influenced.
And 4), up-sampling the category response image to the size of the original image, then carrying out binarization, and then carrying out connected domain detection on the binarized image, wherein the connected domain with the actual physical quantity greater than a set value corresponding to the connected domain is a positioning result, and the density of the tourists is determined according to the number of the positioning results.
In a preferred embodiment, a local maximum point and a region with higher score exist in the activated category response map, and the local maximum point represents that the response value of the category at the position is stronger than that of the neighborhood, and is most likely to be a potential target position; the region with the higher score is the most discriminating region, i.e. the region where the object is located. Points in the category response graph that are greater than a certain threshold may form a connected domain for the target guest count.
The connected component is calculated by using a connected component search algorithm, which is a technology disclosed in the prior art and is not described herein again.
It is known that a head portrait of a person has a certain area on an image, and a physical quantity corresponding to the area of the area, that is, the size of the head of the person, can be obtained through a corresponding relationship between pixels and physical distances. Accordingly, an appropriate threshold value is given to the head size of the person under the imaging apparatus, the size of the area of the region corresponding thereto in the image is determined based on the threshold value, and a connected component smaller than the area of the region corresponding to the threshold value is regarded as a non-head region.
In the present invention, the guest density may be expressed in terms of the number of guests per unit area.
In the invention, the density of the tourists can be presented on the cloud platform by numerical data, or the density of the tourists is divided into four levels of sparse density, moderate density, saturated density and excessive density, and is presented on the cloud platform by level identification. The density sparseness means that the density of the downstream guests under continuous multiple frames or specified interval frames is less than the set lowest value of the guests; the medium density means that the downstream passenger density of continuous multiframe or specified interval frames is between the set lowest value and the set highest allowable value; the density saturation means that the downstream passenger density under continuous multiframes or specified interval frames is between the highest allowable value and the warning value; the density excess means that the downstream passenger density under continuous multiframe or specified interval frames is greater than a warning value. Wherein, the lowest value, the highest allowable value and the warning value are set according to the condition of a specific scenic spot.
Visitors in the scenic spot can learn the pedestrian flow distribution in the scenic spot by accessing the cloud platform through the mobile terminal, and then select routes suitable for playing by themselves or through a scenic spot navigation module. Staff in the scenic spot can learn tourist density everywhere in the scenic spot through scenic spot dispatch system, issue information to the cloud platform to in time implement the stream of people scheduling in the concentrated place of visitor, reduce the emergence of influence visitor's tourism quality and safety such as block up, trample.
According to a second aspect of the present invention, there is provided a tourist distribution analyzing method in a scenic spot, comprising the steps of:
acquiring images in a monitoring range through image acquisition modules distributed in a scenic spot;
the image acquisition module transmits the acquired image information of the tourists in the scenic spot to the information processing subsystem;
the information processing subsystem obtains tourist density data by processing the image information;
the information storage subsystem stores the image information obtained by the image acquisition module and the operation result of the information processing subsystem, records the geographic space data of each facility in the scenic spot and provides information including scenic spot maps and tourist density for the cloud platform;
the staff in the scenic spot can obtain the density of the tourists at each position of the scenic spot through the data obtained by the information processing subsystem, release information to the cloud platform and implement people flow scheduling at the position where the tourists are concentrated in time;
the tourist can obtain the tourist density information released by the scenic spot by accessing the cloud platform, and can obtain the position of the tourist in the scenic spot map by sending corresponding coordinate information to the cloud platform.
Wherein, the cloud platform performs the following functions: establishing connection with a scenic spot scheduling system to realize information interaction with the scenic spot scheduling system;
establishing connection with a mobile terminal to realize interaction with the mobile terminal;
providing a search of the tourist routes based on the scenic spot map, and sequencing the tourist routes based on the demands of tourists; for example, the positions of the tourists are used as radiation centers, the scenic spots adjacent to the tourists are used as starting points of routes for route sorting, and the sorting mode can be that the hot indexes of the scenic spots most adjacent to the tourists are used as parameters for descending sorting, or the crowd density of the scenic spots most adjacent to the tourists is used as parameters for ascending sorting, and the like;
providing introduction of scenic spots along the route, wherein the introduction comprises text, pictures, voice or video information;
an interface for information distribution and transaction for travel agencies, hotels, transportation services, payment application services, etc.
In the invention, the information processing subsystem identifies people through a classification algorithm, and then obtains the amount of tourists in the shooting range of the image acquisition module, namely the density of the tourists. Specifically, the method for obtaining the density of the tourists comprises the following steps:
step 1), adding an LBP (Local Binary pattern) feature extraction module branch on a convolutional neural network, deleting a pooling layer and all full-connection layers after the last layer of a convolutional layer of the convolutional neural network, adding a convolutional layer with the channel number being the total number of tourist classes after the last layer of the convolutional layer, and then connecting a global average pooling layer and a softmax regression layer to obtain an improved convolutional neural network;
step 2), inputting the images in the training sample set into an improved convolutional neural network, and calculating by an LBP (local binary pattern) feature extraction module to obtain a target response relation based on the original image texture; training to obtain a convolutional neural network for positioning through a target response relation strengthening feature map based on textures;
step 3), inputting a video image into the trained convolutional neural network, and performing feature extraction on the feature map of the last convolutional layer by using an activation method to obtain a category response map;
and 4), up-sampling the category response image to the size of the original image, then carrying out binarization, and then carrying out connected domain detection on the binarized image, wherein the connected domain with the actual physical quantity greater than a set value corresponding to the connected domain is a positioning result, and the density of the tourists is determined according to the number of the positioning results.
Examples
Since there is no guest concentration data set currently available, in this embodiment, two guest optical image data sets are made. Wherein, the data is from the video shot in 2017 of the Yihe garden, and the optical image of the visitor is obtained by utilizing a screenshot tool. The two data sets are a training sample set and a test set, wherein the training sample set contains 5639 positive samples of tourists (the types of the tourists comprise head portraits without shelters, head portraits with various caps and head portraits with umbrella shelters), and does not contain 6766 negative samples of the tourists; the test was focused on 1276 samples with guests 953 samples and 323 samples without guests. The training sample sets images, given whether there are guests and the identity of the class of guests in the images, without noting the location of the guests.
Constructing a target convolutional neural network: adding LBP (local binary pattern) feature extraction module branches on a convolutional neural network VGG16 and a GoogleNet, deleting a pooling layer and all full-connected layers after the last convolutional layer of the convolutional neural network, adding a convolutional layer with the channel number being the total number of tourist classes (3) after the last convolutional layer, and then connecting a global average pooling layer and a softmax regression layer to obtain an improved convolutional neural network;
inputting the pictures in the training sample set into an improved convolutional neural network, and calculating by an LBP (local binary pattern) feature extraction module to obtain a target response relation based on the texture of the original image; and (3) enhancing the feature map through a target response relation based on the texture, continuing forward propagation to obtain an image classification prediction result, calculating an error between the prediction result and the image category label, and performing backward propagation on the error in a gradient form until convergence to obtain the target convolutional neural network.
Inputting a video image into a target convolutional neural network, and performing feature extraction on a feature map of the last convolutional layer by using an activation method to obtain a class response map; and (3) up-sampling the category response image to the size of the original image, then carrying out binarization (the threshold value is the gray average value of the background image), then carrying out connected domain detection on the binarized image, wherein the connected domain of which the actual physical quantity is greater than the set value and which corresponds to the connected domain is a positioning result, and determining the density of the tourists according to the number of the positioning results.
When a test sample is detected, when the number of tourists is 95% -105% of the number of accurate values, the identification is considered to be accurate, the detection accuracy rate of the GoogleNet convolutional neural network reaches 98.3%, the detection accuracy rate of the VGG16 convolutional neural network reaches 97.1%, and the detection accuracy rates of samples without tourists are 100% and 99.4% respectively.
The present invention has been described above in connection with preferred embodiments, but these embodiments are merely exemplary and merely illustrative. On the basis of the above, the invention can be subjected to various substitutions and modifications, and the substitutions and the modifications are all within the protection scope of the invention.
Claims (10)
1. A system for analyzing the distribution of visitors in a scenic spot, the system comprising:
cloud platform: the system is connected with a scenic spot scheduling system, realizes interaction with information in the scenic spot, receives access of the mobile terminal, and feeds back a request result to the mobile terminal based on a related instruction requested by the mobile terminal;
scenic spot scheduling system: the system comprises an information acquisition subsystem, an information processing subsystem and an information storage subsystem; the information acquisition subsystem comprises image acquisition modules distributed in the scenic spot, and the image acquisition modules transmit the acquired image information of the tourists in the scenic spot to the information processing subsystem; the information processing subsystem obtains tourist density data by processing the original information; the information storage subsystem stores the image information obtained by the image acquisition module and the operation result of the information processing subsystem, records the geographic space data of each facility in the scenic spot, provides information including scenic spot maps and tourist density for the cloud platform, and is the basis for inquiring the mobile terminal;
a mobile terminal: the method comprises the steps of obtaining information published by a scenic spot by accessing a cloud platform, and obtaining the position of the information in a scenic spot map by sending corresponding coordinate information to the cloud platform.
2. The system of claim 1, wherein the cloud platform comprises a scenic spot scheduling system linking module, a mobile terminal linking module, a scenic spot navigation module, a scenic spot introduction module, and a third party service module, wherein,
the system comprises a scenic spot scheduling system linking module, a scenic spot scheduling system linking module and a scene management module, wherein the scenic spot scheduling system linking module is used for establishing connection with a scenic spot scheduling system and realizing information interaction with the scenic spot scheduling system;
the mobile terminal linking module is used for establishing connection with the mobile terminal and realizing interaction with the mobile terminal;
the scenic spot navigation module is used for providing tourist routes for searching based on a scenic spot map and sequencing the tourist routes based on the requirements of tourists;
the scenic spot introduction module is used for providing introduction of scenic spots along the route, wherein the introduction comprises text, pictures, voice or video information;
and the third-party service module provides an interface for information release and transaction for a travel agency, a hotel, a transportation service department, a payment application service department and the like.
3. The system according to claim 1, wherein the mobile terminal is any electronic product capable of performing man-machine interaction with the guest through a keyboard, a touch screen or a voice control device;
the image acquisition module is a camera device or a video device.
4. The system of claim 1, wherein the method of the information processing subsystem obtaining guest density comprises the steps of:
step 1), adding an LBP (local binary pattern) feature extraction module branch on a convolutional neural network, deleting a pooling layer and all full-connection layers after the last convolutional layer of the convolutional neural network, adding a convolutional layer with the number of channels being the total number of classes of tourists after the last convolutional layer, and then connecting a global average pooling layer and a softmax regression layer to obtain an improved convolutional neural network;
step 2), inputting the images in the training sample set into an improved convolutional neural network, and calculating by an LBP (local binary pattern) feature extraction module to obtain a target response relation based on the original image texture; training to obtain a convolutional neural network for positioning through a target response relation strengthening feature map based on textures;
step 3), inputting a video image into the trained convolutional neural network, and performing feature extraction on the feature map of the last convolutional layer by using an activation method to obtain a category response map;
and 4), up-sampling the category response image to the size of the original image, then carrying out binarization, and then carrying out connected domain detection on the binarized image, wherein the connected domain with the actual physical quantity greater than a set value corresponding to the connected domain is a positioning result, and the density of the tourists is determined according to the number of the positioning results.
5. The system of claim 4, wherein in step 1), the convolutional neural network is selected from VGG16 or GoogleLeNet.
6. The system according to claim 4, characterized in that step 2) comprises the following sub-steps:
step 2.1), acquiring image data of the tourist, determining image types, forming a training sample set, and inputting images in the training sample set into an improved convolutional neural network;
step 2.2), an LBP feature extraction module calculates to obtain an original image texture-based target response relation;
and 2.3) fusing the target response relation based on the texture with the feature map, continuing forward propagation, and training to obtain the target convolutional neural network.
7. The system of claim 6, wherein step 2) further comprises an optimization process for the target convolutional neural network, the process comprising the steps of:
step 2.4) designing a loss function of the multi-class cross entropy, calculating the error of the classification result relative to the label according to the loss function, reversely transmitting the error in a gradient form, and updating the network parameters (convolution kernel weight parameters and additive bias) of the convolution layer;
and 2.5) judging whether the network is converged or not according to the classification error obtained by calculating the loss function, or judging whether the maximum iteration times are reached or not, stopping model training if the network is converged or the maximum iteration times are reached, and otherwise, skipping to the step 2.1).
8. The system according to claim 4, wherein in step 3), the video image is preprocessed before being input into the target convolutional neural network, and the difference image is used for subsequent convolutional processing, which specifically comprises the following steps:
extracting n images from the front end of a video, and extracting a background image of the images by using a time median filtering method, wherein the value of n is between 30 and 50;
and (3) carrying out difference on the currently extracted image and the background image to find a changed or moved area in the image so as to obtain a difference image.
9. The system of claim 4, wherein in step 4), the connected component is calculated using a connected component search algorithm.
10. Method for analysing the distribution of visitors in a scenic spot, carried out by a system according to one of claims 1 to 8, comprising the following steps:
acquiring images in a monitoring range through image acquisition modules distributed in a scenic spot;
the image acquisition module transmits the acquired image information of the tourists in the scenic spot to the information processing subsystem;
the information processing subsystem obtains tourist density data by processing the image information;
the information storage subsystem stores the image information obtained by the image acquisition module and the operation result of the information processing subsystem, records the geographic space data of each facility in the scenic spot, and provides information including scenic spot maps and tourist density for the cloud platform.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910985165.1A CN112668375B (en) | 2019-10-16 | 2019-10-16 | Tourist distribution analysis system and method in scenic spot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910985165.1A CN112668375B (en) | 2019-10-16 | 2019-10-16 | Tourist distribution analysis system and method in scenic spot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112668375A true CN112668375A (en) | 2021-04-16 |
| CN112668375B CN112668375B (en) | 2023-07-07 |
Family
ID=75400406
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910985165.1A Active CN112668375B (en) | 2019-10-16 | 2019-10-16 | Tourist distribution analysis system and method in scenic spot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112668375B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114125722A (en) * | 2021-11-21 | 2022-03-01 | 特斯联科技集团有限公司 | Intelligent management method and system for forest and grassland tourists |
| CN116029864A (en) * | 2022-12-29 | 2023-04-28 | 广州松麓圣方电子科技有限公司 | 5G+ cloud fusion global travel data management system |
| CN116542509A (en) * | 2023-06-21 | 2023-08-04 | 广东致盛技术有限公司 | Campus logistics task management method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105718947A (en) * | 2016-01-21 | 2016-06-29 | 吉林大学 | Lung cancer image fine classification method based on fusion of LBP and wavelet moment features |
| CN108629323A (en) * | 2018-05-11 | 2018-10-09 | 哈尔滨工业大学 | A kind of integrated providing method of scenic spot tourist chain type trip |
| CN109858388A (en) * | 2019-01-09 | 2019-06-07 | 武汉中联智诚科技有限公司 | A kind of intelligent tourism management system |
| CN109859170A (en) * | 2019-01-04 | 2019-06-07 | 中国矿业大学 | A kind of steel wire rope surface damage intelligent monitoring method and system based on LBP feature |
-
2019
- 2019-10-16 CN CN201910985165.1A patent/CN112668375B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105718947A (en) * | 2016-01-21 | 2016-06-29 | 吉林大学 | Lung cancer image fine classification method based on fusion of LBP and wavelet moment features |
| CN108629323A (en) * | 2018-05-11 | 2018-10-09 | 哈尔滨工业大学 | A kind of integrated providing method of scenic spot tourist chain type trip |
| CN109859170A (en) * | 2019-01-04 | 2019-06-07 | 中国矿业大学 | A kind of steel wire rope surface damage intelligent monitoring method and system based on LBP feature |
| CN109858388A (en) * | 2019-01-09 | 2019-06-07 | 武汉中联智诚科技有限公司 | A kind of intelligent tourism management system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114125722A (en) * | 2021-11-21 | 2022-03-01 | 特斯联科技集团有限公司 | Intelligent management method and system for forest and grassland tourists |
| CN114125722B (en) * | 2021-11-21 | 2022-07-19 | 特斯联科技集团有限公司 | Intelligent management method and system for forest and grassland tourists |
| CN116029864A (en) * | 2022-12-29 | 2023-04-28 | 广州松麓圣方电子科技有限公司 | 5G+ cloud fusion global travel data management system |
| CN116542509A (en) * | 2023-06-21 | 2023-08-04 | 广东致盛技术有限公司 | Campus logistics task management method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112668375B (en) | 2023-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108388888B (en) | Vehicle identification method and device and storage medium | |
| WO2021088300A1 (en) | Rgb-d multi-mode fusion personnel detection method based on asymmetric double-stream network | |
| CN109858388A (en) | A kind of intelligent tourism management system | |
| CN108596277B (en) | Vehicle identity recognition method and device and storage medium | |
| CN106998444B (en) | Big data face monitoring system | |
| CN109743683B (en) | Method for determining position of mobile phone user by adopting deep learning fusion network model | |
| CN112668375B (en) | Tourist distribution analysis system and method in scenic spot | |
| US20230334890A1 (en) | Pedestrian re-identification method and device | |
| CN110675408A (en) | High-resolution image building extraction method and system based on deep learning | |
| CN106845373A (en) | Towards pedestrian's attribute forecast method of monitor video | |
| CN110263717A (en) | It is a kind of incorporate streetscape image land used status determine method | |
| CN109086803B (en) | Deep learning and personalized factor-based haze visibility detection system and method | |
| CN109935080A (en) | The monitoring system and method that a kind of vehicle flowrate on traffic route calculates in real time | |
| CN108198202A (en) | A kind of video content detection method based on light stream and neural network | |
| WO2024051067A1 (en) | Infrared image processing method, apparatus, and device, and storage medium | |
| Huang et al. | Comprehensive urban space representation with varying numbers of street-level images | |
| CN110929739B (en) | Automatic impervious surface range remote sensing iterative extraction method | |
| CN112488165A (en) | Infrared pedestrian identification method and system based on deep learning model | |
| CN111897993A (en) | Efficient target person track generation method based on pedestrian re-recognition | |
| CN110163081A (en) | Regional invasion real-time detection method, system and storage medium based on SSD | |
| CN116503209A (en) | Digital twin system based on artificial intelligence and data driving | |
| CN115527118A (en) | Remote sensing image target detection method fused with attention mechanism | |
| Bianchi et al. | A view from the track: Measuring spatial quality of slow mobility routes. Possible integration of GIS and machine learning based methods | |
| CN114529721A (en) | Urban remote sensing image vegetation coverage identification method based on deep learning | |
| CN114898215A (en) | Automatic arrangement method of sound barrier |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |