CN117456588A

CN117456588A - Wisdom museum user management system

Info

Publication number: CN117456588A
Application number: CN202311762892.4A
Authority: CN
Inventors: 林明凯; 杨先亭
Original assignee: Fujian Xianxing Network Service Co ltd
Current assignee: Fujian Xianxing Network Service Co ltd
Priority date: 2023-12-20
Filing date: 2023-12-20
Publication date: 2024-01-26
Anticipated expiration: 2043-12-20
Also published as: CN117456588B

Abstract

The invention belongs to the technical field of information processing, and discloses a user management system of an intelligent museum; the system comprises a user data input module, a user identification module and a user identification module, wherein the user data input module is used for identifying user identity information; the user data processing module is used for generating a user characteristic label according to the user identity information; the intelligent control module comprises a route planning unit, a route matching unit and a data feedback unit; and is connected with the user data processing module; the route planning unit is used for planning the exhibition area into n visit routes; the line matching unit is used for selecting an optimal visiting route according to the user characteristic tag; transmitting the optimal visit route to the mobile equipment of the user through a network, and transmitting the user identity information and the corresponding visit route to a cloud platform server; the technical means such as artificial intelligence, computer vision, operation optimization and the like are fully utilized to establish an accurate, intelligent and strong-adaptability user model, so that differentiated and personalized user management is realized.

Description

Wisdom museum user management system

Technical Field

The invention relates to the technical field of information processing, in particular to a user management system of an intelligent museum.

Background

Patent publication number CN108197961a discloses a user management method and device. The method comprises the following steps: determining an account having an association with the first account; determining a first number N according to the number of accounts associated with the first account, wherein N is a positive integer; a first user equity combination comprising N user equities is returned to the first account. The method and the device have the advantages that the account which is associated with the first account is determined, the first quantity N is determined according to the quantity of the accounts which are associated with the first account, and the first user right combination containing N user rights is returned to the first account, so that each account which is associated with the account can be managed by one account, and the interactivity among users can be improved.

Patent application publication number CN111709773a discloses a user management method, a user management device, a server, and a computer-readable storage medium. The method comprises the following steps: determining acquisition data based on the acquisition behavior of the user on the designated resource; determining a target authority matched with the acquired data; and aiming at the appointed resource, opening the target authority to the user. By the scheme, personalized authority management of the user can be realized, and the acquisition intention of the user to the resource is improved.

The phenomenon of "one-touch" of the service is common in the museum visit service today; the lack of an effective user management system in the museum results in that operators cannot accurately identify the identity information and basic characteristics of different visitors; when a family contains various types of people (such as middle school students with grandpa milk, white collar couples with children, etc.), they have different preferences in terms of visiting environment, content, path, etc., but the museum can hardly distinguish their differences and can only uniformly recommend a route; in reality, the activity track and residence time distribution of the users in the whole visiting process cannot be effectively recorded and analyzed; when a certain area is blocked or is cold clear, the people flow distribution situation is difficult to judge and adjust; for example, if people flow cannot be quickly allocated when a particularly popular area of children is blocked, children can only roll and wait in a narrow space; more importantly, as the data of different types of users cannot be collected and analyzed, the restaurant side cannot improve the route design, the exhibition area arrangement and the like, and the visiting experience centering on the users is difficult to build; the visit satisfaction of users cannot be effectively improved, and further improvement of the operation level and the service quality of the museum is severely restricted.

In view of the above, the present invention proposes a smart museum user management system to solve the above-mentioned problems.

Disclosure of Invention

In order to overcome the defects in the prior art, the invention provides the following technical scheme for achieving the purposes: a smart museum user management system, comprising: the user data input module is used for identifying user identity information;

the user data processing module is used for generating a user characteristic label according to the user identity information;

the intelligent control module comprises a route planning unit, a route matching unit and a data feedback unit; and is connected with the user data processing module;

the route planning unit is used for planning the exhibition area into n visit routes; the line matching unit is used for selecting an optimal visiting route according to the user characteristic tag; transmitting the optimal visit route to the mobile equipment of the user through a network, and transmitting the user identity information and the corresponding visit route to a cloud platform server; all the modules are connected in a wired and/or wireless mode, so that data transmission among the modules is realized.

Further, the user identity information comprises facial image characteristics, user movement tracks and residence time distribution characteristics;

The user performs face scanning registration at the entrance of the museum and collects face images; extracting the features of the face image by using a preset feature extraction model;

setting m cameras along the way in a user visit area, collecting a position image of a user, inputting the position image of the user into a preset regression analysis model, and obtaining a user movement track;

determining a position point for setting a camera according to the plane view of the exhibition hall; the selection of the position points considers the turning position, intersection and exhibition area entrance positions of the route;

each position point is provided with 1 camera, and m cameras are arranged in total; the shooting angle of the camera is downward overlook; collecting an image when a user appears in a shooting image of a camera;

positioning users appearing in each acquired image through an image recognition algorithm, and acquiring space coordinates (x, y) of the users; taking a timestamp of an image shot by the camera as a user occurrence time t; if the same user is identified in different cameras, checking the characteristics to judge that the user is the same person; labeling the position coordinates (X, Y) of each camera on the exhibition hall plan; matching the appearance coordinates of the user with the marked camera coordinates; if a user appears in the fields of view of the camera 1 and the camera 2 in sequence, judging that the user moves from the position of the camera 1 to the position of the camera 2; combining the distance between two camera positions on the plan view And the user appears in the two cameras with time difference +.>Calculating the moving speed of the user>；

Connecting W identified sample points (x, y, t) in series, and drawing a user movement track by combining the movement speed of the user;

acquiring a layout structure and exhibit distribution of a user visit area; according to the layout structure and the exhibit distribution of the user visit area, a Markov random field model is constructed and used as priori constraint knowledge of user behavior analysis; and combining the prior constraint knowledge with the user movement track, and calculating the residence time distribution characteristics of each visit area by using Bayesian reasoning.

Further, the construction mode of the Markov random field model comprises the following steps:

equally dividing the whole user activity area into Q grids, defining Q random variables Xo, and representing the user access state of each grid; when the user enters the o-th grid, xo=1, otherwise xo=0; setting the energy value of the state change of the adjacent areas as 1 and the non-adjacent areas as 0 according to the actual distance between the reference areas; building an energy function from spatial distances between pixelsThe method comprises the steps of carrying out a first treatment on the surface of the Defining a gibbs distribution based on an energy function, wherein +.>Is a distribution normalization factor;

establishing a three-layer conditional random field; the three-layer conditional random field comprises a bottom layer, a middle layer and a high layer; the bottom layer represents position data, the middle layer represents a behavior mode, and the high layer represents user intention;

Marking K groups of user track data samples from a museum monitoring video;

training and learning parameters of the Markov random field model by using K groups of user track data samples, so that the output probability distribution of the Markov random field model is matched with the actual distribution of the user;

further, the calculation mode of the residence time distribution characteristic comprises:

mapping and matching the movement track of the user with the exhibition area flattening diagram, and determining the area where the user is located; calculating the probability of the region where the user is located at each time point;

specifically, the user movement track data is defined as a vector；

The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>=(/>,/>) Indicating that the user is at time +.>Appear in a position；

Locating the userMatching with the area of the exhibition area plan, determining the area of the user +.>The method comprises the steps of carrying out a first treatment on the surface of the At the momentThe user appears in the area +.>The probability of (2) is +.>；

User entry areas given based on a priori constraint knowledgeThe probability of (2) is +.>；

Combining the prior constraint knowledge with the actual observed probability; posterior distribution of region entry probability calculation using bayesian rule；

Wherein,to be in the region->Sample +.>Probability of (2);

indicating that in the case of observing sample->Under the condition of (1) the user is in the area +.>Probability of (2); />For observing sample->Edge probability of (2);

And counting the B user tracks, and calculating the average residence time and variance of each area to form residence time distribution characteristics.

Further, the generating manner of the user characteristic label includes:

judging whether the facial image features exist in a facial image database of the user, and if so, extracting user identity information; if the face image features do not exist, registering the face image features to a face feature image database of the user; searching and extracting basic information of a user from a user information database according to user identity information, wherein the basic information comprises names, age groups, occupation and cultural degrees; processing the basic information of the user into a user basic feature vector;

rasterizing and partitioning the museum area, calculating the average residence time in each grid area based on the user movement track and residence time distribution characteristics of the user, and constructing a residence time thermodynamic diagram;

setting different weight coefficients for different exhibition hall areas according to the residence time thermodynamic diagram, and representing the importance of the areas; calculating the residence time ratio of the user in each exhibition hall area and the surrounding grids, and representing the interest degree of the user in the area;

multiplying the interest degree of the user in each exhibition area by the weight coefficient of the exhibition area, and calculating a weighted average to form the interest preference coefficient of the user in the whole layout of the museum;

And collecting the corresponding relation between all the exhibits and the exhibition hall area, and calculating the exhibit preference coefficient of the user for various exhibits based on the residence time distribution condition of the user in each area.

Further, the calculation mode of the exhibit preference coefficient includes:

the museum is provided with b exhibition areas, and different types of exhibits are exhibited in each area; first, theThe corresponding exhibit category set of the individual region is +.>Wherein each category->The corresponding specific number of exhibits is N_ { or }>,/>-a }; the user is->In->The total residence time in the individual zones is:

the method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Representing user +.>In->Within the region +.>Residence time before individual exhibits;

for categoryThe user is in the area->The browsing preference coefficients of the inner pair of the category exhibits are as follows:

the method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>For user->In->In the individual exhibition area, when the user is looking at the category +.>Interior->When specific exhibits are selected, the residence time of the user is the time before;

user pair categoryThe overall preference coefficient of (2) is a weighted average of the region coefficients, and the overall preference coefficient is:the method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Is->A region weight coefficient preset in a region; the overall preference coefficient is the preference coefficient of the exhibit;

the interest preference coefficients of the users to the museum layout and the preference coefficients of the users to the exhibits of the exhibit category are vectorized and connected to form user interest feature vectors;

Inputting the basic feature vector and the interest feature vector of the user into a pre-trained conditional random field model, learning the association weights among different vectors, and outputting the comprehensive feature vector of the user;

and classifying the comprehensive feature vectors of the users by adopting a K-means clustering algorithm to generate user feature labels representing different user types.

Further, the pre-training process of the conditional random field model includes:

acquiring behavior track data of a user in a library, and acquiring basic information and interest characteristics of the user; vectorizing the basic information and the interest characteristic of the user to obtain a user basic characteristic vector and a user interest characteristic vector; labeling the user category as the supervision information of the conditional random field model, namely outputting; the input features of the conditional random field model are user basic feature vectors and user interest feature vectors; the output label of the conditional random field model is the user category; the user category is the comprehensive feature vector of the user;

defining a conditional random field model of a linear chain, and obtaining characteristics through maximum likelihood estimation function values during trainingWeighting; the data set used for training is The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Is->Input features of individual samples, +.>Is the corresponding output label;

Conditional random field model probability representation as；

Wherein,is a feature function defined on the input and output; />Is the corresponding weight; />Is a condition normalization factor; />Is the base of the exponential function; />An index of the data set used for training;

characteristic function；

Wherein,for the length of the input sequence, < >>Is defined at->A feature function on the position input and output;

likelihood estimation function；

Updating weights using L-BFGS methodObtaining optimal weight by means of iterative optimization>Make->Maximizing; obtaining a conditional random field model.

Further, the way of planning the exhibition area into n visit routes includes:

determining a rasterization block scale s according to the area of the exhibition hall and the number of the exhibits, and dividing the whole exhibition hall area into grids with the size of mx x nx s x s;

defining a network diagram gd= (V, E), wherein V represents nodes of each grid region and E represents a communication relationship between the nodes; constructing a minimum spanning tree MST, and finding out the shortest route connecting all exhibition areas;

based on the minimum spanning tree MST, dividing areas comprising different exhibit topics by using a nearest neighbor method; determining a primary display in the region; calculating the shortest tour route by adopting a search algorithm aiming at each main exhibition area, and connecting all exhibits in the area;

Calculating the total length L of the visit routes in each area, randomly selecting a starting point, and defining the path length of traversing all the exhibits in the area once according to the clockwise sequence as the length of the carpet; if L is greater than g times the length of the carpet, g is a value greater than or equal to 2; recursively splitting the region and recalculating the internal route;

analyzing the relative position relation among all the split areas, and setting connection channels among the areas according to the relative position relation; and selecting n routes with optimal visit order by using a greedy algorithm according to the visit order constraint.

Further, the method for selecting the best visit route according to the user characteristic label comprises the following steps:

user feature labels represent different types of users，/>；/>Representing the type;

user feature labels of each classCorresponds to a recommended visit line +.>；/>；

Defining a route scoring function；

Wherein,for characteristic tag->And (2) with route->Matching degree of->For the length of the route>In order to cover the number of exhibits,and->Is the scoring weight;

degree of matchingThe calculation mode of (2) comprises:

the route comprises the following exhibits: />The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>The last exhibit in the exhibit set;

defining a routeFor the field of exhibits->Coverage of- >；/>Is an integer of 1 or more;

the method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Is the field of exhibits; />For the way->The total number of exhibits on the table;

defining feature tagsCorresponding preference vector->；

Wherein,representing user's field of exhibits->Is a preference for (a) a preference for (b);

initial degree of matching；

If the route isMore users of the overlay->Preference field of exhibits, degree of matching ∈>The higher;

the matching degree of all routes is relatively setNormalized to the 0-1 range;

the method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Is->Maximum value of>Is->Is the minimum of (2);

for input user feature labelsTraverse all->And selecting the visit route with the largest route scoring function value as the optimal visit route.

A smart museum user management method, which is realized based on the smart museum user management system, comprising the following steps: s1, identifying user identity information;

s2, generating a user characteristic label according to the user identity information;

s3, planning the exhibition area into n visit routes; selecting an optimal visit route according to the user characteristic label; and sending the optimal visit route to the mobile equipment of the user through a network, and sending the user identity information and the corresponding visit route to the cloud platform server.

The intelligent museum user management system has the technical effects and advantages that:

The method comprises the steps of acquiring identity and basic information of a user by identifying facial images of tourists entering a museum and combining a user information database, accurately extracting a user interest model by combining the activity track and residence time distribution data of the user in the museum, and realizing characterization expression of different user types; based on a user model, the system adopts a multi-objective comprehensive evaluation mode to select an optimal route from a candidate route set for recommendation, so that the route length, the number of covered exhibits and the degree of matching user interests reach the optimal balance; the system not only greatly improves the visiting experience of tourists and realizes personalized service, but also can assist the museum to realize scientific management and control of the visiting people flow distribution, improve the exhibition area arrangement and adjust the exhibition content, and comprehensively optimize the user experience and the operation efficiency of the museum operation system; the technical means of artificial intelligence, computer vision, operation and research optimization and the like are fully utilized, an accurate, intelligent and highly adaptable user model is established, and differentiated and personalized user management is realized, so that the museum can grow into a learning organization taking the user as a center on the visiting service, and the learning organization and the user jointly progress.

Drawings

FIG. 1 is a schematic diagram of a intelligent museum user management system according to the present invention;

FIG. 2 is a schematic diagram of a residence time thermodynamic diagram of the present invention;

FIG. 3 is a schematic diagram of a method for intelligent museum user management according to the present invention;

FIG. 4 is a schematic diagram of an electronic device of the present invention;

fig. 5 is a schematic diagram of a storage medium of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Embodiment one;

referring to fig. 1, a system for managing users of a smart museum according to this embodiment includes:

the user data input module is used for identifying user identity information;

Further, the method for identifying the user identity information includes:

the user identity information comprises facial image characteristics, user movement tracks and residence time distribution characteristics;

the user performs face scanning registration at the entrance of the museum and collects face images; and extracting the characteristics of the face image by using a preset characteristic extraction model.

The preset mode of the feature extraction model comprises the following steps:

taking a ResNet-50 convolutional neural network as an infrastructure of a feature extraction model, wherein the ResNet-50 convolutional neural network consists of an S-layer convolutional layer and a pooling layer;

selecting y face images from the face image data set as a face image set, and dividing the face image set into a training set and a verification set according to a proportion; the dividing ratio can be 80% training set and 20% verification set; the acquisition requirement of the face image is that the face image is a front face, and the image size is in a uniform format; for example, size 224x224;

preprocessing the training set image, wherein the preprocessing process comprises the steps of face image alignment, clipping and color enhancement; dividing the training set into batches, wherein each batch comprises positive samples and negative samples; the positive sample is a face image pair belonging to the same person; the negative samples are face image pairs belonging to different people;

Performing feature extraction by utilizing an initial a layer of the ResNet-50 convolutional neural network, wherein the feature extraction comprises conv1 to conva/2 layers, and the conv1 to conva/2 layers can extract low-level to high-level features of the face image;

locking all layer parameters after a layer a of ResNet, and training only the parameters of the layer a before; setting the characteristic dimension as 512 channels, namely setting the conva/2 output size as 512x7x7;

the pre-a layer parameter initialization adopts a ResNet-50 convolutional neural network parameter pre-trained on a training set, and then fine adjustment is further carried out;

defining feature extraction and inputting a face image; the input for feature extraction here is a pair of face images, denoted (xa, xb); wherein xa and xb represent two input images, respectively; calculating Euclidean distance between the feature vectors of the face images as a loss function;

setting the input batch size of the ResNet-50 convolutional neural network, for example, the batch size is 32, iteratively updating parameters, continuously reducing the face feature distance of positive samples in the batch, and increasing the face feature distance of negative samples;

iteratively updating the parameters of the previous layer a, and continuously training for d rounds until the loss function value of the verification set is not reduced any more, so as to obtain a feature extraction model;

each position point is provided with 1 camera, and m cameras are arranged in total; the shooting angle of the camera is downward overlook; the shooting range of the camera ensures that the figure and the shadow of the user can be clearly captured; collecting an image when a user appears in a shooting image of a camera;

positioning users appearing in each acquired image through an image recognition algorithm, and acquiring space coordinates (x, y) of the users; taking a timestamp of an image shot by the camera as a user occurrence time t; if the same user is identified in different cameras, checking the characteristics to judge that the user is the same person; labeling the position coordinates (X, Y) of each camera on the exhibition hall plan; matching the appearance coordinates of the user with the marked camera coordinates; if a user appears in the fields of view of the camera 1 and the camera 2 in sequence, judging that the user moves from the position of the camera 1 to the position of the camera 2; combining the distance between two camera positions on the plan viewAnd the user appears in the two cameras with time difference +.>Calculating the moving speed of the user>；

It should be noted that, the sample point (x, y, t) represents a time point t when the user captures the position coordinate (x, y) by the camera; connecting the sample points adjacent in time sequence into a straight line segment, which can represent the movement of a user between two sample points; for example, sample point a (x 1, y1, t 1) and sample point B (x 2, y2, t 2) have a time interval Δt=t2-t 1 therebetween; according to the average moving speed of the userThe movement distance dd1=v×Δtof the user between sample points a and B can be estimated; scaling Dd1 according to the scaling of a certain exhibition hall flattening diagram; connecting A, B two points by drawing a straight line, wherein the length of the line segment is Dd1 after scaling, and the line segment represents the moving track of a user in the time period from t1 to t 2; sequentially connecting more sample points adjacent in time sequence, and drawing a moving track of a user in the whole time range;

when the tracks are crossed or overlapped, judging a movement trend by combining the movement speed, and reconnecting the tracks;

acquiring a layout structure and exhibit distribution of a user visit area; according to the layout structure and the exhibit distribution of the user visit area, a Markov random field model is constructed and used as priori constraint knowledge of user behavior analysis; combining the priori constraint knowledge with the user movement track, and calculating the residence time distribution characteristics of each visit area by using Bayesian reasoning;

The a priori constraint knowledge is the probability of the user entering each of the visited areas.

The acquisition mode of the layout structure comprises the following steps:

dividing grids on a plan view to form a 2D grid space for quantization of space coordinates; recording the functional attribute of each grid unit, for example, whether the grid unit is an exhibit area;

the acquisition mode of the exhibit distribution is as follows: marking the placement positions of various exhibits on a plan; counting the number of each type of exhibits, and recording the coordinate range of the exhibits; the grids are not divided in the range of the exhibit area as a whole.

The construction mode of the Markov random field model comprises the following steps:

equally dividing the whole user activity area into Q grids, defining Q random variables Xo, and representing the user access state of each grid; when the user enters the o-th grid, xo=1, otherwise xo=0; setting the energy value of the state change of the adjacent areas as 1 and the non-adjacent areas as 5 according to the actual distance between the reference areas;

building an energy function from spatial distances between pixelsThe method comprises the steps of carrying out a first treatment on the surface of the The energy function reflects the constraint relation of state change between adjacent areas;

defining a gibbs distribution based on an energy function, whereinIs a distribution normalization factor; />Reflecting the probability of each random variable value;

marking K groups of user track data samples from a museum monitoring video;

predicting the probability of entering each region for newly input user position data; for the new input data, the probability of the random variable taking the value 1, i.e. the probability of the user entering each area, is calculated.

The calculation mode of the residence time distribution characteristics comprises the following steps:

specifically, the user movement track data is defined as a vector；

User entry areas given based on a priori constraint knowledge The probability of (2) is +.>；

Wherein,to be in the region->Sample +.>Probability of (2);

Further, the generating manner of the user feature tag includes:

it should be noted that, the construction method of the facial feature image database includes:

collecting face images of users to form an initial training set; training a face feature extractor by using a deep learning model such as a convolutional neural network; extracting a characteristic vector with a fixed length from each face image to represent the characteristics of the characteristic vector; and storing the characteristic vector serving as an index into a face characteristic image database.

The construction mode of the user information database comprises the following steps:

defining a structure of a user information database, wherein the structure comprises fields of user ID, name, age group, occupation and cultural degree; setting a data acquisition terminal at a museum entrance, registering and registering a user entering the museum, and providing information such as name, age and the like for tourists; loading information into a user information database;

the manner of processing the basic information of the user into the basic feature vector of the user includes:

the user basic information includes: name, age group, occupation, and cultural degree; defining a user description field comprising the 4 feature dimensions; quantifying the age of the user, e.g., a code under 20 years old of 1,20-30 years old of 2, and so on; other category characteristics are ont-hot coded; for example, professions include J types, each type being represented by a J-dimension 0/1 vector, a dimension of value 1 representing the type of profession to which the user belongs; the basic information of the end user is expressed as a G-dimensional vector (G equals the sum of all feature dimensions), which is the basic feature vector;

for example, a user, 22 years old teacher identity, end his user base feature vector is denoted [1 [1 0 0 0 0] ];

Calculating interest characteristics according to the user movement track and residence time distribution characteristics of the user, wherein the interest characteristics comprise interest preference coefficients and exhibit preference coefficients;

specifically, the museum areas are subjected to rasterization and blocking, the average residence time in each grid area is calculated based on the user movement track and residence time distribution characteristics of the user, and a residence time thermodynamic diagram is constructed; as shown in fig. 2;

The calculation mode of the exhibit preference coefficient comprises the following steps:

the museum is provided with b exhibition areas, and different types of exhibits are exhibited in each area; first, the The corresponding exhibit category set of the individual region is +.>Wherein each category->The corresponding specific number of exhibits is N_ { or }>,/>-a }; the user is->In->The total residence time in the individual zones is:

user pair categoryThe overall preference coefficient of (2) is a weighted average of the region coefficients, and the overall preference coefficient is:

the method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Is->A region weight coefficient preset in a region; the overall preference coefficient is the preference coefficient of the exhibit;

The pre-training process of the conditional random field model comprises the following steps:

defining a conditional random field model of a linear chain, and obtaining characteristic weights through maximum likelihood estimation function values during training; specifically, the training target is to maximize the log likelihood of conditional probability distribution, and the model parameters are optimized by using an L-BFGS method to obtain the feature weights;

the above procedure is explained as follows:

data set for training The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Is->Input features of individual samples, +.>Is the corresponding output label;

conditional random field model probability representation as；/>

Wherein,is a feature function defined on the input and output; />Is the corresponding weight; / >Is a condition normalization factor; />Is the base of the exponential function; />An index of the data set used for training;

characteristic function；

training targets of；

Updating weights using L-BFGS methodObtaining optimal weight by means of iterative optimization>Make->Maximizing; obtaining a conditional random field model; the obtained conditional random field model models the association relation among different types of users through the characteristics and the weights thereof.

The method for classifying the comprehensive feature vectors of the users by the K-means clustering algorithm comprises the following steps:

carrying out standardized processing on the comprehensive feature vector data of the user, and eliminating the influence of feature scale; selecting a cluster number,The value is generally determined according to the service requirement; randomly select +.>The individual samples are used as initial cluster centersThe method comprises the steps of carrying out a first treatment on the surface of the For each sample, calculate it and each class center +.>Dividing it into classes closest to each other;

for each class, the class center is recalculatedThe method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>The comprehensive feature vector representing the user is a data sample which is required to be clustered by an algorithm; />Indicate->The number of samples comprised by the class, i.e. class- >The number of user vectors;

when the class center is not changed any more, the algorithm converges, and iteration is terminated; at this point each sample is clustered into nearest classes, each class representing a user type; for a pair ofAnd the class marks out corresponding user characteristic labels to represent different types of users.

Further, the method for planning the exhibition area into n visit routes includes:

based on the minimum spanning tree MST, dividing areas comprising different exhibit topics by using a nearest neighbor method; determining a primary display in the region; and calculating the shortest tour route by adopting a search algorithm aiming at each main exhibition area, and connecting all exhibits in the area.

Further, calculating the total length L of the visit route in each area, randomly selecting a starting point, and defining the path length of traversing all the exhibits in the area once according to the clockwise sequence as the length of the carpet; if L is greater than g times the length of the carpet, g is a value greater than or equal to 2; recursively splitting the region and recalculating the internal route;

Analyzing the relative position relation among all the split areas, and setting connection channels among the areas according to the relative position relation; according to the visit order constraint, selecting n routes with optimal visit order by using a greedy algorithm;

it should be noted that, the search algorithm is an a search algorithm; the search algorithm A utilizes heuristic functions to estimate the shortest path, so that the search efficiency is higher; abstracting the passage of the exhibition area and the position of the exhibited article as nodes; the passage is communicated with the passage, and the exhibited article is communicated with the exhibited article; forming a directed graph or grid map to represent the movable space of the exhibition area; initializing an opening and closing list;

calculating a heuristic score f (α) =g (α) +h (α); wherein g (α) is the actual distance from the start point to the current point α, and h (α) is the estimated distance to the target point set by the user;

selecting the node with the minimum f (alpha) from the opening list for expansion every time, generating adjacent nodes, updating the scores and adding the scores into the opening list; when the route is expanded to the target node, the search is terminated, and the trace-back path is the shortest route; connecting the found shortest route according to a reachable way among the exhibits, so as to ensure that each exhibit is accessed once;

The recursive splitting area may be interpreted as an overall length L of the original area route that is too long, exceeding an acceptable maximum carpet length g times; the region is split according to the equal proportion, and the dividing line avoids the exhibited product region as much as possible, so that two sub-regions are divided; recursively performing an internal shortest route calculation for the two sub-regions until the intra-region route length is less than a threshold; the threshold value can be set according to actual conditions;

various situations may occur in the relative positional relationship between the split regions, for example, the region a is split into two sub-regions A1 and A2; whether a space adjacent relation between the A1 and the A2 is provided with a communicated edge or not needs to be analyzed, so that a connecting channel is arranged, and a connecting passage is ensured between the two sub-areas;

the visit order constraint refers to designating a certain order constraint condition for a visit route, and ensuring logic, time and the like to conform to the visit habit; for example, the time sequence exhibition area needs to visit successively, and the theme-separated area also has visual logic sequence and the like; this is a rule constraint that determines the final routing.

Further, the method for selecting the optimal visit route according to the user feature tag comprises the following steps:

Defining a route scoring function；

degree of matchingThe calculation mode of (2) comprises:

defining a routeFor the field of exhibits->Coverage of->；/>Is an integer of 1 or more;

defining feature tagsCorresponding preference vector->；

initial degree of matching；

If the route isMore users of the overlay->Preference field of exhibits, degree of matching ∈>The higher; />

The matching degree of all routes is relatively setNormalized to the 0-1 range;

According to the embodiment, the facial images of tourists entering the museum are identified, the identity and basic information of the user are acquired by combining with the user information database, and the user interest model is accurately extracted by combining with the activity track and residence time distribution data of the user in the museum, so that the characterization expression of different user types is realized; based on a user model, the system adopts a multi-objective comprehensive evaluation mode to select an optimal route from a candidate route set for recommendation, so that the route length, the number of covered exhibits and the degree of matching user interests reach the optimal balance; the system not only greatly improves the visiting experience of tourists and realizes personalized service, but also can assist the museum to realize scientific management and control of the visiting people flow distribution, improve the exhibition area arrangement and adjust the exhibition content, and comprehensively optimize the user experience and the operation efficiency of the museum operation system; the technical means of artificial intelligence, computer vision, operation and research optimization and the like are fully utilized, an accurate, intelligent and highly adaptable user model is established, and differentiated and personalized user management is realized, so that the museum can grow into a learning organization taking the user as a center on the visiting service, and the learning organization and the user jointly progress.

Embodiment two;

referring to fig. 2, the detailed description of the embodiment is not shown in the description of embodiment 1, and a method for managing intelligent museum users is provided, which includes:

s1, identifying user identity information;

Embodiment three;

referring to fig. 4, an electronic device 500 is also provided according to yet another aspect of the present application. The electronic device 500 may include one or more processors and one or more memories. Wherein the memory has stored therein computer readable code which, when executed by the one or more processors, can perform a smart museum user management method as described above.

The method or system according to embodiments of the present application may also be implemented by means of the architecture of the electronic device shown in fig. 4. As shown in fig. 4, the electronic device 500 may include a bus 501, one or more CPUs 502, a Read Only Memory (ROM) 503, a Random Access Memory (RAM) 504, a communication port 505 connected to a network, an input/output component 506, a hard disk 507, and the like. A storage device, such as a ROM503 or hard disk 507, in the electronic device 500 may store a smart museum user management method provided herein. Further, the electronic device 500 may also include a user interface 508. Of course, the architecture shown in fig. 4 is merely exemplary, and one or more components of the electronic device shown in fig. 4 may be omitted as may be desired in implementing different devices.

Fourth embodiment;

referring to FIG. 5, a computer readable storage medium 600 according to one embodiment of the present application is shown. Computer readable storage medium 600 has stored thereon computer readable instructions. When the computer readable instructions are executed by the processor, a smart museum user management method according to an embodiment of the present application described with reference to the above figures may be performed. Storage medium 600 includes, but is not limited to, for example, volatile memory and/or nonvolatile memory. Volatile memory can include, for example, random Access Memory (RAM), cache memory (cache), and the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, and the like.

In addition, according to embodiments of the present application, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, the present application provides a non-transitory machine-readable storage medium storing machine-readable instructions executable by a processor to perform instructions corresponding to the method steps provided herein, a smart museum user management method. The above-described functions defined in the method of the present application are performed when the computer program is executed by a Central Processing Unit (CPU).

The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with embodiments of the present invention are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center over a wired network or a wireless network. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more sets of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.

Claims

1. A smart museum user management system, comprising: the user data input module is used for identifying user identity information; the user data processing module is used for generating a user characteristic label according to the user identity information; the intelligent control module comprises a route planning unit, a route matching unit and a data feedback unit; and is connected with the user data processing module; the route planning unit is used for planning the exhibition area into n visit routes; the line matching unit is used for selecting an optimal visiting route according to the user characteristic tag; transmitting the optimal visit route to the mobile equipment of the user through a network, and transmitting the user identity information and the corresponding visit route to a cloud platform server; all the modules are connected in a wired and/or wireless mode, so that data transmission among the modules is realized.

2. The intelligent museum user management system of claim 1, wherein the user identity information includes facial image features, user movement trajectories and residence time distribution features; the user performs face scanning registration at the entrance of the museum and collects face images; extracting the features of the face image by using a preset feature extraction model; setting m cameras along the way in a user visit area, collecting a position image of a user, inputting the position image of the user into a preset regression analysis model, and obtaining a user movement track; determining a position point for setting a camera according to the plane view of the exhibition hall; the selection of the location point takes into account the turn of the route, the intersection Fork opening and exhibition area entrance positions; each position point is provided with 1 camera, and m cameras are arranged in total; the shooting angle of the camera is downward overlook; collecting an image when a user appears in a shooting image of a camera; positioning users appearing in each acquired image through an image recognition algorithm, and acquiring space coordinates (x, y) of the users; taking a timestamp of an image shot by the camera as a user occurrence time t; if the same user is identified in different cameras, checking the characteristics to judge that the user is the same person; labeling the position coordinates (X, Y) of each camera on the exhibition hall plan; matching the appearance coordinates of the user with the marked camera coordinates; if a user appears in the fields of view of the camera 1 and the camera 2 in sequence, judging that the user moves from the position of the camera 1 to the position of the camera 2; combining the distance between two camera positions on the plan viewAnd the time difference between the two cameras of the userCalculating the moving speed of the user>The method comprises the steps of carrying out a first treatment on the surface of the Connecting W identified sample points (x, y, t) in series, and drawing a user movement track by combining the movement speed of the user; acquiring a layout structure and exhibit distribution of a user visit area; according to the layout structure and the exhibit distribution of the user visit area, a Markov random field model is constructed and used as priori constraint knowledge of user behavior analysis; and combining the prior constraint knowledge with the user movement track, and calculating the residence time distribution characteristics of each visit area by using Bayesian reasoning.

3. The intelligent museum user management system of claim 2, wherein the markov random field model is constructed in a manner comprising: equally dividing the whole user activity area into Q grids, defining Q random variables Xo, and representing the user access state of each grid; when the user enters the o-th grid, xo=1, otherwise xo=0; ginseng radixThe actual distance between the areas is considered, the energy value of the state change of the adjacent areas is set to be 1, and the non-adjacent areas are set to be 0; building an energy function from spatial distances between pixelsThe method comprises the steps of carrying out a first treatment on the surface of the Defining a gibbs distribution based on an energy function, wherein a normalization factor is distributed +.>；/>A set of spatial distances between pixels; establishing a three-layer conditional random field; the three-layer conditional random field comprises a bottom layer, a middle layer and a high layer; the bottom layer represents position data, the middle layer represents a behavior mode, and the high layer represents user intention; marking K groups of user track data samples from a museum monitoring video; and training and learning parameters of the Markov random field model by using K groups of user track data samples, so that the output probability distribution of the Markov random field model is matched with the actual distribution of the user.

4. A smart museum user management system in accordance with claim 3, wherein said residence time distribution characteristics are calculated by a method comprising: mapping and matching the movement track of the user with the exhibition area flattening diagram, and determining the area where the user is located; calculating the probability of the region where the user is located at each time point; specifically, the user movement track data is defined as a vector ；/> The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>=(/>,/>) Indicating that the user is at time +.>Appear at the position +.>The method comprises the steps of carrying out a first treatment on the surface of the User position +.>Matching with the area of the exhibition area plan, determining the area of the user +.>The method comprises the steps of carrying out a first treatment on the surface of the Then at time->The user is present in the areaThe probability of (2) is +.>The method comprises the steps of carrying out a first treatment on the surface of the User entry area given based on a priori constraint knowledge>The probability of (2) isThe method comprises the steps of carrying out a first treatment on the surface of the Combining the prior constraint knowledge with the actual observed probability; posterior distribution of region entry probability calculation using Bayesian rule>The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>To be in the region->Sample +.>Probability of (2); />Indicating that in the case of observing sample->Under the condition of (1) the user is in the area +.>Probability of (2); />For observing sample->Edge probability of (2); and counting the B user tracks, and calculating the average residence time and variance of each area to form residence time distribution characteristics.

5. The intelligent museum user management system of claim 4, wherein the generating means of the user feature tag comprises: judging whether the facial image features exist in a facial image database of the user, and if so, extracting user identity information; if the face image features do not exist, registering the face image features to a face feature image database of the user; searching and extracting basic information of a user from a user information database according to user identity information, wherein the basic information comprises names, age groups, occupation and cultural degrees; processing the basic information of the user into a user basic feature vector; rasterizing and partitioning the museum area, calculating the average residence time in each grid area based on the user movement track and residence time distribution characteristics of the user, and constructing a residence time thermodynamic diagram; setting different weight coefficients for different exhibition hall areas according to the residence time thermodynamic diagram, and representing the importance of the areas; calculating the residence time ratio of the user in each exhibition hall area and the surrounding grids, and representing the interest degree of the user in the area; multiplying the interest degree of the user in each exhibition area by the weight coefficient of the exhibition area, and calculating a weighted average to form the interest preference coefficient of the user in the whole layout of the museum; and collecting the corresponding relation between all the exhibits and the exhibition hall area, and calculating the exhibit preference coefficient of the user for various exhibits based on the residence time distribution condition of the user in each area.

6. The intelligent museum user management system of claim 5, wherein the calculating means of the exhibit preference coefficient comprises: the museum is provided with b exhibition areas, and different types of exhibits are exhibited in each area; first, theThe corresponding exhibit category set of the individual region is +.>Wherein each category->The corresponding specific number of exhibits is N_ { or }>,/>-a }; the user is->In the first placeThe total residence time in the individual zones is: />The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Representing user +.>In->Within the region +.>Residence time before individual exhibits; for category->The user is in the area->The browsing preference coefficients of the inner pair of the category exhibits are as follows:the method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>For user->In->In the individual exhibition area, when the user is looking at the category +.>Interior->When specific exhibits are selected, the residence time of the user is the time before; user is +.>The overall preference coefficient of (2) is a weighted average of the region coefficients, and the overall preference coefficient is: />The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Is->A region weight coefficient preset in a region; the overall preference coefficient is the preference coefficient of the exhibit; the interest preference coefficients of the users to the museum layout and the preference coefficients of the users to the exhibits of the exhibit category are vectorized and connected to form user interest feature vectors; inputting the basic feature vector and the interest feature vector of the user into a pre-trained conditional random field model, learning the association weights among different vectors, and outputting the comprehensive feature vector of the user; and classifying the comprehensive feature vectors of the users by adopting a K-means clustering algorithm to generate user feature labels representing different user types.

7. The intelligent museum user management system of claim 6, wherein the pre-training process of the conditional random field model comprises: acquiring behavior track data of a user in a library, and acquiring basic information and interest characteristics of the user; vectorizing the basic information and the interest characteristic of the user to obtain a user basic characteristic vector and a user interest characteristic vector; labeling the user category as the supervision information of the conditional random field model, namely outputting; the input features of the conditional random field model are user basic feature vectors and user interest feature vectors; the output label of the conditional random field model is the user category; the user category is the comprehensive feature vector of the user; defining a conditional random field model of a linear chain, and obtaining characteristic weights through maximum likelihood estimation function values during training; the data set used for training is The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Is->Input features of individual samples, +.>Is the corresponding output label; conditional random field model probability is expressed as +.>The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Is a feature function defined on the input and output; />Is the corresponding weight; />Is a condition normalization factor;is the base of the exponential function; />An index of the data set used for training; characteristic function- >The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>For the length of the input sequence, < >>Is defined at->A feature function on the position input and output; likelihood estimation function +.>The method comprises the steps of carrying out a first treatment on the surface of the Updating weights using the L-BFGS method>Obtaining optimal weight by means of iterative optimization>Make->Maximizing; obtaining a conditional random field model.

8. The intelligent museum user management system of claim 7, wherein the means for planning the exhibition area into n visit routes comprises: determining a rasterization block scale s according to the area of the exhibition hall and the number of the exhibits, and dividing the whole exhibition hall area into grids with the size of mx x nx s x s; defining a network diagram gd= (V, E), wherein V represents nodes of each grid region and E represents a communication relationship between the nodes; constructing a minimum spanning tree MST, and finding out the shortest route connecting all exhibition areas; based on the minimum spanning tree MST, dividing areas comprising different exhibit topics by using a nearest neighbor method; determining a primary display in the region; calculating the shortest tour route by adopting a search algorithm aiming at each main exhibition area, and connecting all exhibits in the area; calculating the total length L of the visit routes in each area, randomly selecting a starting point, and defining the path length of traversing all the exhibits in the area once according to the clockwise sequence as the length of the carpet; if L is greater than g times the length of the carpet, g is a value greater than or equal to 2; recursively splitting the region and recalculating the internal route; analyzing the relative position relation among all the split areas, and setting connection channels among the areas according to the relative position relation; and selecting n routes with optimal visit order by using a greedy algorithm according to the visit order constraint.

9. The intelligent museum user management system of claim 8, wherein the selecting the best visit route based on the user characteristic tag comprises: user feature labels represent different types of users，；/>Representing the type; user feature tags per class->Corresponds to a recommended visit line +.>；/>The method comprises the steps of carrying out a first treatment on the surface of the Defining a route scoring function->The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>For characteristic tag->And (2) with route->Matching degree of->For the length of the route>To cover the number of exhibits, the person is added with->And->Is the scoring weight; degree of matchingThe calculation mode of (2) comprises: />The route comprises the following exhibits: />The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>The last exhibit in the exhibit set; define route->For the field of exhibits->Coverage of->；/>Is an integer of 1 or more; />The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Is the field of exhibits; />For the way->The total number of exhibits on the table; definition of the feature tag->Corresponding preference vector->The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Representing user's field of exhibits->Is a preference for (a) a preference for (b); initial match degree->The method comprises the steps of carrying out a first treatment on the surface of the If the route is->More users of the overlay->Preference field of exhibits, degree of matching ∈>The higher; matching degree of all routes relatively +.>Normalized to the 0-1 range; />The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Is->Maximum value of>Is->Is the minimum of (2); for input user feature tag- >Traverse all->And selecting the visit route with the largest route scoring function value as the optimal visit route.

10. A smart museum user management method implemented based on a smart museum user management system as claimed in any one of claims 1 to 9, comprising: s1, identifying user identity information; s2, generating a user characteristic label according to the user identity information; s3, planning the exhibition area into n visit routes; selecting an optimal visit route according to the user characteristic label; and sending the optimal visit route to the mobile equipment of the user through a network, and sending the user identity information and the corresponding visit route to the cloud platform server.