CN117453585A - Cache management method, electronic equipment and storage medium - Google Patents
Cache management method, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117453585A CN117453585A CN202310855924.9A CN202310855924A CN117453585A CN 117453585 A CN117453585 A CN 117453585A CN 202310855924 A CN202310855924 A CN 202310855924A CN 117453585 A CN117453585 A CN 117453585A
- Authority
- CN
- China
- Prior art keywords
- user
- interest preference
- cache
- activity
- friends
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 title claims abstract description 28
- 230000000694 effects Effects 0.000 claims abstract description 104
- 238000012549 training Methods 0.000 claims abstract description 41
- 230000008030 elimination Effects 0.000 claims abstract description 26
- 238000003379 elimination reaction Methods 0.000 claims abstract description 26
- 238000004364 calculation method Methods 0.000 claims abstract description 9
- 239000013598 vector Substances 0.000 claims description 63
- 238000011176 pooling Methods 0.000 claims description 28
- 238000004590 computer program Methods 0.000 claims description 22
- 238000012545 processing Methods 0.000 claims description 18
- 230000006399 behavior Effects 0.000 claims description 16
- 238000012216 screening Methods 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 12
- 230000009467 reduction Effects 0.000 claims description 6
- 230000007246 mechanism Effects 0.000 claims description 5
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 230000006872 improvement Effects 0.000 abstract description 13
- 230000004044 response Effects 0.000 abstract description 5
- 238000000034 method Methods 0.000 description 18
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 238000013473 artificial intelligence Methods 0.000 description 4
- 238000013145 classification model Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 235000019633 pungent taste Nutrition 0.000 description 3
- 230000001174 ascending effect Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F12/0891—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches using clearing, invalidating or resetting means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a cache management method, electronic equipment and a storage medium, wherein activity calculation is performed on a target user by utilizing a user activity prediction model; responding to a cache elimination instruction, and eliminating inactive users from the cache according to the activity level; the user liveness prediction model is obtained through training in the following mode: obtaining known user data to construct a sample set, wherein the known user data comprises user features, liveness tags associated with the user features; model training is carried out by using the sample set to obtain a user liveness prediction model. According to the embodiment of the invention, the user liveness prediction model is utilized to calculate the liveness of the users in the cache, the inactive users are removed according to the liveness in response to the cache elimination instruction, and the cache hit rate is improved, so that the improvement of the cache performance is realized.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a cache management method, an electronic device, and a storage medium.
Background
With the rapid development of the mobile internet, various APP applications are layered endlessly, personalized pushing scenes are more and more, and the requirements on the response speed of the interface are higher and higher. In order to ensure the smoothness of the user, the personalized data (user portrait tag) of the user is queried once and then is cached, and the data of the current user is directly obtained from the cache when the data is subsequently requested again, so that the purposes of reducing response time delay and improving the user experience are achieved.
Because the system server needs to face N users, the media resource labels are M, so the number of the data cached locally is N x M, and when the number of the users reaches tens of millions, the storage pressure of the local cache of the system is huge. Because the memory space of each server is limited, the number of the local caches is generally required to be set, and when the number of the local cache data is larger than the number set by the caches, old data is removed from the cache list.
However, in the prior art, when the local cache meets the elimination condition (for example, the occupancy rate of the cache space is greater than or equal to a preset threshold value), the cache data needs to be eliminated according to the historical access condition of the user, if the cache elimination logic is improper, the local cache frequently adds and removes the user data, and the hit rate of the cache is reduced, so that the cache performance is rapidly reduced. For example, assuming there are two users A, B, user a accesses the system before user B, and in order to ensure smoothness of use of the system, the system will cache data of user a and user B. After the access is finished, the user A may access the system again in a subsequent period of time, and the user B is not accessing the system. However, when the local cache meets the eviction condition, the system will preferentially evict the data of the user a according to the LRU (least recently used) eviction data principle, which obviously is not the same as the actual situation.
Disclosure of Invention
Based on the above, the invention provides a cache management method, electronic equipment and a storage medium, which can utilize a user liveness prediction model to calculate liveness of users in a cache, respond to a cache elimination instruction, eliminate non-active users according to liveness and improve cache hit rate, thereby improving cache performance.
In order to achieve the above object, an embodiment of the present invention provides a cache management method, including:
calculating the activity of a target user by using a user activity prediction model; wherein the target user is a user in a cache;
responding to a cache elimination instruction, and eliminating inactive users from the cache according to the activity level;
the user liveness prediction model is obtained through training in the following mode:
obtaining known user data to construct a sample set, wherein the known user data comprises user features and liveness tags associated with the user features; the user features include at least one of user base attribute features, user interest preference features, user implicit interest preference friend features, user activity scores, and user context features;
And performing model training by using the sample set to obtain a user liveness prediction model.
As an improvement of the above solution, the responding to the cache elimination instruction eliminates the inactive user from the cache according to the activity level, including:
and responding to the cache elimination instruction, and eliminating the users with the lowest activity and preset target number.
As an improvement of the above solution, before the activity calculation of the target user by using the user activity prediction model, the method further includes:
obtaining the latest access time, access frequency and average access time of all users in the cache, so as to calculate the user activity scores of all users;
and screening out the set number of users with the lowest user activity scores to serve as the target users.
As an improvement of the above solution, the obtaining the latest access time, access frequency and average access time of all users in the cache, for calculating user activity scores of all users, includes:
the user activity score is calculated by:
wherein score Active activity For the user activity score, cnt is the number of user requests, t curr T is the current timestamp of the system last Time stamp of last 1 active for user, t avg For the average user request time, λ is the heat fraction function coefficient.
As an improvement of the above scheme, the implicit interest preference friend characteristics of the user include interest preference characteristics of active friends and interest preference characteristics of inactive friends.
As an improvement of the above scheme, the user liveness prediction model is specifically obtained by training in the following manner:
acquiring known user data to construct a sample set; the known user data includes user base attribute characteristics, user interest preference characteristics, interest preference characteristics of the active friends, interest preference characteristics of the inactive friends, user activity scores, and contextual characteristics of the user;
performing dimension reduction processing and pooling processing on the user basic attribute characteristics and the user interest preference characteristics respectively to obtain basic attribute pooling vectors and interest preference characteristic pooling vectors; performing dimension reduction processing and horizontal splicing processing on the context characteristics to obtain context splicing vectors;
based on self-attention mechanism, the interest preference characteristics of the active friends and the interest preference characteristics of the non-active friends are respectively processed to obtain active friend sequence self-attention vectors and non-active friend self-attention vectors;
Performing Attention operation on the context stitching vector, the basic attribute pooling vector and the interest preference feature pooling vector and the active friend sequence self-Attention vector and the inactive friend self-Attention vector respectively to obtain six target Attention vectors;
and inputting the user activity score, the context stitching vector, the basic attribute pooling vector, the interest preference feature pooling vector and the target attention vector into a deep neural network to perform model training so as to obtain a user activity prediction model.
As an improvement of the above scheme, the friend feature of the implicit interest preference of the user is determined by the following ways:
wherein neighbor (u) a ) Friends are preferred for implicit interests of user a,for the ith interest preference score of user a, threshold is the similarity determination threshold, ++>An ith interest preference score for user b;
and acquiring the interest preference characteristics of the implicit interest preference friends as the implicit interest preference friend characteristics of the user.
As an improvement of the above solution, the obtaining the interest preference feature of the implicit interest preference friend as the implicit interest preference friend feature of the user includes:
Calculating the active frequency of the implicit interest preference friends in the average unit time of the latest preset time;
the implicit interest preference friends with the activity frequency larger than a preset activity frequency judging threshold value are used as active friends;
the implicit interest preference friends with the activity frequency smaller than or equal to the activity frequency judging threshold value are used as non-active friends;
and respectively acquiring the interest preference characteristics of the active friends and the inactive friends to serve as the implicit interest preference friend characteristics of the user.
As an improvement of the above solution, the user interest preference feature includes a content category label, a user operation label associated with the content category label; the content category labels are preconfigured with corresponding codes and category weights, and the user operation labels are preconfigured with corresponding operation weights; the user operation includes at least one of playing, collecting, commenting, and focusing.
As an improvement to the above, the contextual characteristics and the liveness flag are determined by:
generating a context feature of the user from a first user behavior log of each session;
and according to the time interval generated by the first behavior log in 2 adjacent sessions of the user, dividing the liveness of the user according to the principle that the shorter the interval time is, the more lively the user is, and obtaining an liveness label.
As an improvement of the above scheme, the method further comprises:
responding to a user portrait tag request, and searching whether user data associated with the request exists in the cache;
if yes, acquiring a user portrait tag associated with the request from the cache and updating the latest access time, the access frequency and the average access time of the user;
if not, obtaining the user portrait tag associated with the request from a user portrait tag library, and writing the user portrait tag into the cache and updating the latest access time, access frequency and average access time of the user when the cache elimination instruction is not triggered.
As an improvement of the above scheme, the method further comprises: and generating the user portrait tag according to the user interest preference characteristics and the user basic attribute characteristics.
As an improvement of the above scheme, the method further comprises:
when the user portrait label is acquired, acquiring reverse data of a label program;
calculating the program scores of all programs according to the inverted data of the label programs and the user portrait labels;
screening the preset number of programs with higher program scores as personalized recall programs for users and feeding back recommendation results.
In order to achieve the above object, an embodiment of the present invention further provides a cache management device, including:
The liveness calculation module is used for calculating liveness of the target user by utilizing the user liveness prediction model; wherein the target user is a user in a cache;
the cache processing module is used for responding to a cache elimination instruction and eliminating inactive users from the cache according to the activity level;
the user liveness prediction model is obtained through training in the following mode:
obtaining known user data to construct a sample set, wherein the known user data comprises user features and liveness tags associated with the user features; the user characteristics comprise at least one of user basic attribute characteristics, user interest preference characteristics, user implicit interest preference friend characteristics, user cache heat score characteristics and user context characteristics;
and performing model training by using the sample set to obtain a user liveness prediction model.
To achieve the above object, an embodiment of the present invention further provides a computer readable storage medium, where the computer readable storage medium includes a stored computer program, where when the computer program runs, a device where the computer readable storage medium is controlled to execute the cache management method according to any one of the embodiments.
Compared with the prior art, the cache management method, the device, the electronic equipment and the storage medium disclosed by the embodiment of the invention are used for calculating the activity of the target user by utilizing the user activity prediction model; wherein the target user is a user in a cache; responding to a cache elimination instruction, and eliminating inactive users from the cache according to the activity level; the user liveness prediction model is obtained through training in the following mode: obtaining known user data to construct a sample set, wherein the known user data comprises user features and liveness tags associated with the user features; the user characteristics comprise at least one of user basic attribute characteristics, user interest preference characteristics, user implicit interest preference friend characteristics, user cache heat score characteristics and user context characteristics; and performing model training by using the sample set to obtain a user liveness prediction model. Therefore, the embodiment of the invention can utilize the user liveness prediction model to calculate the liveness of the users in the cache, respond to the cache elimination instruction, eliminate the non-active users according to the liveness, and improve the cache hit rate, thereby improving the cache performance.
Drawings
In order to more clearly illustrate the technical solutions of the present invention, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart illustrating a cache management method according to an embodiment of the present invention;
FIG. 2 is a diagram showing a recommended API shared cache data structure according to one embodiment of the present invention;
FIG. 3 is a schematic diagram of a distribution of user popularity scores according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a model structure according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a model training set according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, a flow chart of a cache management method according to an embodiment of the invention is shown. Specifically, the cache management method includes steps S1-S2:
s1, performing liveness calculation on a target user by using a user liveness prediction model; wherein the target user is a user in a cache;
s2, responding to a cache elimination instruction, and eliminating inactive users from the cache according to the activity level; the user liveness prediction model is obtained through training in the following mode:
obtaining known user data to construct a sample set, wherein the known user data comprises user features and liveness tags associated with the user features; the user features include at least one of user base attribute features, user interest preference features, user implicit interest preference friend features, user activity scores, and user context features;
and performing model training by using the sample set to obtain a user liveness prediction model.
The Cache is a temporary place for storing data (using frequent data), when the data is needed to be queried, the Cache is searched in the Cache firstly, if the data is found, the data is directly executed, and if the data is not found, the data is searched in a corresponding database, the response time delay can be reduced, the interface response speed is accelerated, and the purpose of user experience is improved.
For training of the user liveness prediction model, known user data is determined by using a log of the last N days every day, and a sample set is constructed according to the known user data, wherein the known user data comprises user characteristics and liveness labels associated with the user characteristics, the user characteristics comprise user basic attribute characteristics, user interest preference characteristics, user implicit interest preference friend characteristics, user cache heat score characteristics (user liveness score), user context characteristics and the like, the sample set is divided into a training set and a test set according to the proportion of 8:2, the training set is used for training a user collaborative liveness multi-classification model (user liveness prediction model), and the test set is used for verifying the effect of the model. And using Cross Entropy (Cross Entropy) as a model optimization objective function, using AUC as a model effect evaluation index, adjusting model super parameters to train a multi-classification model for multiple times, and selecting a model with the highest AUC index as a model for on-line reasoning. After the user collaborative activity multi-classification model is trained offline, the newly trained model is pushed to an online reasoning server, the reasoning server loads the latest model, predicts the activity of the user, determines the non-active user and deletes the data of the non-active user from the cache.
Compared with the prior art, the method and the device for calculating the activity of the cache can utilize the user activity prediction model to calculate the activity of the users in the cache, respond to the cache elimination instruction, eliminate the non-active users according to the activity, and improve the cache hit rate, so that the cache performance is improved.
In a preferred embodiment, the responding to the cache elimination instruction eliminates the inactive user from the cache according to the activity level, and includes: and responding to the cache elimination instruction, and eliminating the users with the lowest activity and preset target number.
It is to be understood that the inactive users are users with the lowest activity and the preset target number may be set according to practical situations, for example, the preset target number is 1, 2 or 3, which is not limited herein.
In a preferred embodiment, before the activity calculation of the target user using the user activity prediction model, the method further comprises:
obtaining the latest access time, access frequency and average access time of all users in the cache, so as to calculate the user activity scores of all users;
and screening out the set number of users with the lowest user activity scores to serve as the target users.
Taking a cache as an example, before carrying out fine screening on an inactive user by using a user activity prediction model, carrying out preliminary screening on the inactive user, if the shared cache meets a rejection trigger condition (cache rejection condition), calculating a popularity score (user activity score) of shared cache data by combining the latest access time, the access frequency and the average access time of each data, and sequencing from small to large according to the popularity score and acquiring Top100 candidate rejected user data as a target user of the fine screening.
It should be understood that the number of settings is not limited to 100, and specific values may be set according to practical applications, and are not limited herein.
In a preferred embodiment, the obtaining the latest access time, access frequency and average access time of all users in the cache for calculating the user activity score of all users includes:
the user activity score is calculated by:
wherein the method comprises the steps of，score Active activity For the user activity score, cnt is the number of user requests, t curr T is the current timestamp of the system last Time stamp of last 1 active for user, t avg For the average user request time, λ is the heat fraction function coefficient.
Illustratively, referring to the recommended API shared cache data structure representation shown in FIG. 2, user activity scores are calculated using equations (1) and (2), and Top100 inactive users are initially screened. Wherein λ in formula (2) is a heat score function coefficient, and is dynamically configurable, and is configured to control the magnitude of heat drop, and the larger the value, the faster the heat score (user activity score) drops, and when λ=3.5, the distribution of the heat scores of the users is shown in fig. 3.
It should be noted that, the user activity score is used to fine-screen the inactive user, in addition to pre-screening a plurality of pieces of user data as candidate elimination data, and also as a key input feature (user cache heat score feature) of the deep learning model.
In a preferred embodiment, the implicit interest-preference friend characteristics of the user include interest-preference characteristics of active friends and interest-preference characteristics of inactive friends.
In a preferred embodiment, the user activity prediction model is specifically trained by:
acquiring known user data to construct a sample set; the known user data includes user base attribute characteristics, user interest preference characteristics, interest preference characteristics of the active friends, interest preference characteristics of the inactive friends, user activity scores, and contextual characteristics of the user;
Performing dimension reduction processing and pooling processing on the user basic attribute characteristics and the user interest preference characteristics respectively to obtain basic attribute pooling vectors and interest preference characteristic pooling vectors; performing dimension reduction processing and horizontal splicing processing on the context characteristics to obtain context splicing vectors;
based on self-attention mechanism, the interest preference characteristics of the active friends and the interest preference characteristics of the non-active friends are respectively processed to obtain active friend sequence self-attention vectors and non-active friend self-attention vectors;
performing Attention operation on the context stitching vector, the basic attribute pooling vector and the interest preference feature pooling vector and the active friend sequence self-Attention vector and the inactive friend self-Attention vector respectively to obtain six target Attention vectors;
and inputting the user activity score, the context stitching vector, the basic attribute pooling vector, the interest preference feature pooling vector and the target attention vector into a deep neural network to perform model training so as to obtain a user activity prediction model.
Specifically, since the deep learning algorithm has natural advantages in terms of mining the relationship between the internal features and the conclusions of things, the proposal adopts the historical access condition of the user in the system to train a deep learning model for predicting the activity of the user, and the method specifically comprises the following steps:
Step 1: training sample construction.
The recommended interface log and the APP end user behavior log are divided according to the Session (the time period from logging in to logging out of the APP by the user is defined as one Session), logs in the Session are ordered according to the ascending sequence of the generation time, and then the recommended log and the APP end log are associated through Session_Id. The training samples (known user data) include user characteristics, liveness tags associated with the user characteristics; the user characteristics include a user base attribute characteristic, a user interest preference characteristic, a user implicit interest preference friend characteristic, a user cache heat score characteristic, and a user context characteristic.
Step 2: liveness model training. Referring to a schematic diagram of a model structure shown in fig. 4, a training sample constructed in step 1 is used to train a user collaborative activity depth model (user activity prediction model), after model training is completed, the model is deployed into an inference server for online real-time calling, and the key steps are as follows:
firstly, inputting basic attributes (gender and age), interest labels and context characteristics of a User into a sharing Tag-Embedding layer, inputting an active friend list and a non-active friend list (active friend sequence and non-active friend sequence characteristics) into the sharing User-Embedding layer to respectively obtain embedded vectors corresponding to characteristic values, horizontally splicing the context characteristic value vectors, inputting the User interest labels and the basic attribute characteristic vectors into a pooling layer (adopting an average pooling mode, namely averaging a plurality of vectors in the same dimension value) to obtain pooling vectors, and inputting the active friend sequence vectors and the non-active friend sequence characteristics into the self-Embedding layer to obtain attention vectors;
Since Attention is used to selectively screen out a small amount of important information from a large amount of information and focus on the important information, most of the unimportant information is ignored. According to the method, context stitching vectors, user interest label pooling vectors and basic attribute pooling vectors are respectively subjected to Attention operation with active friend sequence self-Attention vectors and inactive friend self-Attention vectors, the generated active friend Attention vectors can focus on and fully learn and absorb active relevant features of users in the active friend sequences, and the non-active relevant features of the users in the inactive friend sequences can effectively improve model prediction accuracy.
And finally, inputting the user cache heat score, the context splicing vector, the user interest label pooling vector, the basic attribute pooling vector and the 6 attention vectors into a 2-layer DNN (adopting Relu as an activation function), and finally, inputting the vector output from the topmost end of the DNN into a softMax layer for model training.
In a preferred embodiment, the user implicit interest preference friend feature is determined by:
wherein neighbor (u) a ) Friends are preferred for implicit interests of user a, The ith interest preference score, th for user aThreshold is a similarity determination threshold, +.>An ith interest preference score for user b;
and acquiring the interest preference characteristics of the implicit interest preference friends as the implicit interest preference friend characteristics of the user.
In a preferred embodiment, the obtaining the interest preference feature of the implicit interest preference friend as the implicit interest preference friend feature of the user includes:
calculating the active frequency of the implicit interest preference friends in the average unit time of the latest preset time;
the implicit interest preference friends with the activity frequency larger than a preset activity frequency judging threshold value are used as active friends;
the implicit interest preference friends with the activity frequency smaller than or equal to the activity frequency judging threshold value are used as non-active friends;
and respectively acquiring the interest preference characteristics of the active friends and the inactive friends to serve as the implicit interest preference friend characteristics of the user.
In a preferred embodiment, the user cache popularity score is characterized by the user liveness score; the user interest preference feature is a user content preference feature, and the user content preference feature comprises a content category label and a user operation label associated with the content category label; the content category labels are preconfigured with corresponding codes and category weights, and the user operation labels are preconfigured with corresponding operation weights; the content category comprises a primary category and a secondary category, wherein the primary category comprises at least one of movies, television shows, variety and science and technology; the secondary classification includes at least one of an area a, an area B, and an area C; the user operations include at least one of playing, collecting, commenting, and focusing.
In a preferred embodiment, the contextual characteristics and the liveness tag are determined by:
generating a context feature of the user from a first user behavior log of each session;
and according to the time interval generated by the first behavior log in 2 adjacent sessions of the user, dividing the liveness of the user according to the principle that the shorter the interval time is, the more lively the user is, and obtaining an liveness label.
Illustratively, the step of obtaining a user liveness prediction model includes training a sample configuration and liveness model training. For the training sample construction part, the recommended interface log and the APP end user behavior log are divided according to the Session (the time period from logging in to logging out of the APP by the user is defined as one Session each time), the logs in the Session are ordered according to the ascending sequence of the generation time, and then the recommended log and the APP end log are associated through Session_Id. Referring to the schematic diagram of the model training set shown in fig. 5, the model training set records data of a plurality of users, including user characteristics and user tag liveness tags, and the detailed acquisition process is as follows:
1. user base attribute features. The method comprises the steps of directly acquiring the first interface log of the recommended server side in the Session according to the gender and the age. For example, sex characteristics male (5000), female (5002). User age group characteristics 19-22 years (5022), 31-35 years (5028) are generated according to predefined age group coding rules, wherein the brackets are numbered as category codes.
2. The user interest preference feature, namely the user content preference feature, is specifically generated by adopting the following method:
content encoding, such as primary classification movies (11), television shows (12), shows (13), technology (14), secondary classification into primary classification sub-types, such as variety sub-type a region variety (1321), B region variety (1322), C region variety (1323), etc., technology sub-type leading-edge technology (1421), new material (1422), artificial intelligence (1423), biotechnology (1424), etc. Assuming that a program is classified as a category of one class and a category of two classes as a category of region C, the program content tag is encoded as 13|1323.
User tag coding, setting weights for different operations of a user, such as playing to be 1, collecting, commenting, paying attention to be 1.5, and the like; weights are set for different levels of labels of the program, for example, a primary label of 0.2, a secondary label of 0.5, etc. The codes of the users are the same as the codes of the programs, for example, the users watch the programs of the variety of the area C, the artificial intelligence and the biotechnology, the interest value of the technology is 0.2+0.2, the interest value of the variety of the area C is 0.2, and the interest score of the variety of the area C, the artificial intelligence and the biotechnology tag is 0.5. In connection with the encoding of the program, the user's code is 14:0.4|1423:0.5, 1424:0.5;13:0.2|1323:0.5.
And calculating user labels offline, namely calculating the interest label characteristics of all users once by adopting the total quantity of the recent N-day behavior logs, and calculating the interest label characteristics of the users in real time in an increment way aiming at the users with behaviors on the same day.
3. And (3) calculating the implicit interest preference friends of the user by using cosine similarity according to the defined user interest preference characteristics, wherein the implicit interest preference friends characteristics of the user are similar to the user interest preferences, and a specific calculation formula is shown as a formula (3).
The user behavior log is counted in real time, and the activity frequency of the user in the last time (for example, the last 1 day) in the average unit time (hour) is calculated. According to the active frequency of the user, the implicit interest preference friends of the user are divided into active friends and inactive friends, and the 2 interest friends are used as 2 model key features to participate in subsequent model training, namely:
neighbor(u a )+＝{fred(u)＞threshold + ，u∈neighbor(u a )}； (4)
wherein neighbor (u) a ) + As active friends, fred (u) is the average frequency of activity per unit time of the user, threshold + Threshold is determined for the activity frequency, whereas the non-active friends (neighbor (u) a ) - ) And the same is true.
neighbor(u a )＝neighbor(u a )+∪neighbor(u a ) - 。 (5)
4. The user cache heat score feature is a real-time statistic class feature calculated according to the self characteristics of the user cache data, and is a key element for judging whether the user is active in the future.
5. The context features of the user are used to characterize the time, place and mobile phone information of the user behavior. A user's contextual features, such as cell phone brands, are generated from the first user behavior log per Session, the time (hours level) at which the behavior log was generated. According to the time interval generated by the first behavior log in 2 adjacent sessions of the user, the user is divided into 5 categories of very active (0), more active (1), general (2), less active (3) and inactive (4) as labels of training samples according to the principle that the shorter the interval time is, the more active the user is.
As shown in FIG. 5, for a User1 sex male, ages between 19 and 22 years, the first-class preference synthetic technique and technological interest values are respectively 0.2 and 0.4, the second-class preference artificial intelligence, biotechnology and region C synthetic technique interest values are respectively 0.5, 0.3 and 0.9, the cache heat score is 0.8, active friends similar to the User1 interest are Userx11 and Userx12, inactive friends are Usery11, and APP is used by an iphone mobile phone at 15 afternoon, so that the sex male sex is a very active User; similarly, other users and so on. Notably, each feature domain feature consists of two parts, value and score (value: score), e.g., 5000:1 indicates gender as male, score 1; similarly, the other feature domain feature values are analogized.
In a preferred embodiment, further comprising:
responding to a user portrait tag request, and searching whether user data associated with the request exists in the cache;
if yes, acquiring a user portrait tag associated with the request from the cache and updating the latest access time, the access frequency and the average access time of the user;
if not, obtaining the user portrait tag associated with the request from a user portrait tag library, and writing the user portrait tag into the cache and updating the latest access time, access frequency and average access time of the user when the cache elimination instruction is not triggered.
In a preferred embodiment, when the user portrait tag is acquired, acquiring tag program inverted data;
calculating the program scores of all programs according to the inverted data of the label programs and the user portrait labels;
screening the preset number of programs with higher program scores as personalized recall programs for users and feeding back recommendation results. Preferably, before the recommendation result is fed back, after the user personalized recall program is filtered by using the blacklist and the offline content, the user personalized recall program is finely ordered.
Exemplary, a local shared cache with a limited size is created first, and is used for caching portrait tag information of a user, and key steps of automatic API shared cache update are as follows:
1. when the API initiates a request for acquiring a user portrait tag, firstly searching whether the local shared cache has the data of the user, if so, updating the latest access time, average time and access frequency of the user data, and returning a result;
2. if the user portrait tag library does not exist, firstly acquiring tag information of the user from the user portrait tag library, then judging whether the triggering condition of the rejection cache is met, if not, directly writing the tag information of the user into a shared cache (updating the latest access time, average time and access frequency of the user data) and returning a result;
3. and if the shared cache meets the rejection trigger condition, calculating a hotness score (initially screening inactive users) of the data in the shared cache by combining the latest access time, the average time and the access frequency of each data, sorting from small to large according to the hotness score, and acquiring Top100 candidate obsolete user data.
4. And calling a user collaborative activity judging service, acquiring respective activity scores of Top100 candidate elimination users, and eliminating the least active users (users with the highest model prediction probability).
5. And (3) eliminating the inactive user data returned in the step 4 from the shared cache, writing the newly acquired user portrait tag data into the cache (updating the latest access time, average time and access frequency of the user data) and returning the result.
When the recommendation API acquires the user tag information, acquiring tag program inverted data from a local or Redis, wherein the Score of the program is a hotness value; if multiple tags of the user recall the same program, the user tag interest value is accumulated as the final Score of the program. And sorting the programs according to the scores from large to small, and returning to the TopN program as the personalized recall program of the user. And filtering the blacklist and the offline content, finely sorting the recalled programs through a personalized sorting system, writing exposure after some business breaking rules, returning a recommended result, and ending.
In a preferred embodiment, further comprising: and generating the user portrait tag according to the user interest preference characteristics and the user basic attribute characteristics.
Specifically, 1, coding content labels, namely coding labels of different layers of an article, and establishing an up-down dependency relationship of the labels of the different layers; 2. different weights are set for different operations of the user, and different scores are set for labels of different levels of the program. And generating portrait tag information for each user by using offline and real-time computing services and adopting user basic attribute features, article tag features and user behavior logs.
Compared with the prior art, the embodiment of the invention creates a set of recommended API shared cache updating mechanism, and a two-step screening scheme of carrying out preliminary screening from the perspective of cache data (recently accessed time, average time and frequency) and fine screening from the perspective of user collaborative activity provides a set of API shared cache updating mechanism, and the user activity is further judged on the basis of carrying out preliminary screening on elimination data from the cache data, so that inactive users are accurately screened from candidate elimination data, and the cache hit rate and the self-adaptability are improved. During accurate screening, a pre-trained 'user collaborative activity model' is adopted to determine the user activity, the internal relation between the user characteristics and the activity is deeply mined, a user collaborative activity model structure and a training set acquisition mode are provided, friend activity information which is preferred by the implicit interests of the user and the self heat score of the user cache are introduced into the model, and the model prediction accuracy is effectively improved.
The embodiment of the invention also provides a cache management device, which comprises:
the liveness calculation module is used for calculating liveness of the target user by utilizing the user liveness prediction model; wherein the target user is a user in a cache;
The cache processing module is used for responding to a cache elimination instruction and eliminating inactive users from the cache according to the activity level;
the user liveness prediction model is obtained through training in the following mode:
obtaining known user data to construct a sample set, wherein the known user data comprises user features and liveness tags associated with the user features; the user features include at least one of user base attribute features, user interest preference features, user implicit interest preference friend features, user activity scores, and user context features;
and performing model training by using the sample set to obtain a user liveness prediction model.
It should be noted that, the specific execution steps of the cache management device according to the embodiment of the present invention are the same as those of the cache management method according to any one of the above embodiments, and are not described herein.
Compared with the prior art, the cache management device disclosed by the embodiment of the invention calculates the activity of the target user by utilizing the user activity prediction model; wherein the target user is a user in a cache; responding to a cache elimination instruction, and eliminating inactive users from the cache according to the activity level; the user liveness prediction model is obtained through training in the following mode: obtaining known user data to construct a sample set, wherein the known user data comprises user features and liveness tags associated with the user features; the user characteristics comprise at least one of user basic attribute characteristics, user interest preference characteristics, user implicit interest preference friend characteristics, user cache heat score characteristics and user context characteristics; and performing model training by using the sample set to obtain a user liveness prediction model. Therefore, the embodiment of the invention can utilize the user liveness prediction model to calculate the liveness of the users in the cache, respond to the cache elimination instruction, eliminate the non-active users according to the liveness, and improve the cache hit rate, thereby improving the cache performance.
The embodiment of the invention also provides an electronic device, which comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, wherein the steps in the embodiment of the cache management method are realized by the processor when the computer program is executed, for example, steps S1-S2 in FIG. 1; alternatively, the processor may implement the functions of the modules in the above-described device embodiments when executing the computer program.
The computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to accomplish the present invention, for example. The one or more modules may be a series of computer program instruction segments capable of performing the specified functions, which are used to describe the execution of the computer program in the electronic device. For example, the computer program may be divided into an activity calculating module and a cache processing module, where each module specifically functions as follows:
the liveness calculation module is used for calculating liveness of the target user by utilizing the user liveness prediction model; wherein the target user is a user in a cache;
The cache processing module is used for responding to a cache elimination instruction and eliminating inactive users from the cache according to the activity level;
the user liveness prediction model is obtained through training in the following mode:
obtaining known user data to construct a sample set, wherein the known user data comprises user features and liveness tags associated with the user features; the user characteristics comprise at least one of user basic attribute characteristics, user interest preference characteristics, user implicit interest preference friend characteristics, user cache heat score characteristics and user context characteristics;
and performing model training by using the sample set to obtain a user liveness prediction model.
The specific working process of each module may refer to the working process of the cache management device described in the foregoing embodiment, and will not be described herein.
The electronic equipment can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing equipment. The electronic device may include, but is not limited to, a processor, a memory. Those skilled in the art will appreciate that the electronic devices may also include input-output devices, network access devices, buses, and the like.
The processor may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like that is a control center of the electronic device, connecting various parts of the overall electronic device using various interfaces and lines.
The memory may be used to store the computer program and/or modules, and the processor may implement various functions of the electronic device by running or executing the computer program and/or modules stored in the memory and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as an image playing function, etc.) required for at least one function, etc.; the storage data area may store data created according to the use of the cellular phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart Media Card (SMC), secure Digital (SD) Card, flash Card (Flash Card), at least one disk storage device, flash memory device, or other volatile solid-state storage device.
Wherein the electronic device integrated modules may be stored in a computer readable storage medium if implemented in the form of software functional units and sold or used as a stand alone product. Based on such understanding, the present invention may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth.
While the foregoing is directed to the preferred embodiments of the present invention, it will be appreciated by those skilled in the art that changes and modifications may be made without departing from the principles of the invention, such changes and modifications are also intended to be within the scope of the invention.
Claims (10)
1. A cache management method, comprising:
calculating the activity of a target user by using a user activity prediction model; wherein the target user is a user in a cache;
responding to a cache elimination instruction, and eliminating inactive users from the cache according to the activity level;
the user liveness prediction model is obtained through training in the following mode:
obtaining known user data to construct a sample set, wherein the known user data comprises user features and liveness tags associated with the user features; the user features include at least one of user base attribute features, user interest preference features, user implicit interest preference friend features, user activity scores, and user context features;
and performing model training by using the sample set to obtain a user liveness prediction model.
2. The cache management method as recited in claim 1, further comprising, prior to the activity calculation of the target user using the user activity prediction model:
obtaining the latest access time, access frequency and average access time of all users in the cache, so as to calculate the user activity scores of all users;
And screening out the set number of users with the lowest user activity scores to serve as the target users.
3. The cache management method as recited in claim 2, wherein said obtaining the latest access time, access frequency, and average access time of all users in the cache for calculating the user activity scores of all users comprises:
the user activity score is calculated by:
wherein score Active activity For the user activity score, cnt is the number of user requests, t curr T is the current timestamp of the system last Time stamp of last 1 active for user, t avg For the average user request time, λ is the heat fraction function coefficient.
4. The cache management method according to any one of claims 1 to 3, wherein the implicit interest preference friend characteristics of the user include interest preference characteristics of active friends and interest preference characteristics of inactive friends, and the user activity prediction model is specifically trained by the following modes:
acquiring known user data to construct a sample set; the known user data includes user base attribute characteristics, user interest preference characteristics, interest preference characteristics of the active friends, interest preference characteristics of the inactive friends, user activity scores, and contextual characteristics of the user;
Performing dimension reduction processing and pooling processing on the user basic attribute characteristics and the user interest preference characteristics respectively to obtain basic attribute pooling vectors and interest preference characteristic pooling vectors; performing dimension reduction processing and horizontal splicing processing on the context characteristics to obtain context splicing vectors;
based on self-attention mechanism, the interest preference characteristics of the active friends and the interest preference characteristics of the non-active friends are respectively processed to obtain active friend sequence self-attention vectors and non-active friend self-attention vectors;
performing Attention operation on the context stitching vector, the basic attribute pooling vector and the interest preference feature pooling vector and the active friend sequence self-Attention vector and the inactive friend self-Attention vector respectively to obtain six target Attention vectors;
and inputting the user activity score, the context stitching vector, the basic attribute pooling vector, the interest preference feature pooling vector and the target attention vector into a deep neural network to perform model training so as to obtain a user activity prediction model.
5. A cache management method according to any one of claims 1 to 3, wherein the user implicit interest preference friend characteristics are determined by:
Wherein, neighbor (u) a ) Friends are preferred for implicit interests of user a,for the ith interest preference score of user a, threshold is the similarity determination threshold, ++>An ith interest preference score for user b;
and acquiring the interest preference characteristics of the implicit interest preference friends as the implicit interest preference friend characteristics of the user.
6. The cache management method as recited in claim 5, wherein the obtaining the interest preference feature of the implicit interest preference friend as the implicit interest preference friend feature of the user comprises:
calculating the active frequency of the implicit interest preference friends in the average unit time of the latest preset time;
the implicit interest preference friends with the activity frequency larger than a preset activity frequency judging threshold value are used as active friends;
the implicit interest preference friends with the activity frequency smaller than or equal to the activity frequency judging threshold value are used as non-active friends;
and respectively acquiring the interest preference characteristics of the active friends and the inactive friends to serve as the implicit interest preference friend characteristics of the user.
7. A cache management method according to any one of claims 1 to 3, wherein said user interest preference feature comprises a content category label, a user operation label associated with said content category label; the content category labels are preconfigured with corresponding codes and category weights, and the user operation labels are preconfigured with corresponding operation weights; the user operation includes at least one of playing, collecting, commenting, and focusing.
8. A cache management method according to any one of claims 1 to 3, wherein said context feature and said liveness flag are determined by:
generating a context feature of the user from a first user behavior log of each session;
and according to the time interval generated by the first behavior log in 2 adjacent sessions of the user, dividing the liveness of the user according to the principle that the shorter the interval time is, the more lively the user is, and obtaining an liveness label.
9. An electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the cache management method according to any one of claims 1 to 8 when the computer program is executed.
10. A computer readable storage medium, characterized in that the computer readable storage medium comprises a stored computer program, wherein the computer program when run controls a device in which the computer readable storage medium is located to perform the cache management method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310855924.9A CN117453585A (en) | 2023-07-12 | 2023-07-12 | Cache management method, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310855924.9A CN117453585A (en) | 2023-07-12 | 2023-07-12 | Cache management method, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117453585A true CN117453585A (en) | 2024-01-26 |
Family
ID=89589821
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310855924.9A Pending CN117453585A (en) | 2023-07-12 | 2023-07-12 | Cache management method, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117453585A (en) |
-
2023
- 2023-07-12 CN CN202310855924.9A patent/CN117453585A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10152479B1 (en) | Selecting representative media items based on match information | |
| Ali et al. | Intelligent web proxy caching approaches based on machine learning techniques | |
| EP3732631A1 (en) | Neural architecture search for dense image prediction tasks | |
| CN112052387B (en) | Content recommendation method, device and computer readable storage medium | |
| US11531831B2 (en) | Managing machine learning features | |
| US10846241B2 (en) | Score-based cache admission and eviction | |
| CN109086439A (en) | Information recommendation method and device | |
| US11360927B1 (en) | Architecture for predicting network access probability of data files accessible over a computer network | |
| US10262265B2 (en) | Systems and methods for generating and communicating application recommendations at uninstall time | |
| CN109033261B (en) | Image processing method, image processing apparatus, image processing device, and storage medium | |
| CN110119477B (en) | Information pushing method, device and storage medium | |
| CN110334356A (en) | Article matter method for determination of amount, article screening technique and corresponding device | |
| Song et al. | Query-less: Predicting task repetition for nextgen proactive search and recommendation engines | |
| US20180336459A1 (en) | Unstructured key definitions for optimal performance | |
| CN111460384A (en) | Policy evaluation method, device and equipment | |
| CN106844538A (en) | A kind of many attribute sort methods and device for being applied to Internet of Things search | |
| CN115879508A (en) | Data processing method and related device | |
| CN118093962A (en) | Data retrieval method, device, system, electronic equipment and readable storage medium | |
| US20160132771A1 (en) | Application Complexity Computation | |
| CN112749296A (en) | Video recommendation method and device, server and storage medium | |
| CN110162535B (en) | Search method, apparatus, device and storage medium for performing personalization | |
| CN117453585A (en) | Cache management method, electronic equipment and storage medium | |
| CN114491093B (en) | Multimedia resource recommendation and object representation network generation method and device | |
| CN115114415A (en) | Question-answer knowledge base updating method and device, computer equipment and storage medium | |
| Baskaran et al. | Study of combined Web prefetching with Web caching based on machine learning technique |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |