CN114066278A

CN114066278A - Method, apparatus, medium, and program product for evaluating article recall

Info

Publication number: CN114066278A
Application number: CN202111388301.2A
Authority: CN
Inventors: 谭云飞; 苗晨曦; 刘晓庆
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-11-22
Filing date: 2021-11-22
Publication date: 2022-02-18
Anticipated expiration: 2041-11-22
Also published as: CN114066278B

Abstract

The disclosure provides an article recall assessment method, an article recall assessment device, an article recall assessment medium and a program product, and relates to the technical field of artificial intelligence. One embodiment of the method comprises: acquiring historical behavior information of a target user, wherein the historical behavior information of the target user is used for representing the operation of the target user on a first article; acquiring a second item from a first vector index library according to the historical behavior information of the target user, wherein the first vector index library is determined based on the following steps: acquiring target map data for an item; constructing a first vector index library by using target graph data, wherein nodes of the target graph data comprise historical behavior information of a plurality of users, and edges of the target graph data are weights corresponding to behavior attributes of the historical behavior information of the plurality of users; and determining a target accuracy index value according to the difference information between the second article and the first article.

Description

Method, apparatus, medium, and program product for evaluating article recall

Technical Field

The present disclosure relates to the field of computers, and more particularly to the field of artificial intelligence, and more particularly to a method, an apparatus, a medium, and a program product for evaluating an item recall.

Background

At present, the evaluation method of article recall takes a loss function of model off-line training as a final target to measure the off-line effect of article recall.

Disclosure of Invention

The embodiment of the disclosure provides an article recall evaluation method, an article recall evaluation device, an article recall evaluation medium and a program product.

In a first aspect, an embodiment of the present disclosure provides an item recall evaluation method, including: acquiring historical behavior information of a target user, wherein the historical behavior information of the target user is used for representing the operation of the target user on a first article; acquiring a second item from a first vector index library according to the historical behavior information of the target user, wherein the first vector index library is determined based on the following steps: acquiring target map data for an item; constructing a first vector index library by using target graph data, wherein nodes of the target graph data comprise historical behavior information of a plurality of users, and edges of the target graph data are weights corresponding to behavior attributes of the historical behavior information of the plurality of users; and determining a target accuracy index value according to the difference information between the second article and the first article.

In a second aspect, an embodiment of the present disclosure provides an apparatus for evaluating an item recall, including: the information acquisition module is configured to acquire historical behavior information of a target user, and the historical behavior information of the target user is used for representing the operation of the target user on the first article; an item recall module configured to obtain a second item from a first vector index repository according to historical behavior information of a target user, wherein the first vector index repository is determined based on the following steps: acquiring target map data for an item; constructing a first vector index library by using target graph data, wherein nodes of the target graph data comprise historical behavior information of a plurality of users, and edges of the target graph data are weights corresponding to behavior attributes of the historical behavior information of the plurality of users; an index determination module configured to determine a target accuracy index value based on difference information between the second item and the first item.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

In a fourth aspect, the disclosed embodiments propose a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method as described in the first aspect.

In a fifth aspect, the disclosed embodiments propose a computer program product comprising a computer program that, when executed by a processor, implements the method as described in the first aspect.

The item recall evaluation method, device, medium, and program product provided by the embodiments of the present disclosure can recall a second item from a first vector index library constructed from target graph data according to historical behavior information of a target user, and determine a target accuracy index value based on difference information between the second item and a first item corresponding to the historical behavior information of the target user, thereby implementing evaluation of the accuracy of the target graph data recall item.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

Other features, objects, and advantages of the disclosure will become apparent from a reading of the following detailed description of non-limiting embodiments which proceeds with reference to the accompanying drawings. The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is an exemplary system architecture diagram in which the present disclosure may be applied;

FIG. 2 is a flow chart of one embodiment of a method of evaluating an item recall according to the present disclosure;

FIG. 3 is a flow chart of one embodiment of a method of evaluating an item recall according to the present disclosure;

FIG. 4 is a schematic diagram of node vectorization according to the present disclosure;

FIG. 5 is a schematic diagram of an implementation adjustment strategy according to the present disclosure;

FIG. 6 is a schematic diagram of one application scenario of an assessment method of item recall according to the present disclosure;

FIG. 7 is a schematic block diagram illustrating one embodiment of an evaluation device for item recall in accordance with the present disclosure;

FIG. 8 is a block diagram of an electronic device used to implement an embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

Fig. 1 illustrates an exemplary system architecture 100 to which embodiments of the item recall evaluation method or item recall evaluation apparatus of the present disclosure may be applied.

As shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The

terminal devices

101, 102, 103 may have installed thereon various client applications, intelligent interactive applications, such as search applications, and the like.

The

terminal devices

101, 102, and 103 may obtain historical behavior information of the target user, where the historical behavior information of the target user is used to represent an operation of the target user on the first article; acquiring a second item from a first vector index library according to the historical behavior information of the target user, wherein the first vector index library is determined based on the following steps: acquiring target map data for an item; constructing a first vector index library by using target graph data, wherein nodes of the target graph data comprise historical behavior information of a plurality of users, and edges of the target graph data are weights corresponding to behavior attributes of the historical behavior information of the plurality of users; and determining a target accuracy index value according to the difference information between the second article and the first article.

The

terminal apparatuses

101, 102, and 103 may be hardware or software. When the

terminal devices

101, 102, and 103 are hardware, the terminal devices may be electronic products that perform human-Computer interaction with a user through one or more modes of a keyboard, a touch pad, a touch screen, a remote controller, voice interaction, or handwriting equipment, such as a PC (Personal Computer), a mobile phone, a smart phone, a PDA (Personal Digital Assistant), a wearable device, a PPC (Pocket PC, palmtop), a tablet Computer, a smart car machine, a smart television, a smart speaker, a tablet Computer, a laptop Computer, a desktop Computer, and the like. When the

terminal apparatuses

101, 102, 103 are software, they can be installed in the above-described electronic apparatuses. It may be implemented as multiple pieces of software or software modules, or as a single piece of software or software module. And is not particularly limited herein.

The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster composed of a plurality of servers, or may be implemented as a single server. When the server 105 is software, it may be implemented as multiple pieces of software or software modules (e.g., to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.

It should be noted that the method for evaluating an item recall provided by the embodiment of the present disclosure is generally executed by the server 105, and accordingly, an evaluation apparatus for an item recall is generally disposed in the server 105.

It should be understood that the number of electronic devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of electronic devices, networks, and servers, as desired for implementation.

With continued reference to FIG. 2, a flow 200 of one embodiment of a method of evaluating an item recall according to the present disclosure is shown. The item recall evaluation method may include the steps of:

step 201, obtaining historical behavior information of a target user, wherein the historical behavior information of the target user is used for representing the operation of the target user on a first article.

In the present embodiment, the execution subject of the evaluation method of item recall (e.g.,

terminal apparatuses

101, 102, 103 shown in fig. 1) may locally acquire or externally acquire the historical behavior information of the target user. The historical behavior information of the target user can be used for characterizing the operation of the target user on the first item, such as clicking on the first item, collecting the first item, and browsing the first item.

In this embodiment, before obtaining the historical behavior information of the target user, the method for evaluating item recall may further include: acquiring a test set; and acquiring historical behavior information of the target user from the test set.

It should be noted that the historical behavior information of the target user may be part or all of the information in the test set.

According to the technical scheme, the historical behavior information of the target user, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the first article are all in accordance with the regulations of relevant laws and regulations, and do not violate the good custom of the public order.

Step 202, acquiring a second item from a first vector index library according to the historical behavior information of the target user, wherein the first vector index library is determined based on the following steps: acquiring target graph data, wherein nodes of the target graph data comprise historical behavior information of a plurality of users, and edges of the target graph data are weights corresponding to behavior attributes of the historical behavior information of the users; a first vector index library is constructed using the target graph data.

In an embodiment, the execution subject may recall the second item from the first vector index repository according to the historical behavior information of the target user. The first vector index repository may be used to recall items from the target user based on historical behavioral information of the target user.

Here, the first vector index repository may be determined based on the following steps:

acquiring target graph data, wherein the target graph data comprises nodes and edges, the nodes of the target graph data comprise historical behavior information of a plurality of users, and the edges of the target graph data are weights corresponding to behavior attributes of the historical behavior information of the users; a first vector index library is constructed using the target graph data.

Correspondingly, in this example, historical behavior information of the plurality of users may be used to characterize the operation of the item by the plurality of users. The behavior attribute of the historical behavior information of the plurality of users may be used to characterize the attribute of the historical behavior information of the plurality of users, such as the behavior times, the behavior type, e.g., click, browse 3 times, and so on.

In this embodiment, the weights corresponding to the behavior attributes of the historical behavior information of the plurality of users may be set according to the accuracy of the recalled articles.

Step 203, determining an accuracy index value according to the difference information between the second article and the first article.

In this embodiment, the executing entity may determine the target accuracy index value according to the difference information between the second item and the first item. The difference information may be used to characterize differences between the second item and the first item, e.g., the first item may comprise items that are different from the items comprised by the second item or the first item may comprise items in an order that is different from the order of the items comprised by the second item. The accuracy index value may be used to characterize the difference between the second item and the first item.

In one example, the first item includes "a, b, c", the second item includes "a, b, e", the first item includes "c" that is different from the second item includes "e", and the accuracy index value is 2/3.

The method for evaluating the article recall provided by the embodiment of the disclosure comprises the steps of firstly, obtaining historical behavior information of a target user, wherein the historical behavior information of the target user is used for representing the operation of the target user on a first article; and then acquiring a second article from a first vector index library according to the historical behavior information of the target user, wherein the first vector index library is determined based on the following steps: acquiring target map data for an item; constructing a first vector index library by using target graph data, wherein nodes of the target graph data comprise historical behavior information of a plurality of users, and edges of the target graph data are weights corresponding to behavior attributes of the historical behavior information of the plurality of users; and finally, determining a target accuracy index value according to the difference information between the second article and the first article. The second item can be recalled from the first vector index library constructed according to the target map data, and the target accuracy index value is determined based on the difference information of the first item and the second item, so that the accuracy of the target map data recall item is evaluated.

With further reference to fig. 3, fig. 3 illustrates a flow 300 of one embodiment of an assessment method of item recall according to the present disclosure. The item recall evaluation method may include the steps of:

step 301, obtaining historical behavior information of the target user, where the historical behavior information of the target user is used to represent the operation of the target user on the first article.

terminal apparatuses

Step 302, according to the historical behavior information of the target user, obtaining a third article from the first vector index library, where the third article is an article corresponding to the historical behavior information of other users, and the historical behavior information of other users is used to represent operations of other users on the third article.

In this embodiment, the execution main body may obtain historical behavior information of other users from the first vector index library according to the historical behavior information of the target user; then, obtaining the operation of a third article based on the historical behavior information of other users; the historical behavior information of the other users can be used for representing the operation of the other users on the third article, and the similarity between the historical behavior information of the other users and the historical behavior information of the target user meets a preset similarity threshold.

It should be noted that the number of other users may be at least one. The preset similarity threshold value can be set according to the accuracy of the recalled articles.

Here, the first vector index repository may be determined based on the following steps: acquiring target graph data, wherein the target graph data comprises nodes and edges, the nodes of the target graph data comprise historical behavior information of a plurality of users, and the edges of the target graph data are weights corresponding to behavior attributes of the historical behavior information of the users; a first vector index library is constructed using the target graph data.

And step 303, acquiring the second article from the second vector index library according to the third article.

In this embodiment, the execution subject may recall the second item from the second vector index library according to the third item. The second vector index library described above may be used to recall the second item from it based on the third item.

Here, the second vector index library may be determined based on the following steps: and constructing the second vector index library by using a nearest neighbor algorithm according to the articles corresponding to the historical behavior information of other users. The nearest neighbor algorithm may search, according to the similarity between the articles, an article that is most similar to the first article corresponding to the historical behavior information of the target user from the articles corresponding to the historical behavior information of the multiple users, so as to construct the second vector index library. This similarity is usually quantified as the distance between the vectors in space, and it can be considered that the closer the data is in space, the higher the similarity between the items.

In this embodiment, the executing entity may perform clustering on the items corresponding to the historical behavior information of the other users whose similarity with the historical behavior information of the target user satisfies a preset similarity threshold, so as to construct the second vector index library.

Correspondingly, in this example, the above-mentioned clustering may be that the similarity between the items satisfies a preset similarity threshold.

In one example, a collaborative filtering recommendation algorithm (e.g., a collaborative filtering algorithm (UserCF) of a user, and a collaborative filtering of an item (ItemCF)) is taken as an example, and a process of acquiring and recommending related items to a user is performed according to historical behavior data of a target user.

Correspondingly, in this example, the historical behavior information of the multiple users and the vectors of the item nodes (i.e., the items corresponding to the historical behavior information of the multiple users) are respectively created by the ANN algorithm into a first vector index library corresponding to the user nodes and a second vector index library corresponding to the commodity nodes, and the latest top k items (i.e., the second items) can be recalled from the historical behavior data of the target user through the first vector index library and the second vector index library.

It should be noted that k may be set according to the accuracy of the article recall.

Step 304, determining a target accuracy index value according to the difference information between the second article and the first article.

In this embodiment, the specific operations of

steps

301 and 304 have been described in detail in

steps

201 and 203, respectively, in the embodiment shown in fig. 2, and are not described again here.

As can be seen from fig. 3, compared with the embodiment corresponding to fig. 2, the method for evaluating an item recall in this embodiment highlights a step of obtaining a second item from a first vector index library according to the historical behavior information of the target user. Therefore, in the scheme described in this embodiment, when the third item is acquired from the first vector index library according to the historical behavior information of the target user, the second item is recalled from the second vector index library. Thereby enabling the second item to be recalled quickly.

In some optional implementations of this embodiment, constructing the first vector index library using the target graph data may include: obtaining a target vector of a node of the target graph data according to a graph embedding algorithm; and constructing a first vector index library according to the target vectors of the nodes of the target graph data by utilizing a nearest neighbor algorithm.

In this implementation, the execution body may obtain a target vector of a node of the target graph data according to a graph embedding algorithm; and then, constructing a first vector index library according to the target vectors of the nodes of the target graph data by utilizing a nearest neighbor algorithm. The nearest neighbor algorithm can search other users which are most similar to the target user from historical behavior information of a plurality of users according to the similarity between the users, so that a first vector index library is constructed. This similarity is usually quantified as the distance between the vectors in space, and it can be considered that the closer the data is in space, the higher the similarity between users.

Correspondingly, in this example, constructing the first vector index repository from the target vectors of the nodes of the target graph data may include: constructing a first vector index library according to other user vectors of other users of which the similarity with the target user vector of the target user meets a preset similarity threshold; the target user vector of the target user may be a vector corresponding to the historical behavior information of the target user, and the other user vectors of the other users may be vectors corresponding to the historical behavior information of the other users.

In this implementation, the graph embedding algorithm may include: a weight walking algorithm, a random walking algorithm, a large-scale information network embedding algorithm and the like. And determining user vectors corresponding to the nodes in the graph according to a graph embedding algorithm.

In this implementation, a target vector of a node of the target graph data may be obtained according to a graph embedding algorithm; and then, constructing a first vector index library by using the target vectors of the nodes of the target graph data.

In some optional implementations of this embodiment, obtaining the target vector of the node of the target graph data according to the graph embedding algorithm may include: inputting historical behavior information of a plurality of users into a first preset vector conversion model, and determining an initial vector of a node of target graph data; and inputting the initial vector into a second preset vector conversion model to obtain a target vector.

In this implementation manner, the execution subject may input historical behavior information of a plurality of users into a first preset vector conversion model to obtain an initial vector of a node of the target graph data; and then, inputting the initial vector into a second preset vector conversion model to obtain a target vector.

Here, the first preset vector conversion model may be used to vectorize the nodes of the target graph data to obtain an initial vector. The first predetermined vector conversion model may include at least one of: the Graph Neural Network Model (GNN) Model, The Graph convolutional Neural Network (GCN) Model, The Graph Attention Network (Graph Attention Networks) Model.

The second predetermined vector transformation model may be used to transform the initial vector again to obtain the target vector. The second predetermined vector conversion model may include: word2vec (word vector) model, word Frequency-Inverse file Frequency (TFIDF) model.

In one example, as shown in FIG. 4, historical behavior information 401 is obtained for a plurality of users; then, inputting historical behavior information of a plurality of users into the GNN model 402 to obtain an initial vector; then, the initial vector is input into the word2vec model 403 to obtain the target vector.

In this implementation manner, the target vector of the node may be obtained through a first preset vector transformation model and a second preset vector transformation model.

In some optional implementations of this embodiment, the target graph data may be determined based on the following steps: acquiring original image data; and sampling the original image data by using a preset random walk mode to obtain target image data.

In this implementation, the execution subject may obtain original graph data; and then, sampling the original image data by using a preset random walk mode to obtain target image data. The raw map data may be non-sampled map data.

In this implementation, the preset random walk manner may be a manner including equiprobable random walks on the graph according to two meta paths (metapath) of uiu and iui. Where "u" represents a user and "i" represents an item.

In one example, when random walk is performed, a node is randomly sampled from original graph data as a starting point (e.g., i) of the random walk, and an adjacent point with associated weight is uniformly and randomly sampled for a node visited last in the process of the random walk until the random walk reaches a preset node (e.g., i) to end the random walk, so as to finally obtain target graph data.

The number of steps of the random walk may be set according to the accuracy of the target drawing data for retrieving the article.

In the implementation mode, the implicit representation information of each node in the graph can be learned through a random walk algorithm. Wherein, the implicit characterizing information can be understood as implicit association information between the nodes reflected in the graph.

In this implementation manner, the original graph data may be adopted in a preset random walk manner to obtain the target graph data.

In some optional implementations of this embodiment, after determining the target accuracy index value, the method for evaluating an item recall may further include:

and responding to the accuracy index value not meeting the preset index threshold value, and executing a preset adjusting strategy, wherein the preset adjusting strategy is used for adjusting the accuracy index value to meet the preset index threshold value.

In this embodiment, when the accuracy index value does not satisfy the preset index threshold, the execution main body may adopt a corresponding adjustment policy to adjust the accuracy index value to satisfy the preset index threshold.

In one example, when the accuracy index value is smaller than the preset index threshold, the parameters of the target graph data are adjusted, and step 202 and step 203 are executed again until the accuracy index value at this time meets the preset index threshold.

In one example, adjusting the policy may further include at least one of: adjusting parameters of a nearest neighbor algorithm and adjusting parameters of a preset random walk algorithm.

It should be noted that any adjustment strategy that can adjust the target accuracy index value may be included in the scope of the embodiments of the present disclosure.

It should be noted that the preset index threshold may be set according to the accuracy requirement of the user for the recalled item (i.e., the second item).

In one example, as shown in FIG. 5, a test set 501 is obtained; then, converting the historical behavior information of the target user in the test set into a vector 502; then, according to the historical behavior information of the target user converted into the vector, a second item 503 is obtained from the first vector index library and the second vector index library; then, a target accuracy index value 504 is determined according to the difference information of the second item and the first item corresponding to the historical behavior information of the target user.

In this implementation, after the target accuracy index value is adjusted to the preset index threshold, article recall may be performed according to the first vector index library and the second vector index library at this time; the recalled item is then recommended to the target user.

In this implementation manner, when the accuracy index value does not satisfy the preset index threshold, the adjusted accuracy index value may be adjusted to satisfy the preset index threshold by executing the preset adjustment policy, so that the accuracy of article recall is improved.

In some optional implementations of this embodiment, the behavior attribute includes a number of behaviors and/or a type of behavior.

In this implementation, the behavior attribute may also include a number of behaviors and/or a type of behavior.

In one example, the edges of the target graph data are:

s＝a*0.1+b*0.5+c。

wherein, a is the number of times that the user clicks the article, b is the number of times that the user converts the article, c is an initial value, and c is generally 0.

It should be noted that the value of c may be set according to the service scenario and/or the service index.

In this implementation, the edges of the target graph data may be determined based on the number of behaviors and the weight corresponding to the behavior type.

In some optional implementation manners of this embodiment, the obtaining of the historical behavior information of the target user includes:

acquiring historical behavior information of a target user within a preset time period;

determining a target accuracy index value according to the difference information between the second article and the first article, comprising:

determining an accuracy index value corresponding to each time point according to difference information between a second article and a first article corresponding to each time point in a preset time period;

and determining a target accuracy index value according to the accuracy index value corresponding to each time point and a preset time period.

In this implementation manner, the executing body may determine the accuracy index value corresponding to each time point according to the difference information between the second article and the first article corresponding to each time point within the preset time period, and determine the target accuracy index value. The preset time period may be a period of acquiring the historical behavior information of the target user, for example, five days. Each time point within the preset time period may be a time point of the preset time period, for example, each of five days.

In one example, considering that the accuracy index value has a certain fluctuation in time sequence, in order to eliminate the influence of the time sequence, the accuracy index value of each day in 5 days is calculated through a time window and then rolling for 5 days, and finally the accuracy index value of 5 days is subjected to mean smoothing to obtain the target accuracy index value.

Wherein t is each time point, x is the accuracy index value corresponding to each time point t, and y is the target accuracy index value.

It should be noted that after the accuracy index value corresponding to each time point, the accuracy index value corresponding to each time point may be adjusted to meet a preset index threshold; and until the accuracy corresponding to each time point in the preset time period is adjusted. Or after the target accuracy index value is determined according to the accuracy index value corresponding to each time point, the target accuracy index value is adjusted to meet a preset index threshold.

In the implementation mode, the consistency problem of the article recall optimization and the online index can be rapidly solved through the mode of smooth mean value, so that the accuracy rate of article recall and the optimization efficiency are greatly improved.

In some optional implementations of this embodiment, the difference information includes: differences in items and/or differences in the order of items.

In this implementation, the accuracy of the item recall may be determined by differences in items and/or differences in item order. The difference in the items may be that the first item is different from the second item. The difference in the order of the articles may be an order sorted by similarity between the articles.

In one example, the target user in the test set clicks on the item set as (a, b, c) (i.e., the first item), and records the item that the target user clicked most recently as d (i.e., the third item), and indexes d into the library with a second vector, resulting in 4 items with the closest vectors and sorted by the relevance size of the vectors as (a, b, e) (i.e., the first item), so the accuracy index value for the item recall is 2/3.

In this implementation, the accuracy of the item recall may be determined from the differences in items and/or differences in item order.

With further reference to fig. 6, fig. 6 illustrates an application scenario diagram of the item recall evaluation method according to the present disclosure. In this application scenario, a target user enters a "glove" keyword on a search interface of a terminal device (e.g.,

terminal devices

101, 102, 103 shown in fig. 1) 601, and the terminal device 601 may recall a plurality of "glove" items.

With further reference to fig. 7, as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an apparatus for evaluating an item recall, which corresponds to the embodiment of the method shown in fig. 2, and which is particularly applicable to various electronic devices.

As shown in fig. 7, the evaluation apparatus 700 for item recall of the present embodiment may include: an information acquisition module 701, an item recall module 702, and an indicator determination module 703. The information acquisition module 701 is configured to acquire historical behavior information of a target user, wherein the historical behavior information of the target user is used for representing the operation of the target user on the first article; an item recall module 702 configured to obtain a second item from a first vector index repository according to historical behavior information of a target user, wherein the first vector index repository is determined based on the following steps: acquiring target map data for an item; constructing a first vector index library by using target graph data, wherein nodes of the target graph data comprise historical behavior information of a plurality of users, and edges of the target graph data are weights corresponding to behavior attributes of the historical behavior information of the plurality of users; an index determination module 703 configured to determine a target accuracy index value according to difference information between the second item and the first item.

In the present embodiment, in the evaluation apparatus 700 for item recall: the detailed processing and the technical effects of the information obtaining module 701, the article recall module 702 and the index determining module 703 can refer to the related descriptions of step 201 and step 203 in the corresponding embodiment of fig. 2, which are not described herein again.

In some optional implementations of the present embodiment, the evaluation apparatus 700 for item recall further includes: a vector deriving module configured to derive target vectors for nodes of the target graph data according to a graph embedding algorithm; an index repository construction module configured to construct a first vector index repository from target vectors of nodes of the target graph data using a nearest neighbor algorithm.

In some optional implementations of this embodiment, the vector derivation module is further configured to: inputting historical behavior information of a plurality of users into a first preset vector conversion model, and determining an initial vector of a node of target graph data; and inputting the initial vector into a second preset vector conversion model to obtain a target vector.

In some optional implementations of the present embodiment, the evaluation apparatus 700 for item recall further includes: a data acquisition module configured to acquire raw map data; and the data obtaining module is configured to sample the original image data by using a preset random walk mode to obtain target image data.

In some optional implementations of this embodiment, the node of the target graph data further includes: the article node is an article corresponding to the historical behavior information of the plurality of users;

an item recall module 702 further configured to: acquiring a third article from the first vector index library according to the historical behavior information of the target user, wherein the third article is an article corresponding to the historical behavior information of other users, and the historical behavior information of the other users is used for representing the operation of the other users on the third article;

according to the third article, obtaining the second article from a second vector index library, wherein the second vector index library is determined based on the following steps: and constructing a second vector index library by using a nearest neighbor algorithm according to the articles corresponding to the historical behavior information of other users.

In some optional implementations of the present embodiment, the evaluation apparatus 700 for item recall further includes: a policy execution module configured to execute a preset adjustment policy in response to the accuracy index value not satisfying a preset index threshold, wherein the preset adjustment policy is used to adjust the accuracy index value to satisfy the preset index threshold.

In some optional implementations of this embodiment, the information obtaining module 701 is further configured to: acquiring historical behavior information of a target user within a preset time period;

an index determination module 703, further configured to: determining an accuracy index value corresponding to each time point according to difference information between a second article and a first article corresponding to each time point in a preset time period; and determining a target accuracy index value according to the accuracy index value corresponding to each time point and a preset time period.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

FIG. 8 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.

A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

Computing unit 801 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The calculation unit 801 executes the respective methods and processes described above, such as the evaluation method of the item recall. For example, in some embodiments, the method of evaluating an item recall may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and/or installed onto device 800 via ROM 802 and/or communications unit 809. When loaded into RAM 803 and executed by computing unit 801, a computer program may perform one or more steps of the method of assessing an item recall described above. Alternatively, in other embodiments, computing unit 801 may be configured to perform the method of evaluation of item recalls in any other suitable manner (e.g., by way of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

Artificial intelligence is the subject of studying computers to simulate some human mental processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), both at the hardware level and at the software level. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural voice processing technology, machine learning/deep learning, a big data processing technology, a knowledge map technology and the like.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in this disclosure may be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions mentioned in this disclosure can be achieved, and are not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method of assessing item recalls, comprising:

acquiring historical behavior information of a target user, wherein the historical behavior information of the target user is used for representing the operation of the target user on a first article;

acquiring a second item from a first vector index library according to the historical behavior information of the target user, wherein the first vector index library is determined based on the following steps: acquiring target map data for an item; constructing a first vector index library by using the target graph data, wherein nodes of the target graph data comprise historical behavior information of a plurality of users, and edges of the target graph data are weights corresponding to behavior attributes of the historical behavior information of the users;

and determining a target accuracy index value according to the difference information between the second article and the first article.

2. The method of claim 1, wherein the building a first vector index library using the target graph data comprises:

obtaining a target vector of a node of the target graph data according to a graph embedding algorithm;

and constructing the first vector index library according to the target vectors of the nodes of the target graph data by utilizing a nearest neighbor algorithm.

3. The method of claim 2, wherein the deriving target vectors for nodes of the target graph data according to a graph embedding algorithm comprises:

inputting historical behavior information of a plurality of users into a first preset vector conversion model, and determining an initial vector of a node of the target graph data;

and inputting the initial vector into a second preset vector conversion model to obtain the target vector.

4. The method according to any one of claims 1-3, wherein the target map data is determined based on:

acquiring original image data;

and sampling the original image data by using a preset random walk mode to obtain the target image data.

5. The method of any of claims 1-4, wherein the node of the target graph data further comprises: the article node is an article corresponding to the historical behavior information of a plurality of users;

the obtaining of the second item from the first vector index library according to the historical behavior information of the target user includes:

acquiring a third article from a first vector index library according to the historical behavior information of the target user, wherein the third article is an article corresponding to the historical behavior information of other users, and the historical behavior information of the other users is used for representing the operation of the other users on the third article;

according to the third article, obtaining the second article from a second vector index library, wherein the second vector index library is determined based on the following steps: and constructing the second vector index library by using a nearest neighbor algorithm according to the articles corresponding to the historical behavior information of other users.

6. The method according to any one of claims 1-5, further comprising:

and responding to the accuracy index value not meeting a preset index threshold value, and executing a preset adjusting strategy, wherein the preset adjusting strategy is used for adjusting the accuracy index value to meet the preset index threshold value.

7. The method of claim 1, wherein the behavior attributes comprise a number of behaviors and/or a type of behavior.

8. The method of claim 1, wherein the obtaining historical behavior information of the target user comprises:

the determining a target accuracy index value according to the difference information between the second item and the first item includes:

determining an accuracy index value corresponding to each time point according to difference information between a second article and the first article corresponding to each time point in a preset time period;

and determining the target accuracy index value according to the accuracy index value corresponding to each time point and a preset time period.

9. The method of claim 1, wherein the difference information comprises: differences in items and/or differences in the order of items.

10. An item recall assessment apparatus comprising:

the information acquisition module is configured to acquire historical behavior information of a target user, wherein the historical behavior information of the target user is used for representing the operation of the target user on a first article;

an item recall module configured to obtain a second item from a first vector index library according to historical behavior information of the target user, wherein the first vector index library is determined based on the following steps: acquiring target map data for an item; constructing a first vector index library by using the target graph data, wherein nodes of the target graph data comprise historical behavior information of a plurality of users, and edges of the target graph data are weights corresponding to behavior attributes of the historical behavior information of the users;

an index determination module configured to determine a target accuracy index value based on difference information between the second item and the first item.

11. The apparatus of claim 10, the apparatus further comprising:

a vector derivation module configured to derive target vectors for nodes of the target graph data according to a graph embedding algorithm;

an index repository construction module configured to construct the first vector index repository from target vectors of nodes of the target graph data using a nearest neighbor algorithm.

12. The apparatus of claim 11, wherein the vector derivation module is further configured to:

13. The apparatus of any of claims 10-11, further comprising:

a data acquisition module configured to acquire raw map data;

and the data obtaining module is configured to sample the original image data by using a preset random walk mode to obtain the target image data.

14. The apparatus of any of claims 10-13, wherein the node of the target graph data further comprises: the article node is an article corresponding to the historical behavior information of a plurality of users;

the item recall module further configured to:

15. The apparatus of any of claims 10-14, further comprising:

a policy enforcement module configured to enforce a preset adjustment policy in response to the accuracy index value not satisfying a preset index threshold, wherein the preset adjustment policy is used to adjust the accuracy index value to satisfy the preset index threshold.

16. The apparatus of claim 10, wherein the behavior attribute comprises a number of behaviors and/or a type of behavior.

17. The apparatus of claim 10, wherein the information acquisition module is further configured to:

the metric determination module further configured to:

18. The apparatus of claim 10, wherein the difference information comprises: differences in items and/or differences in the order of items.

19. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

20. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-9.

21. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-9.