CN111259249A

CN111259249A - Data screening method, device, equipment and storage medium

Info

Publication number: CN111259249A
Application number: CN202010066506.8A
Authority: CN
Inventors: 许金泉
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2020-01-20
Filing date: 2020-01-20
Publication date: 2020-06-09
Anticipated expiration: 2040-01-20
Also published as: CN111259249B

Abstract

The application discloses a data screening method, a data screening device, data screening equipment and a storage medium, and relates to the field of intelligent search. The specific implementation scheme is as follows: the method is applied to the electronic equipment and comprises the following steps: receiving a data screening request, wherein the data screening request is a deep user screening request and/or a deep interest screening request; responding to a data screening request, and acquiring corresponding depth users and/or corresponding depth interests from a predetermined depth interest user point pair set; the depth interest user point pair set is determined according to the non-biased scores of the converged users for the interests; outputting the corresponding depth user and/or the corresponding depth interest. The bias scores related to the interests and the bias scores related to the users in the scores of the interests of the users can be effectively eliminated, so that the influence of the long tail effect can be effectively eliminated, and the depth users of the interests and/or the depth hobby interests of the users can be accurately determined. And further, the intelligent level of the application program can be improved.

Description

Data screening method, device, equipment and storage medium

Technical Field

The application relates to the technical field of data processing, in particular to an intelligent search technology.

Background

With the rapid development of the mobile internet, the personalized recommendation technology has also been rapidly developed. It is important in personalized recommendation to determine the deep hobby user of interest or to determine the deep hobby interest of the user. The technology for determining the deep interest of the user or the interest of the deep interest of the user is the technology for screening the related data.

In the prior art, a hard threshold truncation method, a soft threshold truncation method or a hard threshold truncation set soft threshold truncation method is generally adopted when determining a deep hobby user of interest or determining the deep hobby interest of the user. When these methods are used, long tail effects are present, for example: a user has a high natural score for all interests and is likely to be a deep hobby user of an interest, however, the user is not actually a deep hobby user of the interest. Or another user has a low natural score for all interests, some of which is the user's deep hobby interests but cannot be recalled.

Therefore, in the prior art, the depth hobby user of interest is determined, or the depth hobby user of interest is determined, due to the existence of the long tail effect, the depth hobby user of interest cannot be determined accurately, or the depth hobby interest of the user cannot be determined accurately.

Disclosure of Invention

The embodiment of the application provides a data screening method, a data screening device, data screening equipment and a storage medium, and solves the technical problems that in the prior art, a deep hobby user of interest can not be accurately determined, or the deep hobby user of interest can not be accurately determined due to the existence of a long tail effect in a method for determining the deep hobby interest of the user.

A first aspect of an embodiment of the present application provides a data screening method, where the method is applied to an electronic device, and the method includes:

receiving a data screening request, wherein the data screening request is a deep user screening request and/or a deep interest screening request; responding to the data screening request, and acquiring corresponding depth users and/or corresponding depth interests from a predetermined depth interest user point pair set; the depth interest user point pair set is determined according to the converged unbiased scores of the interests of the users; outputting the corresponding depth user and/or the corresponding depth interest.

In the embodiment of the application, the deep interest user point pair set is predetermined, and each user interest point pair in the deep interest user point pair set is determined according to the converged unbiased score of each interest of each user, so that the bias score related to the interest and the bias score related to the user in the score of each interest of each user can be effectively eliminated, the influence of the long tail effect can be effectively eliminated, and the interest of the deep user and/or the deep hobby interest of the user can be accurately determined. And further, the intelligent level of the application program can be improved.

Further, the method as described above, before the obtaining the corresponding depth user and/or the corresponding depth interest from the predetermined set of depth interest user points, further includes;

calculating the unbiased scores of the interests of the users during the current iteration; judging whether a preset convergence condition is met; if the preset convergence condition is met, determining the unbiased score of each interest of each user in the current iteration as the converged unbiased score of each interest of each user; and determining the deep interest user point pair set by adopting the converged unbiased scores of the interests of the users.

In the embodiment of the application, when the determined deep interest user point pair set is determined, the unbiased score of each interest of each user in the current iteration is determined, if the condition that the preset convergence condition is met is determined, the unbiased score of each interest of each user in the current iteration is determined to be the unbiased score of each interest of each user after convergence, and then the deep interest user point pair set is determined according to the unbiased score of each interest of each user after convergence, so that the interest user point pairs in the determined deep interest user point pair set can be the deep interest user point pairs with the influence of the long tail effect eliminated, and further the depth interest of the depth users and/or the users with the interest determined are more accurate.

Further, the method as described above, the calculating unbiased scores of interests of users at the current iteration includes:

acquiring unbiased scores of each user on each interest during last iteration; calculating bias scores of each user in the last iteration and bias scores of each interest in the last iteration according to the unbiased scores of each user in the last iteration on each interest; and calculating the unbiased scores of the interests of the users in the current iteration according to the unbiased scores of the interests of the users in the last iteration, the user bias scores corresponding to the last iteration and the interest bias scores corresponding to the last iteration.

In the embodiment of the application, the unbiased scores of the interests of the users in the last iteration, the user bias scores corresponding to the user in the last iteration and the interest bias scores corresponding to the user in the last iteration are adopted to calculate the unbiased scores of the interests of the users in the current iteration, and the bias scores related to the interests and the bias scores related to the users can be effectively eliminated in each iteration, so that the accuracy of the unbiased scores of the users in the interests after convergence is further improved.

Further, the method for calculating the bias scores of the users in the previous iteration and the bias scores of the interests in the previous iteration according to the unbiased scores of the interests of the users in the previous iteration includes:

acquiring unbiased scores corresponding to all interests of each user during last iteration and respectively sorting in a descending order; determining the unbiased scores of preset percentage points in all interests of each user as the biased scores of each user in the last iteration; acquiring unbiased scores corresponding to all the users of all the interests in the last iteration and respectively sorting in a descending order; and determining the unbiased scores of preset percentage points in all the interest users as the bias scores of the interest in the last iteration.

In the embodiment of the application, when the offset scores of all users in the last iteration and the interest offset scores in the last iteration are calculated, the non-offset scores arranged at the preset percentage points are determined as the corresponding offset scores, so that the determined offset scores can be more accurate.

Further, the calculating the unbiased score of each interest of each user in the current iteration according to the unbiased score of each interest of each user in the previous iteration, the user biased score corresponding to the previous iteration, and the interest biased score corresponding to the previous iteration includes:

calculating each first difference value of the unbiased score of each user for each interest during the last iteration and the user bias score corresponding to the last iteration; calculating each first difference value and each second difference value of the interest bias score corresponding to the last iteration; and determining the second difference values as unbiased scores of the interests of the users in the current iteration.

In the embodiment of the application, when the unbiased scoring of each interest of each user in the current iteration is calculated according to the unbiased scoring of each interest of each user in the last iteration, the user bias score corresponding to the last iteration and the interest bias score corresponding to the last iteration, the unbiased scoring of each interest of each user in the current iteration is calculated in a mode of subtracting the corresponding user bias score and the corresponding interest bias score, so that the calculated unbiased scoring of each interest of each user in the current iteration can be quicker and more accurate.

Further, the method as described above, after calculating unbiased scores of the interests of the users at the current iteration, further includes:

calculating a numerical value corresponding to a loss function in the current iteration according to the unbiased scores of the interests of the users in the current iteration; the judging whether the preset convergence condition is met includes: judging whether the difference value between the numerical value of the loss function in the current iteration and the numerical value of the loss function in the last iteration is smaller than a preset threshold value or not; if the difference is smaller than a preset threshold value, determining that a preset convergence condition is met; and if the difference is greater than or equal to a preset threshold value, determining that the preset convergence condition is not met.

In the embodiment of the application, the offset-free scores of the interests of the users are continuously calculated in an iterative manner by adopting a loss function, and the convergence degree can be accurately represented by the numerical values corresponding to the loss function, so that the offset-free scores of the interests of the users are continuously calculated in an iterative manner by adopting the loss function, the calculated offset-free scores of the interests after convergence can be more accurate, and the determined depth interest user point pair set is more accurate.

Further, the calculating a value corresponding to the loss function in the current iteration according to the unbiased score of each user's interest in the current iteration according to the method described above includes:

determining a depth interest point pair set of all current users and a depth user point pair set of all current interests according to the unbiased scores of all the interests of all the users during the current iteration;

and calculating a numerical value corresponding to the loss function in the current iteration according to the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests.

In the embodiment of the application, firstly, the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests are determined according to the unbiased scores of all the interests of all the users during current iteration, then, the numerical values corresponding to the loss functions during the current iteration are calculated according to the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests, the relation between the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests can be accurately represented by the numerical values corresponding to the loss functions, and the convergence degree of the unbiased scores of all the interests of all the users during the current iteration can be accurately represented by the relation between the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests.

Further, the determining a set of deep interest point pairs of all current users and a set of deep user point pairs of all current interests according to unbiased scores of the interests of the users in the current iteration according to the method described above includes:

acquiring unbiased scores corresponding to all interests of each user during current iteration and respectively sorting in a descending order; screening out interests corresponding to the pre-set percentage quantile points of all users during current iteration; determining all the users and the corresponding interest point pairs during the current iteration as a depth interest point pair set of all the users; acquiring unbiased scores corresponding to all users of each interest during current iteration and respectively sorting in a descending order; screening out users corresponding to the pre-set percentage quantile points of all the interests in the current iteration; and determining all interests and the corresponding point pairs formed by the users during the current iteration as a depth user point pair set of all the interests.

In the embodiment of the application, the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests are determined in a mode that the corresponding non-bias scores are located at the preset percentage quantile points, so that the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests can be accurately determined.

Further, the calculating, according to the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests, a value corresponding to the loss function in the current iteration includes:

calculating the current intersection of the depth interest point pair set of all the current users and the depth user point pair set of all the current interests; calculating the current quotient value of the number of the point pairs in the current intersection and the number of the point pairs in the depth interest point pair set of all the current users; and determining the difference value between the value 1 and the current quotient value as a value corresponding to the loss function in the current iteration.

In this embodiment of the present application, since the larger the current quotient value of the number of the point pairs in the current intersection and the number of the point pairs in the depth interest point pair sets of all the current users is, the closer the depth interest point pair sets of all the current users are to the depth user point pair sets of all the current interests is, and the smaller the value corresponding to the loss function in the corresponding current iteration is, when the value of the loss function is near zero, it is indicated that the depth interest point pair sets of all the current users are very close to the depth user point pair sets of all the current interests. The loss function determined in this way for the current iteration is more accurate.

Further, the method as described above, the determining the deep interest user point pair set by using the converged unbiased score of each user's interest includes:

acquiring a converged depth interest point pair set of all users or a converged depth user point pair set of all interests corresponding to the unbiased scores of the converged users for the interests; and determining the depth interest point pair set of all the converged users or the depth interest point pair set of all the converged users as the depth interest point pair set.

In the embodiment of the application, the depth interest point pair sets of all users after convergence or the depth user point pair sets of all interests after convergence are infinitely close to the same, so that the depth interest point pair sets can be accurately represented by adopting the depth interest point pair sets of all users after convergence or the depth user point pair sets of all interests after convergence as the depth interest user point pair sets.

Further, the method as described above, if it is determined that the preset convergence condition is not satisfied, further includes:

calculating the unbiased scores of the interests of the users in the next iteration; and calculating a numerical value corresponding to the loss function in the next iteration according to the unbiased scores of the interests of the users in the next iteration.

In the embodiment of the application, if the preset convergence condition is determined not to be met, iteration is continued, whether the preset convergence condition is met or not is determined by adopting a numerical value mode corresponding to a loss function, the calculated unbiased score of each interest after convergence can be more accurate, and the determined depth interest user point pair set is more accurate.

A second aspect of the embodiments of the present application provides a data screening apparatus, where the apparatus is located in an electronic device, and the apparatus includes:

the request receiving module is used for receiving a data screening request, wherein the data screening request is a deep user screening request and/or a deep interest screening request; the data screening module is used for responding to the data screening request and acquiring a corresponding depth user and/or a corresponding depth interest from a predetermined depth interest user point pair set; the depth interest user point pair set is determined according to the converged unbiased scores of the interests of the users; and the data output module is used for outputting the corresponding depth user and/or the corresponding depth interest.

Further, the apparatus as described above, further comprising;

the unbiased score calculation module is used for calculating unbiased scores of the interests of the users in the current iteration; the convergence judging module is used for judging whether a preset convergence condition is met; the unbiased score determining module is used for determining the unbiased score of each interest of each user in the current iteration as the converged unbiased score of each interest of each user if the unbiased score of each interest of each user is determined to meet the preset convergence condition; and the point pair set determining module is used for determining the deep interest user point pair set by adopting the converged unbiased scores of the interests of the users.

Further, in the apparatus as described above, the unbiased score calculation module is specifically configured to:

Further, in the apparatus as described above, when the unbiased score calculation module calculates, according to unbiased scores of the interests of the users in the last iteration, the biased scores of the users in the last iteration and the biased scores of the interests in the last iteration, the unbiased score calculation module is specifically configured to:

Further, in the apparatus described above, when the unbiased score calculation module calculates the unbiased score of each interest of each user in the current iteration according to the unbiased score of each interest of each user in the previous iteration, the user biased score corresponding to the previous iteration, and the interest biased score corresponding to the previous iteration, the unbiased score calculation module is specifically configured to:

Further, the apparatus as described above, further comprising:

the loss function numerical value calculation module is used for calculating the numerical value corresponding to the loss function in the current iteration according to the unbiased score of each user for each interest in the current iteration;

the convergence judging module is specifically configured to: judging whether the difference value between the numerical value of the loss function in the current iteration and the numerical value of the loss function in the last iteration is smaller than a preset threshold value or not; if the difference is smaller than a preset threshold value, determining that a preset convergence condition is met; and if the difference is greater than or equal to a preset threshold value, determining that the preset convergence condition is not met.

Further, in the apparatus as described above, the loss function value calculating module is specifically configured to:

determining a depth interest point pair set of all current users and a depth user point pair set of all current interests according to the unbiased scores of all the interests of all the users during the current iteration; and calculating a numerical value corresponding to the loss function in the current iteration according to the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests.

Further, in the apparatus described above, when determining the depth interest point pair sets of all current users and the depth user point pair sets of all current interests according to the unbiased scores of the interests of the users in the current iteration, the loss function numerical value calculation module is specifically configured to:

Further, in the apparatus as described above, when the loss function value calculation module calculates the value corresponding to the loss function in the current iteration according to the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests, the loss function value calculation module is specifically configured to:

Further, in the apparatus as described above, the point pair set determining module is specifically configured to:

Further, the apparatus as described above, the unbiased score calculation module, further configured to:

if the preset convergence condition is determined not to be met, calculating the unbiased score of each user on each interest during the next iteration; the loss function numerical calculation module is further configured to: and calculating a numerical value corresponding to the loss function in the next iteration according to the unbiased scores of the interests of the users in the next iteration.

A third aspect of the embodiments of the present application provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of the first aspects.

A fourth aspect of embodiments of the present application provides a non-transitory computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any one of the first aspects.

A fifth aspect of embodiments of the present application provides a computer program comprising program code for performing the method according to the first aspect when the computer program is run by a computer.

Drawings

The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:

fig. 1 is an application scenario diagram of a data screening method that can implement the embodiment of the present application;

FIG. 2 is a schematic flow chart of a data screening method according to a first embodiment of the present application;

FIG. 3 is a first schematic view of an intelligent search operation interface in a data filtering method according to a first embodiment of the present application;

FIG. 4 is a second schematic view of an intelligent search operation interface in the data filtering method according to the first embodiment of the present application;

FIG. 5 is a schematic flow chart of a data screening method according to a second embodiment of the present application;

FIG. 6 is a schematic flowchart of step 201 of a data screening method according to a second embodiment of the present application;

fig. 7 is a schematic flowchart of step 2012 in the data screening method according to the second embodiment of the present application;

fig. 8 is a schematic flowchart of step 2013 in a data screening method according to a second embodiment of the present application;

FIG. 9 is a schematic flow chart illustrating step 204 of a data screening method according to a second embodiment of the present application;

FIG. 10 is a schematic flow chart of a data screening method according to a third embodiment of the present application;

FIG. 11 is a flowchart illustrating a step 302 of a data screening method according to a third embodiment of the present application;

fig. 12 is a schematic flowchart of step 3021 in the data screening method according to the third embodiment of the present application;

FIG. 13 is a schematic structural diagram of a data screening apparatus according to a fourth embodiment of the present application;

fig. 14 is a schematic structural diagram of a data screening apparatus according to a fifth embodiment of the present application;

fig. 15 is a block diagram of an electronic device for implementing the data filtering method according to the embodiment of the present application.

Detailed Description

The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

For clear understanding of the technical solutions of the present application, the following explains the apparatuses and terms referred to in the present application:

long tail effect: the long tail effect is considered to be a complete traitor to the traditional "two-eight law". The twenty-eight law defines the mainstream and calculates the efficiency of input and output. It extends throughout life and society. "two-eight law" represents an unbalanced relationship, i.e., a few mainstream people (or things) can cause major, significant effects. The effect of those large parts of people (or things) is often neglected. While the minority accounts for about 20% and the majority accounts for about 80%. The effects of most of these people (or things) are immediate long tail effects. In the embodiment of the application, the existence of the long tail effect causes that about 80% of users are not deep users aiming at the same interest i; for the same user u, about 80% of the interests are not deep interests.

Interest: interest points are different for different application scenarios. For example, in a news-type application, the application may provide news information to the user, each news information having an information tag, which is an interest. For example, for movie-like information, the interest may be a label for each movie. Also, for example, in stock-like information, the interest may be a tag for each stock. As another example, in a shopping application, the shopping application may provide items to the user, each of which may have an informative tag that is also of interest. For example, for an appliance type of merchandise, the interest may be a label for each type of appliance. For fruit type merchandise, the interest may be a label for each fruit.

An application scenario of the data screening method provided by the embodiment of the present application is described below. The data screening method provided by the embodiment of the application can be applied to various application programs with intelligent recommendation functions, and the application program is taken as a news information application program as an example for explanation. In an application scene corresponding to the data screening method provided by the embodiment of the application, a user opens a news information application program. Each piece of information in the news information application program has an information tag, and the information tag determines the interest corresponding to each piece of information. In order to better provide news information service for users, when the users browse information, the application program can determine browsing data, comment data and other information-related data of each user for each information, and calculate the score of each user for each information according to the browsing data, comment data and other information-related data of each information. And analyzes the information scores of the same type of information tags (i.e., the same interests) to determine the raw scores of each user for each interest. After the number of browsing users of each interest reaches a certain number, or the original scores of the users for each interest reach a certain number, or the conditions of other screening deep users or screening deep interests are met, a data screening request is triggered based on the behavior of the users for the application program. And receiving a data screening request triggered by a user. The data screening request can be a deep user screening request and/or a deep interest screening request. Responding to a data screening request, and acquiring corresponding depth users and/or corresponding depth interests from a predetermined depth interest user point pair set; and determining the deep interest user point pair set according to the converged unbiased scores of the interests of the users. And outputting the corresponding deep user when the data screening request is a deep user screening request, and outputting the corresponding deep interest when the data screening request is a deep interest screening request. After outputting the corresponding deep users, pushing other deep users to one deep user so that each deep user of a certain interest can better communicate with the interest. After outputting the corresponding deep interests, the information corresponding to the deep interests can be recommended to the user.

It can be understood that the data screening method provided by the embodiment of the application can also be applied to other application scenarios. As shown in fig. 1, for example, in various applications with intelligent recommendation function, an intelligent search operation interface may also be provided to the user. In the intelligent search operation interface, a user can be provided with one or more deep interest screening users and/or keys for screening the deep interests of the users, and the users can trigger a data screening request by clicking the corresponding keys. And if the deep user screening key is clicked, triggering a deep user screening request. And if the deep interest screening key is clicked, triggering a deep interest screening request. An application program of the electronic equipment responds to a data screening request, and obtains a corresponding depth user and/or a corresponding depth interest from a predetermined depth interest user point pair set; and the depth interest user point pair set is determined according to the converged unbiased scores of the interests of the users, and corresponding depth users and/or corresponding depth interests are output. After outputting the corresponding deep users, the user can obtain the deep users corresponding to the one or more interests so as to better communicate with the deep users of the one or more interests about the one or more interests. After outputting the corresponding deep interests, the information corresponding to the deep interests can be recommended to the user. For example, in FIG. 1, the application is an information-based application. After the user clicks the deep interest screening key, outputting corresponding deep interests respectively as follows: "ax da", "i and my xx". Indicating that the two movies are the deep interests of the user. After the user clicks on the deep interest of "ax reached", two pieces of information about "ax reached" are recommended to the user.

In the embodiment of the application, the deep interest user point pair set is predetermined, each user interest point pair in the deep interest user point pair set is determined according to the converged unbiased score of each interest of each user, and the bias score related to the interest and the bias score related to the user in the score of each interest of each user can be effectively eliminated, so that the influence of the long tail effect can be effectively eliminated, and the interest of the deep user and/or the deep preference interest of the user can be accurately determined. And further, the intelligent level of the application program can be improved.

Embodiments of the present application will be described below in detail with reference to the accompanying drawings.

Example one

Fig. 2 is a schematic flow chart of a data filtering method according to a first embodiment of the present application, and as shown in fig. 2, an execution subject of the embodiment of the present application is a data filtering apparatus, and the data filtering apparatus may be integrated in an electronic device. The data screening method provided by the embodiment includes the following steps.

Step 101, receiving a data screening request, wherein the data screening request is a deep user screening request and/or a deep interest screening request.

Wherein, the deep user is a deep hobby user of interest. The deep interests are deep hobby interests of the user.

In this embodiment, as an optional implementation manner, the data screening method may be applied to various application programs having an intelligent recommendation function. The application program may have an intelligent search operation interface in which a data filtering window may be provided, and a deep user filtering window and a deep interest filtering window may also be provided in the data filtering window.

As shown in fig. 3, in the deep user filtering window, one or more interest identifications may be selected or input, and a deep user filtering request is triggered by clicking a confirmation key of the deep user filtering request, and the electronic device receives the deep user filtering request and performs filtering on deep users of the one or more interests.

As shown in fig. 4, in the deep interest filtering window, one or more user identifications may be selected or input, and a deep interest filtering request is triggered by clicking a confirmation key of the deep interest filtering request, and the electronic device receives the deep interest filtering request and performs filtering of the deep interest of the one or more users.

As shown in fig. 3 and 4, the data filtering window may further include a total confirmation key, after the user selects and inputs the corresponding identifier in the deep user filtering window and the deep interest filtering window, the total confirmation key is clicked to trigger the deep user filtering request and the deep interest filtering request, and the electronic device receives the deep user filtering request and the deep interest filtering request, and performs filtering for the corresponding deep user and filtering for the corresponding deep interest.

As another optional implementation, the user may also speak the data filtering request in a voice manner, that is, the user speaks the deep user filtering request and/or the deep interest filtering request according to the requirement, and the electronic device performs semantic analysis on the deep user filtering request and/or the deep interest filtering request through the radio receiving component to obtain the deep user filtering request and/or the deep interest filtering request.

It is understood that the data filtering request triggered by the user may be received in other manners, which is not limited in this embodiment.

Step 102, responding to a data screening request, and acquiring a corresponding depth user and/or a corresponding depth interest from a predetermined depth interest user point pair set; and determining the deep interest user point pair set according to the converged unbiased scores of the interests of the users.

In this embodiment, the raw scores of the interests of the users are stored in advance, and in the raw scores of the interests of the users, the scores are not accurate due to the long tail effect. E.g. one user u₁The natural score of all the interests is high, and the natural score is easy to become an interest i_xDeep users of, however, user u₁Is not actually of interest i_xThe depth of user. And as a user u₂Natural score of all its interests is low, original interest i_yIs user u₂Is of deep interest, but cannot be recalled.

Therefore, in this embodiment, the bias score related to the interest and the bias score related to the user are calculated according to the pre-stored original scores of the interests of the users, and after the bias scores related to the interest and the bias scores related to the users are eliminated, the non-bias scores of the interests of the users after convergence are calculated. The converged unbiased scores of the interests of the users can enable the scores of the interests of the users to be comparable. And determining a deep interest user point pair set through the converged unbiased scores of the users for the interests. In the deep interest user point pair set, one deep interest identifier corresponds to one deep interest identifier, as can be represented as (u, i). Wherein u is a deep user identifier. i is the depth interest identification.

The values indicate that one deep interest mark may correspond to a plurality of deep interest marks in the deep interest user point pair set. One deep user id may also correspond to multiple deep interest ids. Therefore, when a corresponding deep user and/or a corresponding deep interest are/is obtained from the deep interest user point pair set, a plurality of deep users of a certain interest can be obtained. The obtained depth interest of a certain user can be multiple.

Specifically, in this embodiment, the corresponding deep user and/or the corresponding deep interest may be obtained from the predetermined deep interest user point pair set according to the identification information in the data filtering request. And if the data screening request is a deep user screening request, acquiring a user identifier corresponding to the interest identifier from the deep interest user point pair set according to the interest identifier in the deep user screening request, wherein the user identifier corresponding to the interest identifier is the deep user identifier of the interest. Similarly, if the data screening request is a deep interest screening request, the interest identifier corresponding to the user identifier may be obtained from the deep interest user point pair set according to the user identifier in the deep interest screening request, and the interest identifier corresponding to the user identifier is the deep interest identifier of the user.

It is understood that the interest identifiers in the deep user filtering request can be multiple. Similarly, the number of the user identifiers in the deep interest filtering request may be multiple. And the corresponding depth users and the corresponding depth interests can be simultaneously obtained from the depth interest user point pair set.

And 103, outputting the corresponding depth user and/or the corresponding depth interest.

Specifically, in this embodiment, information of the corresponding depth user and/or the corresponding depth interest may be output on the operation interface. Such as may include an identification of corresponding depth users and/or corresponding depth interests. If the corresponding deep user is output, the contact information and the picture of the deep user can be output. If the corresponding depth interest is output, the picture and other information of the depth interest can be output.

It should be noted that, after outputting the corresponding depth user and/or the corresponding depth interest, the user may perform subsequent operations on the corresponding depth user or the corresponding depth interest through the operation interface.

For example, a deep user contact window may be provided in the operator interface to enable contact with the deep user to better communicate the relevant interests. And a recommendation window of the related information of the deep interest can be provided on the operation interface so as to recommend the related information corresponding to the deep interest to the user.

The data screening method provided by the embodiment receives a data screening request triggered by a user, wherein the data screening request is a deep user screening request and/or a deep interest screening request; responding to a data screening request, and acquiring corresponding depth users and/or corresponding depth interests from a predetermined depth interest user point pair set; the depth interest user point pair set is determined according to the non-biased scores of the converged users for the interests; outputting the corresponding depth user and/or the corresponding depth interest. Because the deep interest user point pair set is predetermined, each user interest point pair in the deep interest user point pair set is determined according to the converged unbiased scores of the interests of the users, and the bias scores related to the interests and the bias scores related to the users in the scores of the interests of the users can be effectively eliminated, the influence of long tail effect can be effectively eliminated, and the deep interest interests of the interested users and/or the deep preference interests of the users can be accurately determined. And further, the intelligent level of the application program can be improved.

Example two

Fig. 5 is a schematic flowchart of a data filtering method according to a second embodiment of the present application, and as shown in fig. 5, the data filtering method provided in this embodiment is based on the data filtering method provided in the first embodiment of the present application, and further includes a step of determining a deep interest user point pair set. The data screening method provided by this embodiment includes the following steps.

Step 201, calculating the unbiased scores of the interests of the users in the current iteration.

It will be appreciated that at the start of the iteration, the unbiased rating for each user for each interest is initialized to the raw rating for each user for each interest.

Wherein the raw scores of the interests of the users can be represented in the form of a data set,

wherein

And u is the user identification. i is an interest identification.

As an alternative, as shown in fig. 6, step 201 includes the following steps:

step 2011, unbiased scores of each user's interests at the last iteration are obtained.

Wherein the unbiased rating of each user's interest at the last iteration can be expressed as

Step 2012, calculating the bias scores of the users in the last iteration and the bias scores of the interests in the last iteration according to the unbiased scores of the interests in the users in the last iteration.

As an alternative embodiment, as shown in fig. 7, step 2012 includes the following steps:

step 2012a, obtaining the unbiased scores corresponding to all the interests of each user during the last iteration and sorting the unbiased scores in a descending order respectively.

In this embodiment, first, for each user, the unbiased scores corresponding to all the interests of the user in the last iteration are obtained. Such as for user u₁The unbiased score corresponding to all of its interests at the last iteration can be expressed as

Wherein i is 1,2,3, ….

Then, for each user, the unbiased scores corresponding to all the interests of the user in the last iteration are sorted in a descending order.

Step 2012b, determining the unbiased score of the predetermined percentage points of all the interests of each user as the biased score of each user in the last iteration.

Wherein the predetermined percentage quantile may be 10%, 30%, etc., and preferably, the predetermined percentage quantile is 20% according to the twenty-eight law.

In this embodiment, for each user, the unbiased score of all interests of the user at the last iteration, which is located at the preset percentage point, is determined, and the unbiased score is determined as the biased score of each user at the last iteration.

E.g. for user u₁With I interests, then the unbiased score at I x 20% is determined as the user u at the last iteration₁The bias score.

Wherein, the bias scores of each user at the last iteration can be expressed as

Step 2012c, obtaining the unbiased scores corresponding to all the users of each interest during the last iteration and sorting the unbiased scores in a descending order respectively.

In this embodiment, first, for each interest, the unbiased scores corresponding to all the users in the last iteration are obtained, for example, for the interest i₁The unbiased score corresponding to all its users at the last iteration can be expressed as

Wherein u is 1,2,3, ….

Then, for each interest, the unbiased scores corresponding to all the users in the last iteration are sorted in a descending order.

Step 2012d, determining the non-bias scores of the preset percentage points in all the users of interest as the bias scores of the interests in the last iteration.

In this embodiment, for each interest, a non-bias score at a preset percentage score point of all users in the previous iteration is determined, and the non-bias score is determined as each interest bias score in the previous iteration.

Wherein each interest bias score at the last iteration can be expressed as

In the embodiment, when the offset scores of each user in the last iteration and the interest offset scores in the last iteration are calculated, the non-offset scores arranged at the preset percentage points are determined as the corresponding offset scores, so that the determined offset scores can be more accurate.

Step 2013, calculating the unbiased scores of the interests of the users in the current iteration according to the unbiased scores of the interests of the users in the last iteration, the user biased scores corresponding to the last iteration and the interest biased scores corresponding to the last iteration.

As an alternative implementation, in this embodiment, as shown in fig. 8, step 2013 includes the following steps.

Step 2013a, calculating first difference values of unbiased scores of the interests of the users in the last iteration and user bias scores corresponding to the last iteration.

Step 2013b, calculating second difference values of the first difference values and the interest bias scores corresponding to the previous iteration.

Step 2013c, determining each second difference as the unbiased score of each user for each interest in the current iteration.

Further, in this embodiment, the unbiased score of each user's interest at the current iteration may be represented as

Then according to steps 2013 a-2013 c,

can be represented by formula (1).

Wherein the content of the first and second substances,

the unbiased rating of each user's interest at the last iteration.

Is the corresponding user bias score at the last iteration,

and biasing the score for the corresponding interest in the last iteration.

Is the first difference.

Is the second difference. U is 1,2,3, …, U, I is 1,2,3, …, I.

In the embodiment, when the unbiased scoring of each interest of each user in the current iteration is calculated according to the unbiased scoring of each interest of each user in the last iteration, the user bias score corresponding to the last iteration and the interest bias score corresponding to the last iteration, the unbiased scoring of each interest of each user in the current iteration is calculated by subtracting the corresponding user bias score and the corresponding interest bias score, so that the calculated unbiased scoring of each interest of each user in the current iteration can be quicker and more accurate.

In the embodiment, the unbiased scores of the interests of the users in the last iteration, the user bias scores corresponding to the user in the last iteration and the interest bias scores corresponding to the user in the last iteration are adopted to calculate the unbiased scores of the interests of the users in the current iteration, and the bias scores related to the interests and the bias scores related to the users can be effectively eliminated in each iteration, so that the accuracy of the unbiased scores of the users in the interests after convergence is further improved.

Step 202, determining whether a preset convergence condition is satisfied, if yes, executing step 203, otherwise executing step 205.

As an optional implementation manner, in this embodiment, the determining whether the preset convergence condition is satisfied may be: and judging whether the current iteration times reach a preset iteration time, if so, determining that a preset convergence condition is met, otherwise, determining that the preset convergence condition is not met.

And step 203, determining the unbiased scores of the interests of the users in the current iteration as converged unbiased scores of the interests of the users.

In this embodiment, if it is determined that the preset convergence condition is satisfied, determining the unbiased score of each interest of each user in the current iteration as the converged unbiased score of each interest of each user.

And step 204, determining a deep interest user point pair set by adopting the converged unbiased scores of the interests of the users.

In this embodiment, as shown in fig. 9, step 204 may include the following steps:

step 2041, a converged depth interest point pair set of all users corresponding to the unbiased scores of the interests of the converged users or a converged depth user point pair set of all interests of the converged users is obtained.

Wherein the converged depth interest point pair set of all users can be represented as a 1. The set of depth user point pairs of all interests after convergence may be represented as B1. Since both a1 and B1 are converged, the point pairs in a1 and B1 are approximately the same.

Step 2042, determine the converged depth interest point pair sets of all users or the converged depth user point pair sets of all interests as a depth interest user point pair set.

In the present embodiment, since the point pairs in a1 and B1 are approximately the same, the depth interest point pair set a1 of all users after convergence or the depth user point pair set B1 of all interests after convergence may be determined as the depth interest point pair set.

In this embodiment, since the depth interest point pair sets of all users after convergence or the depth user point pair sets of all interests after convergence are infinitely close to the same, the depth interest user point pair sets can be accurately represented by using the depth interest point pair sets of all users after convergence or the depth user point pair sets of all interests after convergence as the depth interest user point pair sets.

Step 205, add 1 to the number of iterations.

The iteration number can be represented as k, k in 0,1,2, …, where the current iteration is k +1, and the next iteration is k + 2. And if the preset iteration times are set, the final value of k is the preset iteration times.

After step 205 is executed, step 201 is executed continuously. Until the preset convergence condition is satisfied, the deep interest user point pair set is determined through step 204, and then step 206 is performed.

Step 206, receiving a data screening request, where the data screening request is a deep user screening request and/or a deep interest screening request.

Step 207, responding to the data screening request, and acquiring a corresponding deep user and/or a corresponding deep interest from a predetermined deep interest user point pair set; and determining the deep interest user point pair set according to the converged unbiased scores of the interests of the users.

Step 208, outputting the corresponding depth user and/or the corresponding depth interest.

In this embodiment, the implementation manners of steps 206 to 208 are similar to the implementation manners of steps 101 to 103 in the first embodiment of the present application, and are not described herein again.

In the embodiment, when the determined depth interest user point pair set is determined, the unbiased score of each interest of each user in the current iteration is determined, if the non-biased score of each interest of each user in the current iteration is determined to meet the preset convergence condition, the unbiased score of each interest of each user in the current iteration is determined to be the converged unbiased score of each interest of each user, and then the depth interest user point pair set is determined according to the converged unbiased score of each interest of each user, so that the interest user point pairs in the determined depth interest user point pair set can be the depth interest user point pairs with the influence of the long tail effect eliminated, and further the depth interest of the depth users and/or the users with the interest determined are more accurate.

EXAMPLE III

Fig. 10 is a schematic flowchart of a data filtering method according to a third embodiment of the present application, and as shown in fig. 10, the data filtering method provided in this embodiment is based on the data filtering method provided in the first embodiment of the present application, and further includes a step of determining a deep interest user point pair set. The step of determining the deep interest user point pair set is different from the step of determining the deep interest user point pair set in the second embodiment of the present application. The data screening method provided by this embodiment includes the following steps.

Step 301, calculating unbiased scores of each user's interest in the current iteration.

In this embodiment, the implementation manner of step 301 is similar to that of step 201 in the second embodiment of the present application, and is not described herein again.

And step 302, calculating a numerical value corresponding to the loss function in the current iteration according to the unbiased scores of the interests of the users in the current iteration.

As an alternative embodiment, as shown in fig. 11, step 302 includes the following steps:

step 3021, determining a depth interest point pair set of all current users and a depth user point pair set of all current interests according to the unbiased scores of the interests of the users during the current iteration.

Specifically, in the present embodiment, the set of deep interest point pairs of all the current users may be denoted as a. The set of deep pairs of user points of all current interest may be denoted as B.

Optionally, as shown in fig. 12, step 3021 includes the steps of:

and step 3021a, acquiring unbiased scores corresponding to all interests of each user during current iteration and sorting in a descending order respectively.

In this embodiment, for each user, the unbiased scores corresponding to all the interests of the user during the current iteration are obtained. Such as for user u₁All of its interests correspond to at the current iterationMay be expressed as a no bias score

Wherein i is 1,2,3, …. Then, for each user, the unbiased scores corresponding to all the interests of the user in the current iteration are sorted in a descending order.

Step 3021b, screening out interests corresponding to the pre-set percentage quantile points of all users in the current iteration.

In this embodiment, for each user, an interest corresponding to a preset percentage quantile located in front of the user in the current iteration is determined. And then, an intersection is taken for the interests corresponding to the pre-set percentage quantile points of all the users in the current iteration, and the interests corresponding to the pre-set percentage quantile points of all the users in the current iteration can be screened out.

Step 3021c, determining point pairs composed of all users and corresponding interests in the current iteration as a deep interest point pair set of all users.

For example, if the preset percentage of points is 20%, the set of deep interest point pairs of all the current users can be represented by equation (2):

and step 3021d, acquiring unbiased scores corresponding to all the users of each interest during the current iteration and sorting in a descending order.

In this embodiment, for each interest, the unbiased scores corresponding to all the users of the current iteration are obtained. For example, for interest i₁The unbiased score corresponding to all its users at the current iteration can be expressed as

Wherein u is 1,2,3, …. Then, aiming at each interest, all users of the current iteration are corresponding to each otherThe unbiased scores of (a) are sorted in descending order.

Step 3021e, screening out users corresponding to the pre-set percentage quantile points of all the interests in the current iteration.

In this embodiment, for each interest, a user corresponding to a preset percentage quantile located before the interest in the current iteration is determined. And then, taking an intersection of the users corresponding to the pre-set percentage quantile points of all the interests in the current iteration, and screening the users corresponding to the pre-set percentage quantile points of all the interests in the current iteration.

Step 3021f, determining the point pairs composed of all the interests and the corresponding users in the current iteration as a depth user point pair set of all the interests.

For example, if the preset percentage of points is 20%, the set of deep interest point pairs of all the current users can be represented by equation (3):

in this embodiment, the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests are determined in a manner that the corresponding non-bias scores are located at the pre-set percentage quantile points, so that the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests can be accurately determined.

And step 3022, calculating a value corresponding to the loss function in the current iteration according to the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests.

As an alternative embodiment, step 3022 includes the steps of:

step 3022a, calculating a current intersection of the depth interest point pair set of all the current users and the depth user point pair set of all the current interests.

Wherein the current intersection of the set of deep point of interest pairs of all users at present and the set of deep point of user pairs of all interests at present may be denoted as a ∩ B.

And step 3022b, calculating a current quotient of the number of the point pairs in the current intersection and the number of the point pairs in the depth interest point pair set of all the current users.

Wherein the number of point pairs in the current intersection may be represented as | a ∩ B |. the number of point pairs in the current set of deep interest point pairs for all users may be represented as | a |. the current quotient value may be represented as | a ∩ B |/| a |.

And step 3022c, determining the difference between the value 1 and the current quotient value as a value corresponding to the loss function in the current iteration.

Wherein, the value corresponding to the loss function in the current iteration can be represented as shown in formula (4):

cost^(k+1)＝1-|A∩B|/|A| (4)

wherein, cost^(k+1)Representing the loss function at the current iteration.

It is understood that the larger the value of | A ∩ B |/| A |, the closer the set of deep point-of-interest pairs representing all current users is to the set of deep point-of-user pairs of all current interests.

In this embodiment, since the larger the current quotient of the number of the point pairs in the current intersection and the number of the point pairs in the depth interest point pair sets of all the current users is, the closer the depth interest point pair sets of all the current users are to the depth user point pair sets of all the current interests is, the smaller the value corresponding to the loss function in the corresponding current iteration is, and therefore, when the value of the loss function is near zero, it is indicated that the depth interest point pair sets of all the current users are very close to the depth user point pair sets of all the current interests. The loss function determined in this way for the current iteration is more accurate.

Step 303, determining whether a difference between the value of the loss function in the current iteration and the value of the loss function in the last iteration is smaller than a preset threshold, if so, executing step 304, otherwise, executing step 306.

Judging whether the difference value between the numerical value of the loss function in the current iteration and the numerical value of the loss function in the last iteration is smaller than a preset threshold value or not, and judging whether the preset convergence condition is met or not by adopting the numerical value of the loss function. And if the difference between the numerical value of the loss function in the current iteration and the numerical value of the loss function in the last iteration is smaller than a preset threshold value, determining that the preset convergence condition is met, and if the difference between the numerical value of the loss function in the current iteration and the numerical value of the loss function in the last iteration is larger than or equal to the preset threshold value, determining that the preset convergence condition is not met.

The value of the preset threshold is not limited.

It can be understood that if it is determined that the predetermined convergence condition is satisfied, the value corresponding to the loss function at the current iteration is already very close to 1.

And step 304, determining that a preset convergence condition is met, and determining the unbiased score of each interest of each user in the current iteration as the converged unbiased score of each interest of each user.

And 305, determining a deep interest user point pair set by adopting the converged unbiased scores of the interests of the users.

In this embodiment, the implementation manner of step 305 is similar to that of step 204 in the second embodiment of the present application, and is not described herein again.

Step 306, add 1 to the number of iterations.

In this embodiment, if it is determined that the difference between the value of the loss function in the current iteration and the value of the loss function in the last iteration is greater than or equal to the preset threshold, it is determined that the preset convergence condition is not satisfied. The iteration is continued, and since the current iteration is k +1, the next iteration is k +2 after the iteration number is increased by 1.

It is understood that after step 306, the steps 301 to 303 are continuously executed, and after step 303 is executed, the steps 304 to 305 are selected to be executed, or the step 306 is executed until the preset convergence condition is reached, and after step 305 is executed, the step 307 is executed.

That is, after step 306, the unbiased score for each user's interest at the next iteration is calculated; and calculating a value corresponding to the loss function in the next iteration according to the unbiased score of each user for each interest in the next iteration, judging whether the difference value between the value of the loss function in the next iteration and the value of the loss function in the last iteration is smaller than a preset threshold value, if so, determining that a preset convergence condition is met, and determining the unbiased score of each user for each interest in the next iteration as the unbiased score of each user for each interest after convergence. And determining a deep interest user point pair set by adopting the converged unbiased scores of the users for the interests. Otherwise, the iteration number is continuously increased by 1.

It is understood that the implementation of steps 301-306 is similar for each iteration, and therefore, the description thereof is omitted.

Step 307, receiving a data screening request, where the data screening request is a deep user screening request and/or a deep interest screening request.

Step 308, responding to the data screening request, and acquiring a corresponding deep user and/or a corresponding deep interest from a predetermined deep interest user point pair set; and determining the deep interest user point pair set according to the converged unbiased scores of the interests of the users.

Step 309, outputting the corresponding depth user and/or the corresponding depth interest.

In this embodiment, the implementation manners of steps 307 to 309 are similar to the implementation manners of steps 101 to 103 in the first embodiment of the present application, and are not described herein again.

In the embodiment, the unbiased score of each interest of each user is continuously iteratively calculated by adopting a loss function mode, and the convergence degree can be accurately represented by the numerical value corresponding to the loss function, so that the unbiased score of each interest of each user is continuously iteratively calculated by adopting the loss function mode, the calculated unbiased score of each interest after convergence can be more accurate, and the determined depth interest user point pair set is more accurate.

Example four

Fig. 13 is a schematic structural diagram of a data filtering apparatus according to a fourth embodiment of the present application, and as shown in fig. 13, the data filtering apparatus according to this embodiment is located in an electronic device. The data filtering apparatus 1300 includes: a request receiving module 1301, a data screening module 1302 and a data output module 1303.

The request receiving module 1301 is configured to receive a data screening request, where the data screening request is a deep user screening request and/or a deep interest screening request. The data screening module 1302 is configured to, in response to a data screening request, obtain a corresponding deep user and/or a corresponding deep interest from a predetermined deep interest user point pair set; and determining the deep interest user point pair set according to the converged unbiased scores of the interests of the users. And a data output module 1303, configured to output the corresponding depth user and/or the corresponding depth interest.

The data screening apparatus provided in this embodiment may implement the technical solution of the method embodiment shown in fig. 2, and the implementation principle and technical effect thereof are similar to those of the method embodiment shown in fig. 2, and are not described in detail here.

EXAMPLE five

Fig. 14 is a schematic structural diagram of a data filtering apparatus according to a fifth embodiment of the present application, and as shown in fig. 14, the data filtering apparatus according to the present embodiment is located in an electronic device. The data screening apparatus 1400 further includes, on the basis of the data screening apparatus 1300 provided in the fourth embodiment of the present application: an unbiased score calculation module 1401, a convergence judgment module 1402, an unbiased score determination module 1403, a point pair set determination module 1404, and a loss function value calculation module 1405.

Further, the unbiased score calculation module 1401 is specifically configured to:

acquiring unbiased scores of each user on each interest during last iteration; calculating bias scores of each user in the last iteration and bias scores of each interest in the last iteration according to the unbiased scores of each user in the last iteration on each interest; and calculating the unbiased scores of the interests of the users in the current iteration according to the unbiased scores of the interests of the users in the last iteration, the user bias scores corresponding to the users in the last iteration and the interest bias scores corresponding to the users in the last iteration.

Further, when calculating the biased scores of the users in the previous iteration and the biased scores of the interests in the previous iteration according to the biased scores of the users in the previous iteration, the unbiased score calculation module 1401 is specifically configured to:

acquiring unbiased scores corresponding to all interests of each user during last iteration and respectively sorting in a descending order; determining the unbiased scores of preset percentage points in all interests of each user as the biased scores of each user in the last iteration; acquiring unbiased scores corresponding to all the users of all the interests in the last iteration and respectively sorting in a descending order; and determining the unbiased scores of the preset percentage points in all the users with the interests as the bias scores of the interests in the last iteration.

Further, the unbiased score calculation module 1401, when calculating the unbiased score of each interest of each user in the current iteration according to the unbiased score of each interest of each user in the previous iteration, the user bias score corresponding to the previous iteration, and the interest bias score corresponding to the previous iteration, is specifically configured to:

calculating each first difference value of the unbiased score of each user for each interest during the last iteration and the corresponding user bias score during the last iteration; calculating each first difference value and each second difference value of the interest bias score corresponding to the last iteration; and determining each second difference as the unbiased score of each user for each interest in the current iteration.

Further, the loss function value calculating module 1405 is configured to calculate, according to the unbiased score of each user for each interest in the current iteration, a value corresponding to the loss function in the current iteration.

Accordingly, the convergence judging module 1402 is specifically configured to:

judging whether the difference value between the numerical value of the loss function in the current iteration and the numerical value of the loss function in the last iteration is smaller than a preset threshold value or not; if the difference is smaller than a preset threshold value, determining that a preset convergence condition is met; and if the difference is greater than or equal to the preset threshold, determining that the preset convergence condition is not met.

Further, the loss function value calculating module 1405 is specifically configured to:

determining a depth interest point pair set of all current users and a depth user point pair set of all current interests according to unbiased scores of all the interests of all the users during current iteration; and calculating the value corresponding to the loss function in the current iteration according to the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests.

Further, the loss function value calculating module 1405, when determining the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests according to the unbiased scores of the interests of the users in the current iteration, is specifically configured to:

acquiring unbiased scores corresponding to all interests of each user during current iteration and respectively sorting in a descending order; screening out interests corresponding to the pre-set percentage quantile points of all users during current iteration; determining all users and corresponding interest point pairs in current iteration as a depth interest point pair set of all the users; acquiring unbiased scores corresponding to all users of each interest during current iteration and respectively sorting in a descending order; screening out users corresponding to the pre-set percentage quantile points of all the interests in the current iteration; and determining all interests and corresponding point pairs formed by the users in the current iteration as a depth user point pair set of all the interests.

Further, the loss function value calculating module 1405, when calculating a value corresponding to the loss function in the current iteration according to the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests, is specifically configured to:

calculating the current intersection of the depth interest point pair sets of all the current users and the depth user point pair sets of all the current interests; calculating the current quotient value of the number of the point pairs in the current intersection and the number of the point pairs in the depth interest point pair set of all the current users; and determining the difference value between the value 1 and the current quotient value as the value corresponding to the loss function in the current iteration.

Further, the point pair set determining module 1404 is specifically configured to:

acquiring a converged depth interest point pair set of all users or a converged depth user point pair set of all interests corresponding to the unbiased scores of the interests of the converged users; and determining the depth interest point pair set of all the converged users or the depth interest point pair set of all the converged users as a depth interest user point pair set.

Further, the unbiased score calculation module 1401 is further configured to:

if the preset convergence condition is determined not to be met, calculating the unbiased score of each user on each interest during the next iteration; a loss function value calculation module further configured to: and calculating a numerical value corresponding to the loss function in the next iteration according to the unbiased scores of the interests of the users in the next iteration.

The data screening apparatus provided in this embodiment may implement the technical solutions of the method embodiments shown in fig. 5 to 12, and the implementation principles and technical effects thereof are similar to those of the method embodiments shown in fig. 5 to 12, and are not described in detail herein.

According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.

Fig. 15 is a block diagram of an electronic device according to the data filtering method of the embodiment of the present application. Electronic devices are intended for various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.

As shown in fig. 15, the electronic apparatus includes: one or more processors 1501, memory 1502, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). Fig. 15 illustrates an example of a processor 1501.

The memory 1502 is a non-transitory computer readable storage medium provided herein. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the data filtering method provided by the present application. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the data filtering method provided herein.

The memory 1502, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the data filtering method in the embodiment of the present application (for example, the request receiving module 1301, the data filtering module 1302, and the data output module 1303 shown in fig. 13). The processor 1501 executes various functional applications of the server and data processing, i.e., implements the data filtering method in the above-described method embodiments, by running non-transitory software programs, instructions, and modules stored in the memory 1502.

The memory 1502 may include a program storage area that may store an operating system, an application program required for at least one function, and a data storage area; the storage data area may store data created according to the use of the electronic device of fig. 15, and the like. Further, the memory 1502 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 1502 may optionally include memory located remotely from the processor 1501, which may be connected to the electronic device of fig. 15 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The electronic device of fig. 15 may further include: an input device 1503 and an output device 1504. The processor 1501, the memory 1502, the input device 1503, and the output device 1504 may be connected by a bus or other means, such as the bus connection shown in fig. 15.

The input device 1503 may receive input voice, numeric, or character information and generate key signal inputs associated with user settings and function control of the electronic device of fig. 15, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, or other input devices. The output devices 1504 may include voice playback devices, display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

According to the technical scheme of the embodiment of the application, the deep interest user point pair set is determined in advance, each user interest point pair in the deep interest user point pair set is determined according to the converged unbiased score of each interest of each user, and the bias score related to the interest and the bias score related to the user in the score of each interest of each user can be effectively eliminated, so that the influence of the long tail effect can be effectively eliminated, and the deep interest interests of the interested deep user and/or the user can be accurately determined. And further, the intelligent level of the application program can be improved.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.

The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A data screening method is applied to electronic equipment, and the method comprises the following steps:

receiving a data screening request, wherein the data screening request is a deep user screening request and/or a deep interest screening request;

responding to the data screening request, and acquiring corresponding depth users and/or corresponding depth interests from a predetermined depth interest user point pair set; the depth interest user point pair set is determined according to the converged unbiased scores of the interests of the users;

outputting the corresponding depth user and/or the corresponding depth interest.

2. The method according to claim 1, wherein before the obtaining the corresponding deep user and/or the corresponding deep interest from the predetermined set of deep interest user point pairs, further comprising;

calculating the unbiased scores of the interests of the users during the current iteration;

judging whether a preset convergence condition is met;

if the preset convergence condition is met, determining the unbiased score of each interest of each user in the current iteration as the converged unbiased score of each interest of each user;

and determining the deep interest user point pair set by adopting the converged unbiased scores of the interests of the users.

3. The method of claim 2, wherein calculating unbiased scores for interests of users at the current iteration comprises:

acquiring unbiased scores of each user on each interest during last iteration;

calculating bias scores of each user in the last iteration and bias scores of each interest in the last iteration according to the unbiased scores of each user in the last iteration on each interest;

and calculating the unbiased scores of the interests of the users in the current iteration according to the unbiased scores of the interests of the users in the last iteration, the user bias scores corresponding to the last iteration and the interest bias scores corresponding to the last iteration.

4. The method of claim 3, wherein calculating the biased scores of the users at the previous iteration and the biased scores of the interests at the previous iteration according to unbiased scores of the interests of the users at the previous iteration comprises:

acquiring unbiased scores corresponding to all interests of each user during last iteration and respectively sorting in a descending order;

determining the unbiased scores of preset percentage points in all interests of each user as the biased scores of each user in the last iteration;

acquiring unbiased scores corresponding to all the users of all the interests in the last iteration and respectively sorting in a descending order;

and determining the unbiased scores of preset percentage points in all the interest users as the bias scores of the interest in the last iteration.

5. The method of claim 3, wherein the calculating the unbiased score for each interest of each user at the current iteration according to the unbiased score for each interest of each user at the previous iteration, the user biased score corresponding to the previous iteration, and the interest biased score corresponding to the previous iteration comprises:

calculating each first difference value of the unbiased score of each user for each interest during the last iteration and the user bias score corresponding to the last iteration;

calculating each first difference value and each second difference value of the interest bias score corresponding to the last iteration;

and determining the second difference values as unbiased scores of the interests of the users in the current iteration.

6. The method of claim 2, wherein after calculating unbiased scores for interests of users at the current iteration, further comprising:

calculating a numerical value corresponding to a loss function in the current iteration according to the unbiased scores of the interests of the users in the current iteration;

the judging whether the preset convergence condition is met includes:

judging whether the difference value between the numerical value of the loss function in the current iteration and the numerical value of the loss function in the last iteration is smaller than a preset threshold value or not;

if the difference is smaller than a preset threshold value, determining that a preset convergence condition is met;

and if the difference is greater than or equal to a preset threshold value, determining that the preset convergence condition is not met.

7. The method of claim 6, wherein the calculating a value corresponding to the loss function in the current iteration according to the unbiased score of each user's interest in the current iteration comprises:

8. The method of claim 7, wherein determining the set of deep point-of-interest pairs for all current users and the set of deep point-of-user pairs for all current interests according to unbiased scores for the interests of the users at the current iteration comprises:

acquiring unbiased scores corresponding to all interests of each user during current iteration and respectively sorting in a descending order;

screening out interests corresponding to the pre-set percentage quantile points of all users during current iteration;

determining all the users and the corresponding interest point pairs during the current iteration as a depth interest point pair set of all the users;

acquiring unbiased scores corresponding to all users of each interest during current iteration and respectively sorting in a descending order;

screening out users corresponding to the pre-set percentage quantile points of all the interests in the current iteration;

and determining all interests and the corresponding point pairs formed by the users during the current iteration as a depth user point pair set of all the interests.

9. The method according to claim 7, wherein the calculating a value corresponding to the loss function at the current iteration according to the depth interest point pair set of all the current users and the depth user point pair set of all the current interests includes:

calculating the current intersection of the depth interest point pair set of all the current users and the depth user point pair set of all the current interests;

calculating the current quotient value of the number of the point pairs in the current intersection and the number of the point pairs in the depth interest point pair set of all the current users;

and determining the difference value between the value 1 and the current quotient value as a value corresponding to the loss function in the current iteration.

10. The method of claim 7, wherein said determining the set of deep interest user point pairs using the converged unbiased score for each interest of each user comprises:

acquiring a converged depth interest point pair set of all users or a converged depth user point pair set of all interests corresponding to the unbiased scores of the converged users for the interests;

and determining the depth interest point pair set of all the converged users or the depth interest point pair set of all the converged users as the depth interest point pair set.

11. The method of claim 6, wherein if it is determined that the preset convergence condition is not satisfied, further comprising:

calculating the unbiased scores of the interests of the users in the next iteration;

and calculating a numerical value corresponding to the loss function in the next iteration according to the unbiased scores of the interests of the users in the next iteration.

12. An apparatus for data screening, the apparatus being located in an electronic device, the apparatus comprising:

the request receiving module is used for receiving a data screening request, wherein the data screening request is a deep user screening request and/or a deep interest screening request;

the data screening module is used for responding to the data screening request and acquiring a corresponding depth user and/or a corresponding depth interest from a predetermined depth interest user point pair set; the depth interest user point pair set is determined according to the converged unbiased scores of the interests of the users;

and the data output module is used for outputting the corresponding depth user and/or the corresponding depth interest.

13. The apparatus of claim 12, further comprising;

the unbiased score calculation module is used for calculating unbiased scores of the interests of the users in the current iteration;

the convergence judging module is used for judging whether a preset convergence condition is met;

the unbiased score determining module is used for determining the unbiased score of each interest of each user in the current iteration as the converged unbiased score of each interest of each user if the unbiased score of each interest of each user is determined to meet the preset convergence condition;

and the point pair set determining module is used for determining the deep interest user point pair set by adopting the converged unbiased scores of the interests of the users.

14. The apparatus of claim 13, wherein the unbiased score calculation module is specifically configured to:

15. The apparatus of claim 14, wherein the unbiased score calculation module, when calculating the biased scores of each user at the last iteration and the biased scores of each interest at the last iteration based on unbiased scores of each user's interest at the last iteration, is specifically configured to:

16. The apparatus according to claim 14, wherein the unbiased score calculation module, when calculating the unbiased score for each interest of each user in the current iteration according to the unbiased score for each interest of each user in the previous iteration, the user bias score corresponding to the previous iteration, and the interest bias score corresponding to the previous iteration, is specifically configured to:

17. The apparatus of claim 14, further comprising:

the convergence judging module is specifically configured to:

judging whether the difference value between the numerical value of the loss function in the current iteration and the numerical value of the loss function in the last iteration is smaller than a preset threshold value or not; if the difference is smaller than a preset threshold value, determining that a preset convergence condition is met; and if the difference is greater than or equal to a preset threshold value, determining that the preset convergence condition is not met.

18. The apparatus of claim 17, wherein the loss function numerical computation module is specifically configured to:

19. The apparatus of claim 18, wherein the loss function value calculating module, when determining the set of deep interest point pairs for all current users and the set of deep user point pairs for all current interests according to the unbiased score for each interest of each user at the current iteration, is specifically configured to:

20. The apparatus according to claim 18, wherein the loss function value calculating module, when calculating the value corresponding to the loss function in the current iteration according to the depth interest point pair set of all the current users and the depth user point pair set of all the current interests, is specifically configured to:

21. The apparatus of claim 18, wherein the point pair set determining module is specifically configured to:

22. The apparatus of claim 17, wherein the unbiased score calculation module is further configured to:

23. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

24. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-11.