CN111669651A

CN111669651A - Method and device for determining high-quality content, electronic equipment and storage medium

Info

Publication number: CN111669651A
Application number: CN202010463089.0A
Authority: CN
Inventors: 胡滨; 李光
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2020-05-27
Filing date: 2020-05-27
Publication date: 2020-09-15
Anticipated expiration: 2040-05-27
Also published as: CN111669651B

Abstract

The application discloses a method and a device for determining high-quality content, electronic equipment and a storage medium, which relate to the field of Internet, wherein the method comprises the following steps: the server side combines any two different contents with the same category label into a comparison set; the server side puts the comparison set to M clients in the test set, wherein M is a positive integer greater than one; the server side obtains a review result reported by at least one client side in the test set, wherein the review result is high-quality content determined from two contents in the comparison set, and the review result is generated based on a preset review strategy for the reported client side according to the viewing condition information of the user corresponding to the client side in the test set on the contents in the comparison set. By applying the scheme, the accuracy of the determination result can be improved.

Description

Method and device for determining high-quality content, electronic equipment and storage medium

Technical Field

The present application relates to computer application technologies, and in particular, to a method and an apparatus for determining high-quality content in the internet field, an electronic device, and a storage medium.

Background

In many scenarios, the quality of different contents needs to be compared to determine the quality of the contents. For example, the user a and the user B both issue a food video, and can determine the high-quality content of the food video, that is, determine which video has higher quality and is more interesting, and attract the user, so that more high-quality content is exposed to improve the user's viscosity, and the like.

Currently, the quality content is determined by means of manual review analysis. This method requires a large labor cost, and it is not accurate enough to determine the quality content based on the opinion of one audit analyst.

Disclosure of Invention

The application provides a method and a device for determining high-quality content, electronic equipment and a storage medium.

A method of premium content determination, comprising:

the server side combines any two different contents with the same category label into a comparison set;

aiming at any contrast set, the server side puts the contrast set to M clients in a test set, wherein M is a positive integer greater than one;

the server side obtains a review result reported by at least one client side in the test set; and the evaluation result is the high-quality content determined from the two contents in the comparison set, and is generated by the reporting client based on a preset evaluation strategy according to the viewing condition information of the user corresponding to the client in the test set on the contents in the comparison set.

A method of premium content determination, comprising:

the method comprises the steps that a client side obtains a comparison set released by a server side, wherein the comparison set is released to M client sides in a test set after the server side forms a comparison set by using any two different contents with the same category labels, the client sides are located in the test set, and M is a positive integer larger than one;

and when the client side meets the reporting condition, high-quality content is determined from the two contents in the comparison set according to the obtained checking condition information and based on a preset evaluation strategy, and the high-quality content is reported to the server side as an evaluation result.

A high-quality content determination device, which is applied to a server side, comprises: the system comprises a first content delivery module and a first result acquisition module;

the first content delivery module is used for combining any two different contents with the same category label into a comparison set; aiming at any contrast set, putting the contrast set to M clients in a test set, wherein M is a positive integer greater than one;

the first result obtaining module is configured to obtain a review result reported by at least one client in the test set, where the review result is a high-quality content determined from two contents in the comparison set, and is generated by the reporting client based on a predetermined review policy according to the viewing condition information of the user corresponding to the client in the test set on the contents in the comparison set.

A premium content determining apparatus, applied to a client, comprising: the system comprises a first content acquisition module and a first content processing module;

the first content acquisition module is used for acquiring a comparison set released by a server, wherein the comparison set is released to M clients in a test set after the server combines any two different contents with the same category label into the comparison set, the clients are located in the test set, and M is a positive integer greater than one;

the first content processing module is used for acquiring the viewing condition information of the corresponding user for the content in the comparison set, sending the viewing condition information to other clients in the test set, acquiring the viewing condition information sent by other clients, determining high-quality content from the two contents in the comparison set according to the acquired viewing condition information and a preset review strategy when the viewing condition information meets a report condition, and reporting the high-quality content to the server as a review result.

An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method as described above.

A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method as described above.

One embodiment in the above application has the following advantages or benefits: the client can automatically determine the high-quality content in the comparison set released by the server without manual auditing analysis, so that the labor cost is saved, the high-quality content in the comparison set can be determined by integrating the checking condition information of a plurality of users on the content in the comparison set, the high-quality content can be determined based on the opinion of an auditing and analyzing person, and the accuracy of the determination result is improved. It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:

FIG. 1 is a flow chart of a first embodiment of a premium content determination method according to the present application;

FIG. 2 is a schematic diagram of test sets respectively corresponding to different categories of content according to the present application;

FIG. 3 is a schematic representation of a first converted value and a second converted value obtained as described herein;

FIG. 4 is a flow chart of a second embodiment of a premium content determination method according to the present application;

fig. 5 is a schematic structural diagram illustrating a first embodiment 50 of the premium content determining apparatus according to the present application;

fig. 6 is a schematic structural diagram of a second embodiment 60 of the premium content determining apparatus according to the present application;

FIG. 7 is a block diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In addition, it should be understood that the term "and/or" herein is merely one type of association relationship that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.

Fig. 1 is a flowchart of a first embodiment of a method for determining premium content according to the present application. As shown in fig. 1, the following detailed implementation is included.

In 101, the server side combines any two different contents with the same category label into a contrast set.

In 102, for any contrast set, the server puts the contrast set to M clients in the test set, where M is a positive integer greater than one.

In 103, the server obtains a review result reported by at least one client in the test set, where the review result is a high-quality content determined from two contents in the comparison set, and is generated by the reporting client based on a predetermined review policy according to the viewing condition information of the user corresponding to the client in the test set on the contents in the comparison set.

Therefore, in the embodiment of the method, the client can automatically determine the high-quality content in the comparison set released by the server without manual auditing and analysis, so that the labor cost is saved, the high-quality content in the comparison set can be determined by integrating the viewing condition information of a plurality of users on the content in the comparison set, the high-quality content can be determined based on the opinion of one auditing and analyzing person, and the accuracy of the determination result is improved.

In addition, in the existing manner, the category labels of different contents are mainly generated through manual review analysis, but in this embodiment, the category labels of different contents may be generated by client annotation. The content may be any type of content such as video, articles, etc.

Specifically, the server may deliver the content to be labeled to N clients in the labeling set, where N is a positive integer greater than one, and may obtain a labeling result reported by at least one client in the labeling set, where the labeling result is a generated category label of the content to be labeled, the labeling result is generated by the client performing reporting according to an obtained candidate category information based on a predetermined labeling policy, the candidate category information is generated by determining, for any client in the labeling set, that a corresponding user has viewed the content to be labeled, and then is generated and sent to other clients in the labeling set, and is at least one category to which the content to be labeled, determined according to preference information of the corresponding user for different categories of content, is most likely to belong.

N clients can be selected in advance to form a label set, and the specific value of N can be determined according to actual needs, such as 1000. And the users corresponding to the clients in the annotation set know the preference conditions of the clients to different types of content.

For example, user a: 30% sports, 20% food, 10% car …

And a user B: 30% car, 10% food, 10% sports …

And a user C: 30% sports, 20% car, 5% gourmet …

Where sports, food, cars, etc. represent different categories, the percentages reflect the user's preference for the corresponding category content, e.g., user a has watched 100 videos, where there may be 30% of sports categories, 20% of food categories, 10% of car categories, etc.

The server side puts the content to be labeled to each client side in the labeling set, for any client side, after determining that the corresponding user views the content to be labeled, at least one category to which the content to be labeled most probably belongs can be determined according to the preference condition information of the corresponding user to the content of different categories, and how to determine the at least one category to which the content to be labeled most probably belongs is not limited, for example, the probability that the content to be labeled belongs to each category can be obtained according to a Bayesian classification model, the category corresponding to the probability with the largest value is taken as the category to which the content to be labeled most probably belongs, or the categories corresponding to the first P probabilities can be taken as the categories to which the content to be labeled most probably belongs according to the sequence from the largest probability, P is a positive integer larger than one, and the determined at least one category to which the content to be labeled most probably belongs can be taken as a candidate category, and the determined candidate category information can be sent to other clients in the annotation set, and correspondingly, the candidate category information sent by other clients can also be obtained. When any client side meets the reporting condition, the category label of the content to be labeled can be generated based on the preset labeling strategy according to the acquired candidate category information, and the content to be labeled is reported to the server side. The reporting condition is not limited, and for example, the reporting condition may refer to obtaining candidate category information of a predetermined number of clients.

Generating the category label of the content to be labeled based on the predetermined labeling policy may include: respectively counting the times of determining different categories as candidate categories, sequencing the different categories according to the sequence of the times from large to small, taking the category at the front Q position after sequencing as a category label of the content to be labeled, wherein Q is a positive integer, and the specific value can be determined according to the actual requirement.

For example, for the content to be labeled, the candidate category determined corresponding to the user a is sports, the candidate category determined corresponding to the user B is cars, and the candidate category determined corresponding to the user C is sports, the times that different categories such as sports and cars are determined as candidate categories can be respectively counted, the different categories can be further sorted according to the descending order of the times, and the category that is first after sorting is used as the category label of the content to be labeled.

By the aid of the method, automatic labeling of the category labels of the contents can be achieved by the client without manual examination, analysis and generation, so that labor cost is saved, the category labels of the contents to be labeled can be generated by integrating preference condition information and the like of a plurality of users viewing the contents to be labeled on different categories of contents, and accuracy of labeling results is improved.

After the labeling is finished, for the contents with the same category label, two sets of contents can form a comparison set, and the contents in any two comparison sets are different. For example, content a, content B, and content C are contents with the same category label, three contrast sets may be formed, that is, a contrast set formed by content a and content B, a contrast set formed by content B and content C, and a contrast set formed by content a and content C.

And aiming at each comparison set, the comparison sets can be processed in the same way, namely, the comparison sets are delivered to M clients in the test set, wherein M is a positive integer larger than one, and a review result reported by at least one client in the test set is obtained, wherein the review result is high-quality content determined from two contents in the comparison sets, and is generated for the reported client based on a preset review strategy according to the viewing condition information of a user corresponding to the client in the test set on the contents in the comparison sets.

Different test sets can be generated respectively for different categories of content. Fig. 2 is a schematic diagram of test sets corresponding to different categories of content, as shown in fig. 2, for the content of the sports category, a certain number of sports related users may be extracted from a total number of users (e.g., active users in a post bar), and the clients of these users may be used to form a sports test set corresponding to the sports category, and similarly, a gourmet test set, an automobile test set, and the like may be obtained separately, where the sports test set may be used to determine high-quality sports content, the gourmet test set may be used to determine high-quality gourmet content, and the automobile test set may be used to determine high-quality automobile content, and the like. The number of clients included in each test set may depend on actual needs, for example, 1000 clients constitute a local block chain. In addition, clients in the test set and the label set may overlap, that is, one client may belong to both the test set and the label set.

The M clients in the test set can be divided into a first group of clients and a second group of clients, and the comparison set is respectively delivered to the first group of clients and the second group of clients according to different guiding strategies. Preferably, the number of clients in the test set is even, and M/2 clients are included in the first group of clients and the second group of clients, respectively.

For any contrast set, when the server releases the contrast set to M clients in the test set, the server may release the contrast set to M/2 clients in the test set based on a first guidance policy, where the first guidance policy includes: and guiding a user to firstly check the first content in the contrast set, guiding the user to check the second content in the contrast set if the user checks the first content, and delivering the contrast set to other M/2 clients in the test set based on a second guiding strategy, wherein the second guiding strategy comprises the following steps: and guiding the user to check the second content first, and guiding the user to check the first content if the user checks the second content.

For example, for a video a (first content) and a video B (second content) in the food category, for the M/2 clients corresponding to the first guidance policy, the video a may be presented to the user first, and if the user watches the video a, the user may be guided in some way to further watch the video B, if the video a is superior, the user may possibly continue to watch the video B, and otherwise, the user may abandon the watching, similarly, for the M/2 clients corresponding to the second guidance policy, the video B may be presented to the user first, if the user watches the video B, the user may be guided in some way to further watch the video a, if the video B is superior, the user may possibly continue to watch the video a, and otherwise, the user may abandon the watching.

For any client in the test set, the checking condition information of the corresponding user for the content in the comparison set can be obtained and sent to other clients in the test set, the checking condition information sent by other clients can be obtained, and when the reporting condition is met, high-quality content can be determined from the two contents in the comparison set according to the obtained checking condition information and based on a preset evaluation strategy and is reported to the server as an evaluation result. The reporting condition is not limited, and for example, the reporting condition may refer to that the viewing condition information of a predetermined number of clients is obtained for the first guidance policy and the second guidance policy, respectively.

Preferably, the determining the high-quality content from the two contents in the comparison set based on the predetermined review strategy may include: acquiring a first conversion value corresponding to the first guiding strategy, wherein the first conversion value is the ratio of the number of users who continue to check the second content after checking the first content to the number of users who check the first content; acquiring a second conversion value corresponding to a second guiding strategy, wherein the second conversion value is the ratio of the number of users who continue to check the first content after checking the second content to the number of users who check the second content; and comparing the first conversion value with the second conversion value, if the first conversion value is greater than the second conversion value, determining that the first content is the high-quality content, and if the second conversion value is greater than the first conversion value, determining that the second content is the high-quality content.

Taking a comparative set of a food category video a (first content) and a video B (second content) as an example, as shown in fig. 3, fig. 3 is a schematic diagram of a first conversion value and a second conversion value obtained in the present application, where the first conversion value is 15%, the second conversion value is 18%, and the second conversion value is greater than the first conversion value, so that it can be determined that the video B is a good-quality content compared to the video a.

Through the guide strategy and the review strategy, the high-quality content in the comparison set can be conveniently and accurately determined.

Assuming that the food category comprises three videos, namely a video A, a video B and a video C, and finally determining the ranking of the quality degree of each video through each contrast set formed by synthesis, if the contrast set determines that the video B is the high-quality content compared with the video A and the video C is the high-quality content compared with the video B, the ranking of the quality degree of the three videos can be obtained as follows: the goodness of video C > goodness of video B > goodness of video a. The ranking can be generated by the client and then sent to the server, or the ranking can be generated by the server. Therefore, when the subsequent server side executes operations such as pushing food videos to the user, videos with the ranking closer to the front can be preferentially pushed, so that the pushing quality is improved, the user viscosity is further improved, and the like.

Fig. 4 is a flowchart of a method for determining premium content according to a second embodiment of the present application. As shown in fig. 4, the following detailed implementation is included.

In 401, a client obtains a contrast set delivered by a server, where the contrast set is delivered to M clients in a test set after the server composes any two different contents with the same category label into the contrast set, the clients are in the test set, and M is a positive integer greater than one.

In 402, the client acquires the viewing condition information of the corresponding user for the content in the comparison set, sends the viewing condition information to other clients in the test set, acquires the viewing condition information sent by other clients, and determines high-quality content from the two contents in the comparison set according to the acquired viewing condition information and based on a predetermined evaluation strategy when the viewing condition information meets the reporting condition, and reports the high-quality content as an evaluation result to the server.

The client can obtain a comparison set released by the server based on a first guiding strategy, wherein the first guiding strategy comprises the following steps: and guiding the user to firstly view the first content in the contrast set, and guiding the user to view the second content in the contrast set if the user views the first content. Or, the client obtains a comparison set released by the server based on a second guiding strategy, and the second guiding strategy comprises: and guiding the user to check the second content first, and guiding the user to check the first content if the user checks the second content. The M clients in the test set can be divided into a first group of clients and a second group of clients, the first group of clients are launched based on a first guide strategy, and the second group of clients are launched based on a second guide strategy.

Accordingly, the manner of determining the high-quality content from the two contents in the comparison set based on the predetermined review strategy may include: acquiring a first conversion value corresponding to the first guiding strategy, wherein the first conversion value is the ratio of the number of users who continue to check the second content after checking the first content to the number of users who check the first content; acquiring a second conversion value corresponding to a second guiding strategy, wherein the second conversion value is the ratio of the number of users who continue to check the first content after checking the second content to the number of users who check the second content; and comparing the first conversion value with the second conversion value, if the first conversion value is greater than the second conversion value, determining that the first content is the high-quality content, and if the second conversion value is greater than the first conversion value, determining that the second content is the high-quality content.

If the client is located in the labeling set at the same time, before the client acquires the contrast set delivered by the server, the client can also acquire the content to be labeled delivered by the server, the content to be labeled is delivered to N clients in the labeling set by the server, N is a positive integer greater than one, if the corresponding user is determined to check the content to be labeled, the client can determine at least one category to which the content to be labeled most possibly belongs according to the preference condition information of the corresponding user on the content of different categories, the determined category is used as a candidate category and is sent to other clients in the labeling set, the candidate category information sent by other clients is acquired, and when the reporting condition is met, the category label of the content to be labeled can be generated according to the acquired candidate category information and based on a preset strategy label and is reported to the server.

The manner of generating the category label of the content to be labeled based on the predetermined labeling strategy may include: and respectively counting the times of determining different categories as candidate categories, sequencing the different categories according to the sequence of the times from large to small, and taking the category at the front Q position after sequencing as a category label of the content to be labeled, wherein Q is a positive integer.

It is noted that while for simplicity of explanation, the foregoing method embodiments are described as a series of acts or combination of acts, those skilled in the art will appreciate that the present application is not limited by the order of acts, as some steps may, in accordance with the present application, occur in other orders and concurrently. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application. In addition, for parts which are not described in detail in a certain embodiment, reference may be made to the related descriptions in other embodiments.

The above is a description of method embodiments, and the embodiments of the present application are further described below by way of apparatus embodiments.

Fig. 5 is a schematic structural diagram of a first embodiment 50 of the premium content determining apparatus according to the present application. The device can be applied to a server, as shown in fig. 5, and includes: a first content delivery module 501 and a first result acquisition module 502.

A first content delivery module 501, configured to combine any two different contents with the same category label into a comparison set; and aiming at any contrast set, putting the contrast set to M clients in the test set, wherein M is a positive integer greater than one.

The first result obtaining module 502 is configured to obtain a review result reported by at least one client in the test set, where the review result is a high-quality content determined from two contents in the comparison set, and is generated by the reporting client based on a predetermined review policy according to the viewing condition information of the user corresponding to the client in the test set on the contents in the comparison set.

Preferably, the first content delivery module 501 may divide M clients into a first group of clients and a second group of clients, and deliver the comparison set to the first group of clients and the second group of clients according to different guiding policies, where M is an even number, and the first group of clients and the second group of clients include M/2 clients, respectively.

The first content delivery module 501 may deliver the comparison set to the first group of clients based on a first lead policy, the first lead policy including: and guiding the user to firstly check the first content in the contrast set, guiding the user to check the second content in the contrast set if the user checks the first content, and delivering the contrast set to a second group of clients based on a second guiding strategy, wherein the second guiding strategy comprises the following steps: and guiding the user to check the second content first, and guiding the user to check the first content if the user checks the second content.

Accordingly, the predetermined review policy may include: comparing a first conversion value corresponding to the first guide strategy with a second conversion value corresponding to the second guide strategy, wherein the first conversion value is the ratio of the number of users who continue to check the second content after checking the first content to the number of users who check the first content, and the second conversion value is the ratio of the number of users who continue to check the first content after checking the second content to the number of users who check the second content; and if the first conversion value is greater than the second conversion value, determining that the first content is the high-quality content, and if the second conversion value is greater than the first conversion value, determining that the second content is the high-quality content.

The device shown in fig. 5 may further include: a second content delivery module 503 and a second result obtaining module 504.

A second content delivery module 503, configured to deliver the content to be labeled to N clients in the labeling set, where N is a positive integer greater than one.

A second result obtaining module 504, configured to obtain a labeling result reported by at least one client in the labeling set, where the labeling result is a generated category label of the content to be labeled; the marking result is generated by the reported client side according to the acquired candidate category information and based on a preset marking strategy; the candidate category information is generated after determining that the corresponding user views the content to be annotated for any client in the annotation set, and is sent to other clients in the annotation set, and is at least one category to which the content to be annotated most probably belongs determined according to the preference condition information of the corresponding user for different categories of content.

Accordingly, the predetermined annotation policy can comprise: and respectively counting the times of determining different categories as candidate categories, sequencing the different categories according to the sequence of the times from large to small, and taking the category at the front Q position after sequencing as a category label of the content to be labeled, wherein Q is a positive integer.

Fig. 6 is a schematic structural diagram of a second embodiment 60 of the premium content determining apparatus according to the present application. The apparatus is applicable to a client, as shown in fig. 6, and includes: a first content obtaining module 601 and a first content processing module 602.

The first content obtaining module 601 is configured to obtain a comparison set delivered by a server, where the comparison set is a comparison set composed of any two different contents with the same category label and delivered to M clients in a test set by the server, the clients are located in the test set, and M is a positive integer greater than one.

The first content processing module 602 is configured to obtain viewing condition information of the corresponding user for the content in the comparison set, send the viewing condition information to other clients in the test set, and obtain viewing condition information sent by other clients.

The first content obtaining module 601 may obtain a comparison set delivered by the server based on a first guiding policy, where the first guiding policy includes: and guiding the user to firstly view the first content in the contrast set, and guiding the user to view the second content in the contrast set if the user views the first content.

Alternatively, the first content obtaining module 601 may obtain a comparison set delivered by the server based on a second guidance policy, where the second guidance policy includes: and guiding the user to check the second content first, and guiding the user to check the first content if the user checks the second content.

The M clients in the test set can be divided into a first group of clients and a second group of clients, the first group of clients are launched based on a first guide strategy, and the second group of clients are launched based on a second guide strategy.

When the reporting condition is met, the first content processing module 602 may obtain a first conversion value corresponding to the first guidance policy, where the first conversion value is a ratio of the number of users who continue to check the second content after checking the first content to the number of users who check the first content, and obtain a second conversion value corresponding to the second guidance policy, where the second conversion value is a ratio of the number of users who continue to check the first content after checking the second content to the number of users who check the second content, compare the first conversion value with the second conversion value, determine that the first content is the high-quality content if the first conversion value is greater than the second conversion value, and determine that the second content is the high-quality content if the second conversion value is greater than the first conversion value.

The device shown in fig. 6 may further include: a second content acquisition module 603 and a second content processing module 604.

A second content obtaining module 603, configured to obtain content to be labeled delivered by the server, where the content to be labeled is delivered by the server to N clients in the label set, where N is a positive integer greater than one, and the clients are located in the label set.

The second content processing module 604 is configured to, when it is determined that the corresponding user views the content to be annotated, determine at least one category to which the content to be annotated is most likely to belong according to preference information of the corresponding user for different categories of content, send the determined category as a candidate category to other clients in the annotation set, acquire candidate category information sent by the other clients, and when a reporting condition is met, generate a category tag of the content to be annotated based on a predetermined annotation policy according to the acquired candidate category information, and report the category tag to the server.

When the reporting condition is met, the second content processing module 604 may respectively count the number of times that different categories are determined as candidate categories, sort the different categories according to the order of the number of times from large to small, and use the category at the top Q after sorting as a category tag of the content to be labeled, where Q is a positive integer.

For a specific work flow of the device embodiments shown in fig. 5 and fig. 6, reference is made to the related description in the foregoing method embodiments, and details are not repeated.

In a word, by adopting the scheme of the embodiment of the application device, the client can automatically determine the high-quality content in the contrast set released by the server without manual examination and analysis, so that the labor cost is saved, the high-quality content in the contrast set can be determined by integrating the viewing condition information of a plurality of users on the content in the contrast set, the high-quality content can be determined based on the opinion of one examination and analysis personnel, and the accuracy of the determination result is improved; in addition, the client can be used for realizing automatic labeling of the category labels of the contents without manual examination, analysis and generation, so that the labor cost is saved, the preference condition information of a plurality of users viewing the contents to be labeled on different categories of contents and the like can be integrated to generate the category labels of the contents to be labeled, and the accuracy of labeling results and the like are improved.

According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.

Fig. 7 is a block diagram of an electronic device according to the method of the embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.

As shown in fig. 7, the electronic apparatus includes: one or more processors Y01, a memory Y02, and interfaces for connecting the various components, including a high speed interface and a low speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information for a graphical user interface on an external input/output device (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 7, one processor Y01 is taken as an example.

Memory Y02 is a non-transitory computer readable storage medium as provided herein. Wherein the memory stores instructions executable by at least one processor to cause the at least one processor to perform the methods provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the methods provided herein.

Memory Y02 is provided as a non-transitory computer readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the methods of the embodiments of the present application. The processor Y01 executes various functional applications of the server and data processing, i.e., implements the method in the above-described method embodiments, by executing non-transitory software programs, instructions, and modules stored in the memory Y02.

The memory Y02 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the electronic device, and the like. Additionally, the memory Y02 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory Y02 may optionally include memory located remotely from processor Y01, which may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, blockchain networks, local area networks, mobile communication networks, and combinations thereof.

The electronic device may further include: an input device Y03 and an output device Y04. The processor Y01, the memory Y02, the input device Y03, and the output device Y04 may be connected by a bus or other means, and the bus connection is exemplified in fig. 7.

The input device Y03 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, track pad, touch pad, pointer, one or more mouse buttons, track ball, joystick, or other input device. The output device Y04 may include a display device, an auxiliary lighting device, a tactile feedback device (e.g., a vibration motor), and the like. The display device may include, but is not limited to, a liquid crystal display, a light emitting diode display, and a plasma display. In some implementations, the display device can be a touch screen.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific integrated circuits, computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable logic devices) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a cathode ray tube or a liquid crystal display monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area networks, wide area networks, blockchain networks, and the internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present application can be achieved, and the present invention is not limited herein.

The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining premium content, comprising:

2. The method of claim 1,

the delivering the contrast set to M clients in the test set corresponding to the category label includes: and dividing the M clients into a first group of clients and a second group of clients, and respectively delivering the contrastive sets to the first group of clients and the second group of clients according to different guiding strategies.

3. The method of claim 2,

the delivering the contrasted sets to the first group of clients and the second group of clients according to different guiding strategies respectively comprises:

delivering the comparison set to the first group of clients based on a first boot policy, the first boot policy comprising: guiding a user to check first content in the comparison set firstly, and guiding the user to check second content in the comparison set if the user checks the first content;

delivering the comparison set to the second group of clients based on a second boot policy, the second boot policy comprising: and guiding the user to firstly view the second content, and guiding the user to view the first content if the user views the second content.

4. The method of claim 3,

the predetermined review policy includes:

comparing a first conversion value corresponding to the first guidance policy with a second conversion value corresponding to the second guidance policy, where the first conversion value is a ratio of the number of users who continue to view the second content after viewing the first content to the number of users who view the first content, and the second conversion value is a ratio of the number of users who continue to view the first content after viewing the second content to the number of users who view the second content;

and if the first conversion value is greater than the second conversion value, determining that the first content is high-quality content, and if the second conversion value is greater than the first conversion value, determining that the second content is high-quality content.

5. The method of claim 1,

before the server combines any two different contents with the same category label into a contrast set, the method further includes:

the server side puts the content to be marked to N clients in a marking set, wherein N is a positive integer greater than one;

the server side obtains a labeling result reported by at least one client side in the labeling set; the labeling result is a generated category label of the content to be labeled; the marking result is generated by the reported client based on a preset marking strategy according to the acquired candidate category information; the candidate category information is generated after determining that the corresponding user views the content to be annotated for any client in the annotation set, and is sent to other clients in the annotation set, and is at least one category to which the content to be annotated most probably belongs determined according to the preference condition information of the corresponding user for different categories of content.

6. The method of claim 5,

the predetermined annotation strategy comprises: and respectively counting the times of determining different categories as candidate categories, sequencing the different categories according to the sequence of the times from large to small, and taking the category at the front Q position after sequencing as a category label of the content to be labeled, wherein Q is a positive integer.

7. A method for determining premium content, comprising:

8. The method of claim 7,

the client acquires the contrast set released by the server and comprises the following steps:

obtaining the comparison set launched by the server based on a first guiding strategy, wherein the first guiding strategy comprises: guiding a user to check first content in the comparison set firstly, and guiding the user to check second content in the comparison set if the user checks the first content;

or, obtaining the comparison set launched by the server based on a second guiding policy, where the second guiding policy includes: guiding a user to check the second content firstly, and guiding the user to check the first content if the user checks the second content;

the M clients are divided into a first group of clients and a second group of clients, the first group of clients are launched by the contrast set based on the first guiding strategy, and the second group of clients are launched by the contrast set based on the second guiding strategy.

9. The method of claim 8,

the determining the high-quality content from the two contents in the comparison set based on the predetermined review strategy comprises the following steps:

acquiring a first conversion value corresponding to the first guiding strategy, wherein the first conversion value is the ratio of the number of users who continue to view the second content after viewing the first content to the number of users who view the first content;

acquiring a second conversion value corresponding to the second guiding strategy, wherein the second conversion value is a ratio of the number of users who continue to view the first content after viewing the second content to the number of users who view the second content;

and comparing the first conversion value with the second conversion value, if the first conversion value is greater than the second conversion value, determining that the first content is high-quality content, and if the second conversion value is greater than the first conversion value, determining that the second content is high-quality content.

10. The method of claim 7,

if the client is located in the label set at the same time, before the client obtains the contrast set delivered by the server, the method further includes:

the client acquires the content to be marked released by the server, wherein the content to be marked is released by the server to N clients in the marking set, and N is a positive integer greater than one;

if the client determines that the corresponding user views the content to be labeled, determining at least one category to which the content to be labeled most possibly belongs according to the preference condition information of the corresponding user on the content of different categories, sending the determined category as a candidate category to other clients in the labeling set, acquiring candidate category information sent by other clients, and when the reporting condition is met, generating a category label of the content to be labeled based on a preset labeling strategy according to the acquired candidate category information and reporting the category label to the server.

11. The method of claim 10,

the generating of the category label of the content to be labeled based on the predetermined labeling strategy comprises: and respectively counting the times of determining different categories as candidate categories, sequencing the different categories according to the sequence of the times from large to small, and taking the category at the front Q position after sequencing as a category label of the content to be labeled, wherein Q is a positive integer.

12. A premium content determining apparatus, wherein the apparatus is applied to a server, and comprises: the system comprises a first content delivery module and a first result acquisition module;

13. The apparatus of claim 12,

the first content delivery module is further configured to divide the M clients into a first group of clients and a second group of clients, and deliver the comparison set to the first group of clients and the second group of clients according to different guidance policies.

14. The apparatus of claim 13,

the first content delivery module delivers the comparison set to the first group of clients based on a first guiding policy, wherein the first guiding policy comprises: guiding a user to firstly view first content in the contrasted set, guiding the user to view second content in the contrasted set if the user views the first content, and delivering the contrasted set to the second group of clients based on a second guiding strategy, wherein the second guiding strategy comprises: and guiding the user to firstly view the second content, and guiding the user to view the first content if the user views the second content.

15. The apparatus of claim 14,

the predetermined review policy includes:

16. The apparatus of claim 12,

the device further comprises: a second content delivery module and a second result acquisition module;

the second content delivery module is used for delivering the content to be labeled to N clients in a labeling set, wherein N is a positive integer greater than one;

the second result obtaining module is configured to obtain a labeling result reported by at least one client in the labeling set, where the labeling result is a generated category tag of the content to be labeled; the marking result is generated by the reported client based on a preset marking strategy according to the acquired candidate category information; the candidate category information is generated after determining that the corresponding user views the content to be annotated for any client in the annotation set, and is sent to other clients in the annotation set, and is at least one category to which the content to be annotated most probably belongs determined according to the preference condition information of the corresponding user for different categories of content.

17. The apparatus of claim 16,

18. A premium content determining apparatus, applied to a client, comprising: the system comprises a first content acquisition module and a first content processing module;

19. The apparatus of claim 18,

the first content obtaining module obtains the comparison set delivered by the server based on a first guiding strategy, wherein the first guiding strategy comprises: guiding a user to check first content in the comparison set firstly, and guiding the user to check second content in the comparison set if the user checks the first content;

or, the first content obtaining module obtains the comparison set delivered by the server based on a second guiding policy, where the second guiding policy includes: guiding a user to check the second content firstly, and guiding the user to check the first content if the user checks the second content;

20. The apparatus of claim 19,

the first content processing module obtains a first conversion value corresponding to the first guidance policy, where the first conversion value is a ratio of the number of users who continue to view the second content after viewing the first content to the number of users who view the first content, and obtains a second conversion value corresponding to the second guidance policy, where the second conversion value is a ratio of the number of users who continue to view the first content after viewing the second content to the number of users who view the second content, and compares the first conversion value with the second conversion value, and if the first conversion value is greater than the second conversion value, it is determined that the first content is a high-quality content, and if the second conversion value is greater than the first conversion value, it is determined that the second content is a high-quality content.

21. The apparatus of claim 18,

the device further comprises: a second content acquisition module and a second content processing module;

the second content obtaining module is configured to obtain content to be labeled released by the server, where the content to be labeled is released by the server to N clients in the label set, N is a positive integer greater than one, and the clients are located in the label set;

the second content processing module is configured to, when it is determined that the corresponding user views the content to be annotated, determine at least one category to which the content to be annotated is most likely to belong according to preference condition information of the corresponding user for different categories of content, send the determined category as a candidate category to other clients in the annotation set, acquire candidate category information sent by the other clients, and when a reporting condition is met, generate a category tag of the content to be annotated based on a predetermined annotation policy according to the acquired candidate category information, and report the category tag to the server.

22. The apparatus of claim 21,

and the second content processing module respectively counts the times of determining different categories as candidate categories, sorts the different categories according to the sequence of the times from large to small, and takes the category at the front Q position after sorting as the category label of the content to be labeled, wherein Q is a positive integer.

23. An electronic device, comprising:

at least one processor; and

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

24. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-11.