CN110728370B

CN110728370B - Training sample generation method and device, server and storage medium

Info

Publication number: CN110728370B
Application number: CN201910872173.5A
Authority: CN
Inventors: 敖红波; 林涛
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2019-09-16
Filing date: 2019-09-16
Publication date: 2020-08-14
Anticipated expiration: 2039-09-16
Also published as: CN110728370A

Abstract

The present disclosure relates to a method, an apparatus, a system, a server and a storage medium for generating training samples, wherein the method may include: receiving a display event reported through an application program interface, wherein the display event is generated and reported when a recommendation list is sent to an account application, and the recommendation list comprises a plurality of recommendation objects sent to the account; in response to the received display event, acquiring account characteristic information of an account and object characteristic information of an object related to the recommendation list from an account information system, wherein the account characteristic information and the object characteristic information are stored in the account information system when the recommendation list is generated; splicing the account characteristic information and the object characteristic information with corresponding label information respectively to generate a training sample; and outputting the training sample. The embodiment of the disclosure can generate the training sample in real time to improve the real-time performance and accuracy of the recommendation model.

Description

Training sample generation method and device, server and storage medium

Technical Field

The present disclosure relates to the field of internet information processing, and in particular, to a method and an apparatus for generating training samples, a server, and a storage medium.

Background

Along with the gradual enrichment of internet contents, each large website can establish a recommendation system, so that an account can quickly and accurately find favorite contents according to information recommended by the recommendation system.

The recommendation system can train the recommendation model in a machine learning manner, wherein a training sample for training the recommendation model can be generated according to specific behaviors (such as clicks, praise, concerns and the like) of the account in the internet application and objects (such as commodities, videos, news and the like) acted by the behaviors.

In the related art, when a training sample is generated, historical behavior information of an account in a period of time needs to be collected, and the training sample used for training a recommendation model is generated according to the historical behavior information of the account. The interest and the demand of the account in the historical period can be reflected according to the training sample generated by the historical behavior information of the account, however, the interest and the demand of the account may have changed along with the passage of time, so that the current interest and the demand of the account cannot be accurately reflected by the recommendation model obtained by using the historical behavior information, and the accuracy of the recommendation model is further influenced.

Disclosure of Invention

The disclosure provides a method and a device for generating a training sample, a server and a storage medium, which are used for at least solving the problem that the accuracy of a recommendation model is influenced by training the recommendation model only according to account historical behavior information in the related art. The technical scheme of the disclosure is as follows:

according to a first aspect of embodiments of the present disclosure, a method for generating a training sample is provided, which may include:

receiving a display event reported through an application program interface, wherein the display event is generated and reported when a recommendation list is sent to an account application, and the recommendation list comprises a plurality of recommendation objects sent to the account;

in response to the received display event, acquiring account characteristic information of an account and object characteristic information of an object related to the recommendation list from an account information system, wherein the account characteristic information and the object characteristic information are stored in the account information system when the recommendation list is generated;

splicing the account characteristic information and the object characteristic information with corresponding label information respectively to generate a training sample;

and outputting the training sample.

Optionally, after receiving the presentation event reported through the application program interface, the method may further include:

receiving an operation event reported through an application program interface, wherein the operation event is generated and reported when the operation of the account on the recommendation list is received;

responding to the received operation event, and acquiring corresponding account characteristic information and object characteristic information of an operation object operated by the operation event;

and splicing the account characteristic information and the object characteristic information corresponding to the operation event with the label of the operation event, and generating and outputting a training sample of the operation event.

Optionally, the operations performed by the account on the recommendation list include a positive operation and a negative operation, wherein,

the forward operation comprises at least one of: the method comprises the following steps of performing an opening operation on any recommended object in the recommendation list, and performing a selection operation on a button on a display page of any recommended object in the recommendation list, wherein the button comprises: the button is used for establishing an association relation with an uploading account of the recommended object, the button is used for agreeing, and the button is used for forwarding the recommended object to other platforms;

the negative operation at least comprises: and sending a notification of canceling the recommendation or reducing the recommendation to the server sending the recommendation list.

Optionally, after acquiring the account feature information of the account and the object feature information of the object related to the recommendation list from the account information system, the method may further include:

caching the acquired account characteristic information and the acquired object characteristic information in a preset memory;

the acquiring of the corresponding account characteristic information and the object characteristic information of the object operated by the operation event includes:

and searching the account characteristic information and the object characteristic information corresponding to the operation event in the preset memory.

Optionally, after the step of searching the account characteristic information and the object characteristic information corresponding to the operation event in the preset memory, the method may further include:

and if the account characteristic information and the object characteristic information corresponding to the operation event do not exist in the preset memory, acquiring the account characteristic information and the object characteristic information corresponding to the operation event from the account information system.

Optionally, the step of acquiring, from an account information system, account characteristic information of an account and object characteristic information of an object related to the recommendation list in response to the received presentation event includes:

acquiring an account identifier of an account corresponding to the display object and an object identifier of an object related to the recommendation list corresponding to the display event;

acquiring account characteristic information corresponding to the account identification from the account information system;

and acquiring object characteristic information corresponding to the object identification from the account information system.

Optionally, after the step of acquiring the account feature information of the account and the object feature information of the object related to the recommendation list from the account information system in response to the received presentation event, the method may further include:

carrying out format conversion on the account identifications representing different types of accounts to obtain a public characteristic domain, wherein the account identifications in the public characteristic domain have a uniform format;

the public characteristic domain and the account characteristic information are cached in a preset memory in an associated mode;

carrying out format conversion on the object identifications representing different types of objects to obtain an object feature domain, wherein the object identifications in the object feature domain have a uniform format;

and the object characteristic domain and the object characteristic information are cached in a preset memory in an associated mode.

According to a second aspect of embodiments of the present disclosure, there is provided an apparatus for generating a training sample, the apparatus may include:

the system comprises a receiving module, a processing module and a display module, wherein the receiving module is configured to receive a display event reported through an application program interface, the display event is generated and reported when a recommendation list is sent to an account application, and the recommendation list comprises a plurality of recommendation objects sent to the account;

the characteristic obtaining module is configured to obtain account characteristic information of an account and object characteristic information of objects related to the recommendation list from an account information system in response to the received presentation event, wherein the account characteristic information and the object characteristic information are stored in the account information system when the recommendation list is generated;

the splicing module is configured to splice the account characteristic information and the object characteristic information with corresponding label information respectively to generate a training sample;

an output module configured to output the training samples.

Optionally, the receiving module is further configured to receive an operation event reported through an application program interface, where the operation event is generated and reported when receiving that the account performs an operation on the recommendation list;

the characteristic obtaining module is further configured to obtain corresponding account characteristic information and object characteristic information of an object operated by the operation event in response to the received operation event;

the splicing module is further configured to splice the account characteristic information and the object characteristic information corresponding to the operation event with the label of the operation event, and generate and output a training sample of the operation event.

Optionally, the apparatus may further include:

the cache module is configured to cache the acquired account characteristic information and the acquired object characteristic information in a preset memory;

the characteristic obtaining module is further configured to search the preset memory for account characteristic information and object characteristic information corresponding to the operation event.

Optionally, the feature obtaining module is further configured to obtain, if account feature information and object feature information corresponding to the operation event do not exist in the preset memory, the account feature information and the object feature information corresponding to the operation event from the account information system.

Optionally, the feature obtaining module includes:

the identification obtaining sub-module is configured to obtain the account identification of the account and the object identification of the object related to the recommendation list corresponding to the display event according to the display event;

the account characteristic obtaining sub-module is configured to obtain account characteristic information of the account from the account information system according to the account identification;

and the object characteristic obtaining sub-module is configured to obtain object characteristic information of the objects related to the recommendation list from the account information system according to the object identification.

Optionally, the feature obtaining module further includes:

the public characteristic domain construction submodule is configured to perform format conversion on the account identifications representing different types of accounts to obtain a public characteristic domain, and the account identifications in the public characteristic domain have a uniform format;

the account information caching submodule is configured to cache the public characteristic domain and the account characteristic information in a preset memory in an associated mode;

the object feature domain construction submodule is configured to perform format conversion on the object identifications representing different types of objects to obtain an object feature domain, and the object identifications in the object feature domain have a uniform format;

and the object information caching submodule is configured to cache the object characteristic domain and the object characteristic information in a preset memory in an associated manner.

According to a third aspect of the embodiments of the present disclosure, there is provided a training sample generation system, including: the system comprises a sample splicing server, an account information system and an application program interface;

the sample splicing server is used for receiving a display event reported through the application program interface, wherein the display event is generated and reported when a recommendation list is sent to an account application, and the recommendation list comprises a plurality of recommendation objects sent to the account;

the sample splicing server is further configured to acquire, in response to the received presentation event, account feature information of an account and object feature information of an object related to the recommendation list from the account information system, where the account feature information and the object feature information are stored in the account information system when the recommendation list is generated;

the sample splicing server is further configured to splice the account characteristic information and the object characteristic information with corresponding label information respectively to generate a training sample;

the sample splicing server is also used for outputting the training samples.

Alternatively,

the sample splicing server is further configured to receive an operation event reported through the application program interface, where the operation event is generated and reported when the operation of the account on the recommendation list is received;

the sample splicing server is further configured to respond to the received operation event, and acquire corresponding account characteristic information and object characteristic information operated by the operation event;

the sample splicing server is further configured to splice the account characteristic information and the object characteristic information corresponding to the operation event with the label of the operation event, and generate and output a training sample of the operation event.

Alternatively,

the sample splicing server is also used for caching the acquired account characteristic information and the acquired object characteristic information in a preset memory;

the sample splicing server is further configured to search the preset memory for account characteristic information and object characteristic information corresponding to the operation event.

Alternatively,

the sample splicing server is further configured to, if account characteristic information and object characteristic information corresponding to the operation event do not exist in the preset memory, acquire the account characteristic information and the object characteristic information corresponding to the operation event from the account information system.

Optionally, the sample stitching server is further configured to obtain, according to the presentation event, an account identifier of the account and an object identifier of an object related to the recommendation list corresponding to the presentation event;

the sample splicing server is further configured to acquire account feature information of the account from the account information system according to the account identifier;

the sample splicing server is further configured to acquire object feature information of the object related to the recommendation list from the account information system according to the object identification.

Alternatively,

the sample splicing server is further configured to perform format conversion on the account identifiers representing different types of accounts to obtain a common feature domain, where the account identifiers in the common feature domain have a uniform format;

the sample splicing server is further configured to cache the public characteristic domain and the account characteristic information in the preset memory in an associated manner;

the sample splicing server is further configured to perform format conversion on the object identifiers representing different types of objects to obtain an object feature domain, where the object identifiers in the object feature domain have a uniform format;

the sample splicing server is further configured to cache the object feature domain and the object feature information in the preset memory in an associated manner.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a server, including:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the training sample generation method described above.

According to a fifth aspect of embodiments of the present disclosure, there is provided a storage medium, wherein instructions of the storage medium, when executed by a processor of a server, enable the server to perform the above-mentioned training sample generation method.

The technical scheme provided by the embodiment of the disclosure at least has the following beneficial effects: when a recommendation list is generated, storing the account characteristic information and the object characteristic information according to which the recommendation list is generated in an account information system, when a display event or an operation event related to the recommendation list is received, acquiring the account characteristic information and the object characteristic information according to the display event or the operation event, splicing to obtain a training sample, and finally outputting the training sample. According to the method, after the account triggers the operation on the recommendation object, the training sample can be generated in real time, the operation of the account on the recommendation list, the account information when the recommendation list is generated and the recommendation object are fed back to the recommendation model, and therefore the accuracy of the recommendation model is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

FIG. 1 is a flow diagram illustrating a method of generating training samples in accordance with an exemplary embodiment;

FIG. 2 is a flow chart illustrating a method of generating training samples in accordance with another exemplary embodiment;

FIG. 3 is a flow chart illustrating a method of generating training samples in accordance with yet another exemplary embodiment;

FIG. 4 is a flow chart illustrating a method of generating training samples in accordance with yet another exemplary embodiment;

FIG. 5 is a block diagram illustrating an apparatus for generating training samples in accordance with an exemplary embodiment;

fig. 6 is a block diagram illustrating a system for generating training samples according to an exemplary embodiment.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

The present disclosure may be applied to the following scenarios: the recommendation system predicts the behavior of the account by using the recommendation model, recommends the content meeting the interests of the account according to the prediction result, and the accuracy of the prediction result of the recommendation model is related to the quality of the training sample.

Fig. 1 is a flowchart illustrating a method for generating training samples according to an exemplary embodiment, and as shown in fig. 1, the method for generating training samples may include the following steps:

step 101: receiving a display event reported through an application program interface, wherein the display event is generated and reported when a recommendation list is sent to an account application, and the recommendation list comprises a plurality of recommendation objects sent to the account.

In the embodiment of the disclosure, the recommendation list is generated by a recommendation model according to the account characteristic information and the object characteristic information of the account, at least one recommendation object for the account is included, optionally, the account characteristic information may include account identification, age, interest, historical click list, gender, online duration, activity, attention information, browsing information, forwarding information, praise information, saving information, collection information, comment information, posting information, and the like, the range and the category of the account characteristic information may be different for different accounts, the object characteristic information may include object categories, such as other accounts, video, images, audio, documents, etc., or time of affairs, science popularization, entertainment, education, history, literature and the like, and the interest and hobbies of the account can be accurately predicted according to the account characteristic information and the object characteristic information, so that a recommended object which is more in line with the account requirement is recommended to the account.

In the disclosed embodiment, when the generated recommendation list is displayed to a user, a display event may be reported to a recommendation server that sends the recommendation list, so as to confirm that the recommendation list corresponding to the display event starts to display an account, optionally, when the client displays the recommendation list on an interface, an event log of the client is used as a display event log, and the display event is reported to a splicing server through an application program interface, or, when the recommendation server sends the recommendation list to the client, the corresponding display event is generated according to a recommendation object in the recommendation list and an account to which the client receiving the recommendation list belongs, and is reported to the splicing server, where the splicing server and the recommendation server are distinguished according to functions implemented by the servers and may be a plurality of servers independent of each other, or by the same server.

Step 102: in response to the received display event, acquiring account characteristic information of an account and object characteristic information of an object related to the recommendation list from an account information system, wherein the account characteristic information and the object characteristic information are stored in the account information system when the recommendation list is generated.

In the embodiment of the disclosure, because the generation of the recommendation list and the presentation of the recommendation list have a certain time difference, when the recommendation list is presented, account characteristic information of an account targeted by the recommendation list and object characteristic information of a recommendation object in the recommendation list may change, such as an increase or decrease in activity, or an increase or decrease in heat of the recommendation object, and because generating a training sample requires splicing the account characteristic information, the object characteristic information, and a tag corresponding to an event, at this time, if the account characteristic information and the object characteristic information are directly obtained when the recommendation list is presented, a situation that the account characteristic information, the object characteristic information, and the tag are not matched on a time line occurs, so that the training sample cannot reflect an actual operation intention of the account, for this, in the embodiment of the disclosure, the account characteristic information and the object characteristic information according to be stored when the recommendation list is generated, when a display event is received, the account characteristic information corresponding to the recommendation list and the object characteristic information of all recommended objects in the recommendation list are obtained in response to the display event, so that the account characteristic information, the object characteristic information and the tags are matched on a timeline.

In the embodiment of the present disclosure, the account characteristic information and the object characteristic information according to which the recommendation list is generated may be stored in an account information system, the account information system may be another system independent of the recommendation server and the splicing server, or may be deployed at the recommendation server and/or the splicing server, and when the account characteristic information is stored, the account characteristic information may be stored in a classified manner according to different recommendation lists, that is, information of the recommendation list, the account characteristic information, and the object characteristic information are stored correspondingly each time the recommendation list is generated, or the account characteristic information and the object characteristic information may be stored separately, and the like.

In the embodiment of the present disclosure, the account characteristic information and the object characteristic information are obtained in the account information system in response to the display event, the account characteristic information and the object characteristic information corresponding to the recommendation list may be obtained in the account information system according to the recommendation list identifier in the display event, such as the setup time and the number of the recommendation list, or the corresponding account characteristic information and the object characteristic information may be obtained in the account information system according to the display event.

Step 103: and splicing the account characteristic information and the object characteristic information with corresponding label information respectively to generate a training sample.

In the embodiment of the disclosure, after the account characteristic information and the object characteristic information are obtained, the account characteristic information and the object characteristic information are respectively spliced with the label information, wherein the label information is used for describing the event type, and after the display event is received, the label information corresponding to the display event can be obtained and spliced, so that the corresponding training sample is obtained.

Step 104: and outputting the training sample.

In the embodiment of the disclosure, the obtained account characteristic information and the object characteristic information of the training samples are matched with the label information on the time line, so that the training samples with higher quality can be obtained, the output training samples can be used for training new recommendation models, and the recommendation models can be maintained on line through the training samples, thereby further improving the accuracy of the recommendation models.

Fig. 2 is a flowchart illustrating a method for generating a training sample according to another exemplary embodiment, where as shown in fig. 2, after receiving a presentation event reported through an application program interface on the basis of fig. 1, the method for generating a training sample may further include the following steps:

step 105: and receiving an operation event reported through an application program interface, wherein the operation event is generated and reported when the operation of the account on the recommendation list is received.

In the embodiment of the present disclosure, after receiving the presentation event, the operation event reported may be received through the application program interface, and optionally, the operation event may be an operation event log generated according to an operation of the account, where the operation time is different according to a difference in operation of the account, and the type of the operation event obtained is also different, and optionally, the operation performed by the account on the recommendation list includes a positive operation and a negative operation, where,

the forward operation comprises at least one of: the method comprises the following steps of performing an opening operation on any recommended object in the recommendation list, and performing a selection operation on a button on a display page of any recommended object in the recommendation list, wherein the button comprises: the recommendation system comprises a button for establishing an association relation with an uploading account of the recommended object, a button for agreeing, and a button for forwarding the recommended object to other platforms.

In the embodiment of the present disclosure, the operations performed by the account on the recommendation list may be classified into positive operations and negative operations, wherein, the forward operation may include operations of opening, praise, forwarding, paying attention to the recommended object in the recommended list, uploading account number, collecting and the like by the account, taking a certain video recommendation object as an example, the account sees the cover, title or brief introduction of the video recommendation object to generate interest for the video recommendation object, opens the video recommendation object to watch the content of the video recommendation object, further selects the button of the video recommendation object display page due to the quality of the content after the account watches the video recommendation object, thereby executing the operations of praise, coin-in, sharing, comment, and uploading account concerning the video recommendation object, and at this time, the operation is a forward operation because the operation indicates that the video recommendation object really meets the requirement of the account.

In the embodiment of the present disclosure, for the recommended objects in the recommendation list, an account may be interested in some recommended objects or may not be interested in some recommended objects, at this time, a negative operation of the account on the recommended objects that are not interested may be received, and optionally, a selection operation performed on a button on a display page of any recommended object in the recommendation list may be a selection operation performed on the button, where the button is used to indicate that the account is not interested in the recommended objects, or the recommendation of the recommended objects is cancelled or reduced through operations such as sliding and long-pressing, and a notification of canceling the recommendation or reducing the recommendation is sent to the recommendation service end that generates the recommendation list according to the negative operation of the account.

Step 106: and responding to the received operation event, and acquiring corresponding account characteristic information and object characteristic information of an object operated by the operation event.

In the embodiment of the disclosure, when the object feature information is obtained, only the object feature information of the operation object of the operation event, such as a recommended object complied with by an account, a recommended object shared by the account, and the like, may be obtained to indicate that the recommended object meets the preference of the user, or the object feature information of other recommended objects in the recommendation list may also be obtained at the same time to indicate that the account prefers the recommended object among a plurality of recommended objects to meet the requirements of different model training.

Step 107: and splicing the account characteristic information and the object characteristic information corresponding to the operation event with the label of the operation event, and generating and outputting a training sample of the operation event.

In the embodiment of the present disclosure, the operation event may further include operation types, such as a praise operation, a forwarding operation, an attention operation, a collection operation, a download operation, a comment operation, and the like, and different operation types may correspond to different tags, and the training samples corresponding to the operation event may be obtained and output by splicing the different tags with the account characteristic information and the object characteristic information of the corresponding operation event.

In the prior art, when an account generates a series of discontinuous time operations for a recommended object, such as click on, approval, coin insertion, comment, sharing and the like for the same video recommended object, because each operation needs to report an operation event, a sample is spliced, and under the condition that the time interval is uncertain, sample splicing omission is highly likely to occur, and the splicing rate is low.

Fig. 3 is a flowchart illustrating a method for generating training samples according to yet another exemplary embodiment, where as shown in fig. 3, on the basis of fig. 2, optionally after step 102, the method may further include:

step 1021: and caching the acquired account characteristic information and the acquired object characteristic information in a preset memory.

In the implementation of the present invention, after the account characteristic information and the object characteristic information corresponding to the display event are obtained in response to the display event, the account characteristic information and the object characteristic information may be cached in a preset memory of the splicing server, optionally, the preset memory may be a local LRU (least recently used algorithm) of the splicing server, and the LRU is a cache replacement policy commonly used in computer science, that is, the recently unused memory is replaced to make room for storing other contents, and is a temporary storage policy, so as to ensure fast storage and reading of the object characteristic information and improve the efficiency of real-time splicing of the sample.

As shown in fig. 3, optionally, step 106 includes:

step 1061: and searching the account characteristic information and the object characteristic information corresponding to the operation event in the preset memory.

In the embodiment of the invention, in the process of splicing the display events, the account characteristic information and the object characteristic information corresponding to the display events are stored in the local LRU cache of the splicing server, and at the moment, when the reported operation events are subsequently received, the corresponding account characteristic information and the corresponding object characteristic information can be directly obtained from the cache without being obtained from an account information system.

Optionally, after the step 1061, the method may further include:

step 1062: and if the account characteristic information and the object characteristic information corresponding to the operation event do not exist in the preset memory, acquiring the account characteristic information and the object characteristic information corresponding to the operation event from the account information system.

In the implementation of the invention, it is possible that the reporting display event is before when the recommendation list is displayed, and then the account operates the recommendation object in the recommendation list to report the operation event after, however, due to instability of information transmission, the later reported operation event is received by the splicing server earlier than the first reported presentation event, or, the account characteristic information and the object characteristic information corresponding to the operation event do not exist in the preset memory due to the data damage, loss and the like in the preset memory, at this time, acquiring account characteristic information and object characteristic information corresponding to the operation time from the account information system according to the operation event, optionally, all object feature information in the recommendation list corresponding to the operation event may be acquired according to the recommendation list identifier, the account identifier, or the object identifier, and caching the account characteristic information and the object characteristic information in a preset memory of the splicing server.

Fig. 4 is a flowchart illustrating a method for generating training samples according to still another exemplary embodiment, where, as shown in fig. 4, step 102 includes, on the basis of fig. 1:

step 1022: and acquiring the account identification of the account corresponding to the display event and the object identification of the object related to the recommendation list corresponding to the display event.

In the embodiment of the present disclosure, in the account identifier and the object identifier corresponding to the presentation event, the account identifier may represent Identity information of the received and presented recommendation list account, such as an account ID (Identity number), a Device Identity (Device Identity) of a client to which the account belongs, and the like, and the object identifier may represent simple information of the object, and may be an object ID, an object abbreviation, an object author ID, and the like.

Step 1023: and acquiring account characteristic information corresponding to the account identification from the account information system.

Step 1024: and acquiring object characteristic information corresponding to the object identification from the account information system.

After step 102, the method may further comprise:

step 1025: and carrying out format conversion on the account identifications representing different types of accounts to obtain a common characteristic domain, wherein the account identifications in the common characteristic domain have a uniform format.

In the implementation of the present invention, optionally, the account identifier may be multiple, for example, the account identifier includes an account ID, a client DID to which the account belongs, and the like, and since the ID uniquely identifies an identity of an independent account or an object in the application system, which is different from other accounts or objects, by using a continuously increasing integer, and the DID is an independent string number generated according to hardware specific information, the ID may also be used in the application system to distinguish different accounts, at this time, the account ID is an increasing integer, and the client DID to which the account belongs is an independent string number, which are different in format, and are inconvenient to query, in this embodiment, the account identifier may be converted into the same format, so that different account identifiers are in the same format, and are convenient to store, read, query, and the like.

In the embodiment of the present disclosure, the preset rule may be Hash (Hash) of different account identifiers, so as to convert different account identifiers into the same format, and construct a public feature domain corresponding to the account identifier.

Step 1026: and the public characteristic domain and the account characteristic information are cached in a preset memory in an associated mode.

In the embodiment of the disclosure, after the public feature domain is obtained, the public feature domain and the account feature information are cached in the preset memory in an associated manner, optionally, the public feature domain and the account feature information may be stored in the preset memory in a KV (Key-Value) manner, so that subsequent other operation events can be quickly and easily obtained, and thus, the sample is spliced in real time, optionally, the public feature domain and the account feature information may be stored in a tmpfs (memory file system) of the preset memory, the tmpfs is a memory-based file system, and has an advantage of a fast reading speed.

Step 1027: and carrying out format conversion on the object identifications representing different types of objects to obtain an object feature domain, wherein the object identifications in the object feature domain have a uniform format.

Step 1028: and the object characteristic domain and the object characteristic information are cached in a preset memory in an associated mode.

In the embodiment of the present invention, optionally, there may be multiple object identifiers, and step 1027 and step 1028 are similar to step 1025 and step 1026, and refer to the discussion of step 1025 and step 1026 specifically, which is not described herein again.

The technical scheme provided by the embodiment of the disclosure at least has the following beneficial effects: when a recommendation list is generated, storing the account characteristic information and the object characteristic information according to which the recommendation list is generated in a preset memory, when a display event or an operation event related to the recommendation list is received, acquiring the account characteristic information and the object characteristic information according to the display event or the operation event, splicing to obtain a training sample, and finally outputting the training sample. According to the method, after the account triggers the operation on the recommendation object, the training sample can be generated in real time, the operation of the account on the recommendation list, the account information when the recommendation list is generated and the recommendation object are fed back to the recommendation model, and therefore the accuracy of the recommendation model is improved.

Fig. 5 is a block diagram illustrating an apparatus for generating training samples according to an example embodiment. Referring to fig. 5, the apparatus may include a receiving module 501, a feature obtaining module 502, a splicing module 503, and an output module 504.

A receiving module 501, configured to receive a display event reported through an application program interface, where the display event is generated and reported when a recommendation list is sent to an account application, where the recommendation list includes a plurality of recommendation objects sent to the account;

a feature obtaining module 502 configured to obtain, in response to the received presentation event, account feature information of an account and object feature information of an object related to the recommendation list from an account information system, where the account feature information and the object feature information are stored in the account information system when the recommendation list is generated;

the splicing module 503 is configured to splice the account feature information and the object feature information with corresponding label information, respectively, to generate a training sample;

an output module 504 configured to output the training samples.

Optionally, the receiving module 501 is further configured to receive an operation event reported through an application program interface, where the operation event is generated and reported when receiving that the account performs an operation on the recommendation list.

Optionally, the characteristic obtaining module 502 is further configured to, in response to the received operation event, obtain corresponding account characteristic information and object characteristic information of an object operated by the operation event.

Optionally, the splicing module 503 is further configured to splice the account characteristic information and the object characteristic information corresponding to the operation event with the label of the operation event, and generate and output a training sample of the operation event.

Optionally, the apparatus may further include:

The characteristic obtaining module 502 is further configured to obtain account characteristic information and object characteristic information corresponding to the operation event from the account information system if the account characteristic information and the object characteristic information corresponding to the operation event do not exist in the preset memory.

Optionally, the feature obtaining module 502 includes:

Optionally, the feature obtaining module 502 further includes:

Fig. 6 is a block diagram illustrating a system for generating training samples according to an exemplary embodiment. Referring to fig. 6, the system may include a sample stitching server 601, an account information system 602, and an application program interface 603.

The sample stitching server 601 is configured to receive a display event reported by the application program interface 603, where the display event is generated and reported when a recommendation list is sent to an account application, where the recommendation list includes a plurality of recommendation objects sent to the account;

the sample stitching server 601 is further configured to, in response to the received presentation event, acquire account feature information of an account and object feature information of an object related to the recommendation list from the account information system 602, where the account feature information and the object feature information are stored in the account information system 602 when the recommendation list is generated;

the sample splicing server 601 is further configured to splice the account characteristic information and the object characteristic information with corresponding label information, respectively, to generate a training sample;

the sample stitching server 601 is further configured to output the training sample.

Optionally, the sample stitching server 601 is further configured to receive an operation event reported through the application program interface 603, where the operation event is generated and reported when receiving that the account performs an operation on the recommendation list;

optionally, the sample stitching server 601 is further configured to, in response to the received operation event, obtain corresponding account characteristic information and object characteristic information operated by the operation event;

optionally, the sample stitching server 601 is further configured to stitch the account characteristic information and the object characteristic information corresponding to the operation event with the label of the operation event, and generate and output a training sample of the operation event.

Optionally, the sample stitching server 601 further includes a preset memory 6011, and the sample stitching server 601 is further configured to cache the acquired account characteristic information and the acquired object characteristic information in the preset memory;

optionally, the sample stitching server 601 is further configured to search the preset memory for account characteristic information and object characteristic information corresponding to the operation event.

Optionally, the sample stitching server 601 is further configured to, if account characteristic information and object characteristic information corresponding to the operation event do not exist in the preset memory, obtain the account characteristic information and the object characteristic information corresponding to the operation event from the account information system.

Optionally, the sample stitching server 601 is further configured to obtain, according to the display event, an account identifier of the account and an object identifier of an object related to the recommendation list corresponding to the display event;

the sample stitching server 601 is further configured to obtain account feature information of the account from the account information system 602 according to the account identifier;

the sample stitching server 601 is further configured to obtain object feature information of the object related to the recommendation list from the account information system according to the object identifier.

Optionally, the sample stitching server 601 is further configured to perform format conversion on the account identifiers representing different types of accounts to obtain a common feature domain, where the account identifiers in the common feature domain have a uniform format;

the sample splicing server 601 is further configured to cache the public feature domain and the account feature information in a preset memory in an associated manner;

the sample stitching server 601 is further configured to perform format conversion on the object identifiers representing different types of objects to obtain an object feature domain, where the object identifiers in the object feature domain have a uniform format;

the sample stitching server 601 is further configured to cache the object feature domain and the object feature information in a preset memory in an associated manner.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

In an exemplary embodiment, there is also provided a server, including: a processor, a memory for storing the processor-executable instructions. Wherein the processor is configured to execute the instructions to implement the training sample generation method described above.

In an exemplary embodiment, a storage medium comprising instructions, such as a memory comprising instructions, executable by a processor to perform the above method is also provided. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The account information, the user information, the device information, and the like, referred to in the embodiments of the present specification, are acquired by authorization of a user or an account and are subjected to subsequent processing and analysis.

Embodiments of the present disclosure are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the disclosed embodiments have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the disclosure.

The method and the device for generating the training samples provided by the present disclosure are introduced in detail, and a specific example is applied in the text to explain the principle and the implementation of the present disclosure, and the description of the above embodiment is only used to help understand the method and the core idea of the present disclosure; meanwhile, for a person skilled in the art, based on the idea of the present disclosure, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present disclosure should not be construed as a limitation to the present disclosure.

Claims

1. A method for generating training samples, the method comprising:

and outputting the training sample.

2. The method of claim 1, wherein after receiving the presentation event reported through the application program interface, the method further comprises:

3. The method of claim 2, wherein the operations performed by the account on the recommendation list include a positive operation and a negative operation, wherein,

4. The method of claim 2, after obtaining the account characteristic information of the account and the object characteristic information of the objects related to the recommendation list from the account information system, the method further comprising:

5. The method according to claim 4, wherein after the step of searching the account characteristic information and the object characteristic information corresponding to the operation event in the preset memory, the method further comprises:

6. The method of claim 1, wherein the step of obtaining account characteristic information of an account and object characteristic information of objects related to the recommendation list from an account information system in response to the received presentation event comprises:

acquiring an account identifier of an account corresponding to the display event and an object identifier of an object related to the recommendation list corresponding to the display event;

7. The method of claim 1, wherein after the step of obtaining account characteristic information for an account and object characteristic information for objects for which the recommendation list relates from an account information system in response to the received presentation event, the method further comprises:

carrying out format conversion on account identifications representing different types of accounts to obtain a public characteristic domain, wherein the account identifications in the public characteristic domain have a uniform format;

carrying out format conversion on object identifications representing different types of objects to obtain an object feature domain, wherein the object identifications in the object feature domain have a uniform format;

8. An apparatus for generating training samples, the apparatus comprising:

an output module configured to output the training samples.

9. The apparatus of claim 8,

the receiving module is further configured to receive an operation event reported through an application program interface, wherein the operation event is generated and reported when the operation of the account on the recommendation list is received;

10. The apparatus of claim 9, wherein the operations performed by the account on the recommendation list include a positive operation and a negative operation, wherein,

11. The apparatus of claim 9, further comprising:

12. The apparatus of claim 11,

the feature obtaining module is further configured to obtain account feature information and object feature information corresponding to the operation event from the account information system if the account feature information and the object feature information corresponding to the operation event do not exist in the preset memory.

13. The apparatus of claim 8, wherein the feature obtaining module comprises:

14. The apparatus of claim 13, wherein the feature obtaining module further comprises:

the public characteristic domain construction submodule is configured to perform format conversion on account identifications representing different types of accounts to obtain a public characteristic domain, and the account identifications in the public characteristic domain have a uniform format;

the object feature domain construction sub-module is configured to perform format conversion on object identifications representing different types of objects to obtain an object feature domain, wherein the object identifications in the object feature domain have a uniform format;

15. A system for generating training samples, the system comprising: the system comprises a sample splicing server, an account information system and an application program interface;

the sample splicing server is also used for outputting the training samples.

16. The system of claim 15,

17. The system of claim 16, wherein the operations performed by the account on the recommendation list include a positive operation and a negative operation, wherein,

18. The system of claim 16,

19. The system of claim 18,

20. The system according to claim 15, wherein the sample stitching server is further configured to obtain an account identifier of the account according to the presentation event, and an object identifier of an object related to the recommendation list corresponding to the presentation event;

21. The system of claim 20,

the sample splicing server is further configured to cache the public characteristic domain and the account characteristic information in a preset memory in an associated manner;

22. A server, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the method of generating training samples of any one of claims 1 to 7.

23. A storage medium in which instructions, when executed by a processor of a server, enable the server to perform a method of generating training samples according to any one of claims 1 to 7.