CN114971716A

CN114971716A - Service interface quality evaluation method and device, equipment, medium and product thereof

Info

Publication number: CN114971716A
Application number: CN202210594064.3A
Authority: CN
Inventors: 葛莉
Original assignee: Guangzhou Huanju Shidai Information Technology Co Ltd
Current assignee: Guangzhou Huanju Shidai Information Technology Co Ltd
Priority date: 2022-05-27
Filing date: 2022-05-27
Publication date: 2022-08-30

Abstract

The application relates to a service interface quality evaluation method, a device, equipment, a medium and a product thereof, wherein the method comprises the following steps: acquiring an advertisement case set, wherein the advertisement case set comprises a plurality of advertisement cases generated by preset service interfaces; obtaining an advertisement characteristic vector according to the advertisement characteristic information code of each advertisement case, respectively presetting positive marking characteristics and negative marking characteristics in the advertisement characteristic vector, and correspondingly generating positive samples and negative samples of the advertisement case; respectively predicting the advertisement effectiveness indexes of the positive samples and the negative samples by adopting a preset advertisement effectiveness prediction model, wherein the model is trained to be in a convergence state in advance; and according to the advertisement performance indexes of the positive samples and the negative samples, comparing and calculating to obtain the quality indexes of the service interface. The method and the device can determine the quality index for the function corresponding to the service interface for generating the advertisement file in the e-commerce platform, realize early intervention evaluation on related functions, and facilitate timely optimization and upgrading of the e-commerce platform.

Description

Service interface quality evaluation method and device, equipment, medium and product thereof

Technical Field

The present application relates to the field of e-commerce information technology, and in particular, to a service interface quality assessment method and a corresponding apparatus, computer device, computer-readable storage medium, and computer program product.

Background

The e-commerce platform is generally provided with an advertisement delivery system, various intelligent advertisement creative functions are continuously online in the advertisement delivery system for facilitating the delivery of advertisements by merchant users, and a corresponding service interface is provided for the merchant users who want to deliver advertisements to call so as to automatically generate corresponding advertisement documentations according to basic information provided by the merchant users. Through these service interfaces, the e-commerce platform can provide bulk, rich advertising creatives to merchant users.

The internet product evaluates the effectiveness of a function, and in particular, to determine the impact of the intelligent ad creative function on the advertising service, it is necessary to compare the results of an ad using this function with those without this function. AB double-blind stochastic experiments are commonly used, however, they are not performed in all scenarios, in which case causal questions need to be answered by means of already observed data.

Specifically, for the advertisement service, in the early stage of the intelligent advertisement creative function opening, the main measurement index is the adoption rate of the file, and after the adoption rate meets the requirement, the function is rapidly popularized to all users, so that a part of users can not use the intelligent advertisement creative function, and can not leave a contrast group for performing an AB experiment.

Disclosure of Invention

The present application aims to solve the above problems and provide a service interface quality assessment method and a corresponding device, computer equipment, computer readable storage medium, computer program product,

The technical scheme is adopted to adapt to various purposes of the application as follows:

in one aspect, a method for evaluating quality of service interface is provided, which is adapted to one of the purposes of the present application, and comprises the following steps:

acquiring an advertisement case set, wherein the advertisement case set comprises a plurality of advertisement cases generated by preset service interfaces;

obtaining an advertisement characteristic vector according to the advertisement characteristic information code of each advertisement case, respectively presetting positive marking characteristics and negative marking characteristics in the advertisement characteristic vector, and correspondingly generating positive samples and negative samples of the advertisement case;

respectively predicting the advertisement effectiveness indexes of the positive samples and the negative samples by adopting a preset advertisement effectiveness prediction model, wherein the advertisement effectiveness prediction model is trained to be in a convergence state in advance;

and according to the advertisement performance indexes of the positive samples and the negative samples, comparing and calculating to obtain the quality indexes of the service interface.

Optionally, before the step of predicting the advertisement performance indicators of the positive samples and the negative samples respectively by using a preset advertisement performance prediction model, the method includes the following steps:

acquiring a training data set, wherein the training data set comprises a plurality of delivered advertisement documents and advertisement performance indexes generated after delivery;

acquiring an advertisement characteristic vector according to the advertisement characteristic information code of each advertisement case, and generating a training sample according to whether the advertisement case is generated by a preset service interface or not and correspondingly presetting positive marking characteristics or negative marking characteristics in the advertisement characteristic vector;

and performing iterative training on the advertisement effectiveness prediction model to a convergence state by adopting the training samples corresponding to the advertisement documents in the training data set, wherein the advertisement effectiveness indexes generated after the training samples are put are adopted to supervise the prediction result of the model.

Optionally, the method for obtaining the advertisement feature vector according to the advertisement feature information code of each advertisement copy includes the following steps:

calling the advertisement characteristic information of the advertisement file from a data interface provided by an advertisement delivery system;

performing data cleaning on the advertisement characteristic information to obtain characteristic data, and converting the characteristic data into corresponding characteristic values;

and coding the characteristic values corresponding to the specific configuration information into advertisement characteristic vectors according to a preset sequencing rule.

Optionally, the advertisement characteristic information includes any multiple items of the following items:

the merchant release characteristic information is used for indicating the advertisement release scale of the merchant user to which the corresponding advertisement file belongs;

the commodity configuration characteristic information is used for indicating commodities corresponding to the corresponding advertising copy;

audience configuration characteristic information for indicating audience user groups of the corresponding advertising copy;

budget settlement feature information indicating consumable funds for the corresponding advertising copy;

the bidding configuration characteristic information is used for indicating bidding constraint information of the corresponding advertisement case;

and the release configuration characteristic information is used for indicating release constraint information of the corresponding advertisement file.

Optionally, the advertisement copy set includes the advertisement copy in the training data set; or the like, or a combination thereof,

the advertisement effect index is used for indicating click through rate, collection rate, purchase adding rate, conversion rate or input-output ratio corresponding to the advertisement putting effect; or the like, or, alternatively,

the advertisement case comprises any one or more items of texts, pictures, videos and animations; or the like, or, alternatively,

the service interface is used for calling a preset document template to generate an advertisement document corresponding to given information, and the given information comprises product words and/or brand words of the target commodity.

Optionally, the step of obtaining the quality index of the service interface according to comparison calculation of advertisement performance indexes of the positive sample and the negative sample includes the following steps:

respectively calculating the mean values of the advertisement performance indexes of the positive samples and the negative samples corresponding to the advertisement file sets;

and subtracting the average value corresponding to the negative sample from the average value corresponding to the positive sample to obtain a difference value, and using the difference value as the quality index of the service interface.

Optionally, after the step of obtaining the quality index of the service interface according to the advertisement performance index comparison calculation of the positive sample and the negative sample, the method includes the following steps:

and judging whether the quality index is lower than a preset threshold value, and sending an alarm message to a preset communication interface when the quality index is lower than the preset threshold value.

On the other hand, it is an object of the present invention to provide a service interface quality evaluation apparatus, which includes a document acquisition module, a coding processing module, an index prediction module, and a comparison evaluation module, wherein: the file acquisition module is used for acquiring an advertisement file set, wherein the advertisement file set comprises a plurality of advertisement files generated by preset service interfaces; the coding processing module is used for coding the advertisement characteristic information of each advertisement case to obtain an advertisement characteristic vector of each advertisement case, presetting positive marking characteristics and negative marking characteristics in the advertisement characteristic vector respectively, and correspondingly generating positive samples and negative samples of the advertisement case; the index prediction module is used for predicting the advertisement effectiveness indexes of the positive samples and the negative samples respectively by adopting a preset advertisement effectiveness prediction model, and the advertisement effectiveness prediction model is trained to be in a convergence state in advance; and the comparison evaluation module is used for comparing and calculating the advertisement performance indexes of the positive samples and the negative samples to obtain the quality indexes of the service interfaces.

Optionally, prior to the index prediction module, the index prediction module includes: the data set acquisition module is used for acquiring a training data set, wherein the training data set comprises a plurality of released advertisement documents and advertisement effect indexes generated after the release of the advertisement documents; the sample generating module is used for obtaining an advertisement characteristic vector according to the advertisement characteristic information code of each advertisement file, and generating a training sample according to whether the advertisement file is generated by a preset service interface or not and correspondingly presetting positive marking characteristics or negative marking characteristics in the advertisement characteristic vector; and the training execution module is used for carrying out iterative training on the advertisement effectiveness prediction model to a convergence state by adopting the training samples corresponding to the advertisement documents in the training data set, wherein the advertisement effectiveness indexes generated after the training samples are put are adopted to supervise the prediction result of the model.

Optionally, the encoding processing module or the sample generating module includes: the information calling unit is used for calling the advertisement characteristic information of the advertisement file from a data interface provided by an advertisement delivery system; the data cleaning unit is used for carrying out data cleaning on the advertisement characteristic information to obtain characteristic data and converting the characteristic data into corresponding characteristic values; and the characteristic coding unit is used for coding the characteristic values corresponding to the specific configuration information into advertisement characteristic vectors according to a preset sorting rule.

Optionally, the advertisement feature information includes any multiple of the following items: the merchant release characteristic information is used for indicating the advertisement release scale of the merchant user to which the corresponding advertisement file belongs; the commodity configuration characteristic information is used for indicating commodities corresponding to the corresponding advertising copy; audience configuration characteristic information for indicating audience user groups of the corresponding advertising copy; budget settlement feature information indicating consumable funds for the corresponding advertising copy; bid configuration feature information indicating bid constraint information for a corresponding advertising copy; and the release configuration characteristic information is used for indicating release constraint information of the corresponding advertisement file.

Optionally, the advertisement copy set includes the advertisement copy in the training data set; or the advertisement effectiveness index is used for indicating click through rate, collection rate, purchase rate, conversion rate or input-output ratio corresponding to the advertisement putting effectiveness; or the advertisement file comprises any one or more items of text, pictures, videos and animations; or the service interface is used for calling a preset document template to generate an advertisement document corresponding to given information, and the given information comprises product words and/or brand words of the target commodity.

Optionally, the alignment evaluation module includes: the mean value statistical unit is used for respectively calculating the mean values of the advertisement performance indexes of the positive samples and the negative samples corresponding to the advertisement file sets; and the index comparison unit is used for subtracting the average value corresponding to the negative direction sample from the average value corresponding to the positive direction sample to obtain a difference value which is used as the quality index of the service interface.

Optionally, the comparison evaluation module further comprises: and the decision processing module is used for judging whether the quality index is lower than a preset threshold value or not, and sending an alarm message to a preset communication interface when the quality index is lower than the preset threshold value.

In yet another aspect, a computer device adapted for one of the purposes of the present application is provided, comprising a central processing unit and a memory, the central processing unit being configured to invoke execution of a computer program stored in the memory to perform the steps of the service interface quality assessment method described herein.

In a further aspect, a computer-readable storage medium is provided, which stores a computer program implemented according to the service interface quality assessment method in the form of computer-readable instructions, and when the computer program is called by a computer, executes the steps included in the method.

In yet another aspect, a computer program product is provided to adapt another object of the present application, and includes computer program/instructions, which when executed by a processor, implement the steps of the service interface quality assessment method described in any of the embodiments of the present application.

Compared with the prior art, the application has various advantages, at least comprising the following aspects:

firstly, the application applies a counter-fact inference principle, generates advertisement characteristic vectors based on advertisement characteristic information of advertisement documents generated by a service interface, correspondingly generates positive samples and negative samples by respectively presetting positive marking characteristics and negative marking characteristics of the advertisement characteristic vectors of each advertisement document so as to represent data corresponding to a fact result and a counter-fact result, predicts the positive samples and the negative samples respectively through an advertisement effectiveness prediction model to obtain corresponding advertisement effectiveness indexes, and determines corresponding quality indexes based on comparison results of the advertisement effectiveness indexes of the two types of samples so as to measure the overall quality corresponding to the service interface, thereby solving the problem of effectively measuring the advertisement document generation success performance realized by the service interface.

Secondly, the advertisement success prediction model is applied after training, and because the function of the service interface is evaluated without depending on the advertisement success index generated after the actual delivery of the advertisement file generated by the service interface in the application process, the comparison experiment can be realized only by depending on the advertisement file generated by the service interface, therefore, the early intervention can be realized during the gray scale period of the on-line of the corresponding function of the service interface, the intervention can be realized without depending on the advertisement file to generate the corresponding advertisement success data, the early evaluation of the corresponding function can be realized in advance, and the corresponding function can be ensured to be optimized and upgraded in time.

In addition, the method realizes that the counter fact inference principle is migrated to the field of E-commerce advertisement service function evaluation, and for the advertisement effectiveness prediction model adopted by the method, the method is suitable for predicting the advertisement effectiveness index aiming at the advertisement characteristic vector determined according to the counter fact inference principle, and is actually equivalent to enriching the function of the advertisement effectiveness prediction model, so that the method is suitable for evaluating the advantages and disadvantages of the related E-commerce advertisement service function, and economic scale benefits are created for an E-commerce platform.

Drawings

The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

fig. 1 is a flowchart illustrating an exemplary embodiment of a service interface quality evaluation method according to the present application.

FIG. 2 is a schematic flow chart illustrating the process of training an advertisement performance prediction model using a training data set according to the present application.

Fig. 3 is a flowchart illustrating a process of obtaining an advertisement feature vector according to advertisement feature information encoding in an embodiment of the present application.

Fig. 4 is a flowchart illustrating a service interface quality evaluation method according to another embodiment of the present application.

FIG. 5 is a schematic block diagram of a service interface quality assessment apparatus of the present application;

fig. 6 is a schematic structural diagram of a computer device used in the present application.

Detailed Description

Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present application.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.

It will be understood by those within the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

As will be appreciated by those skilled in the art, "client," "terminal," and "terminal device" as used herein include both devices that are wireless signal receivers, which are devices having only wireless signal receivers without transmit capability, and devices that are receive and transmit hardware, which have receive and transmit hardware capable of two-way communication over a two-way communication link. Such a device may include: cellular or other communication devices such as personal computers, tablets, etc. having single or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service), which may combine voice, data processing, facsimile and/or data communication capabilities; a PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, internet/intranet access, a web browser, a notepad, a calendar and/or a GPS (Global Positioning System) receiver; a conventional laptop and/or palmtop computer or other device having and/or including a radio frequency receiver. As used herein, a "client," "terminal device" can be portable, transportable, installed in a vehicle (aeronautical, maritime, and/or land-based), or situated and/or configured to operate locally and/or in a distributed fashion at any other location(s) on earth and/or in space. The "client", "terminal Device" used herein may also be a communication terminal, a web terminal, a music/video playing terminal, such as a PDA, an MID (Mobile Internet Device) and/or a Mobile phone with music/video playing function, and may also be a smart tv, a set-top box, and the like.

The hardware referred to by the names "server", "client", "service node", etc. is essentially an electronic device with the performance of a personal computer, and is a hardware device having necessary components disclosed by the von neumann principle such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, an output device, etc., a computer program is stored in the memory, and the central processing unit calls a program stored in an external memory into the internal memory to run, executes instructions in the program, and interacts with the input and output devices, thereby completing a specific function.

It should be noted that the concept of "server" as referred to in this application can be extended to the case of a server cluster. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided, and in physical space, the servers may be independent from each other but can be called through an interface, or may be integrated into one physical computer or a set of computer clusters. Those skilled in the art will appreciate this variation and should not be so limited as to restrict the implementation of the network deployment of the present application.

One or more technical features of the present application, unless expressly specified otherwise, may be deployed to a server for implementation by a client remotely invoking an online service interface provided by a capture server for access, or may be deployed directly and run on the client for access.

Unless specified in clear text, the neural network model referred to or possibly referred to in the application can be deployed in a remote server and used for remote call at a client, and can also be deployed in a client with qualified equipment capability for direct call.

Various data referred to in the present application may be stored in a server remotely or in a local terminal device unless specified in the clear text, as long as the data is suitable for being called by the technical solution of the present application.

The person skilled in the art will know this: although the various methods of the present application are described based on the same concept so as to be common to each other, they may be independently performed unless otherwise specified. In the same way, for each embodiment disclosed in the present application, the same inventive concept is proposed, and therefore, concepts expressed in the same manner and concepts expressed in terms of the same are equally understood, and even though the concepts are expressed differently, they are merely convenient and appropriately changed.

Unless expressly stated otherwise, the technical features of the embodiments disclosed in the present application may be cross-linked to form a new embodiment, so long as the combination does not depart from the spirit of the present application and can satisfy the requirements of the prior art or solve the disadvantages of the prior art. Those skilled in the art will appreciate variations therefrom.

The service interface quality evaluation method can be programmed into a computer program product and is deployed in a client or a server to run, for example, in an exemplary application scenario of the application, the service interface quality evaluation method can be deployed in a server of an e-commerce platform, so that the method can be executed by accessing an interface opened after the computer program product runs and performing human-computer interaction with a process of the computer program product through a graphical user interface.

Referring to fig. 1, in an exemplary embodiment of the method for evaluating quality of service interface of the present application, the method includes the following steps:

step S2100, obtaining an advertisement case set, wherein the advertisement case set comprises a plurality of advertisement cases generated by preset service interfaces;

preparing an advertisement case set, wherein the advertisement case set is composed of advertisement cases. In one embodiment, the advertisement documents in the advertisement document set are all advertisement documents generated by a service interface preset in the e-commerce platform, so that the quality index of the document generation function corresponding to the service interface is evaluated according to the advertisement documents generated by the service interface. In another embodiment, a small amount of advertisement documents from other sources are allowed to be doped with the advertisement documents generated by the service interface as a main body, so that in the process of evaluating the quality index, the quality index is harmonized by using the advertisement documents from other sources, and the determination of the quality index is prevented from completely depending on the advertisement documents generated by the service interface.

It is understood that the service interface is an interface provided by a document generation function which is provided by an advertisement delivery system of the e-commerce platform and is suitable for generating an advertisement document for a user, and the user can generate a corresponding advertisement document through the service interface for advertisement delivery by calling the service interface.

In one embodiment, the service interface is configured to obtain given information input by a user, where the given information may be a target product, or a product word and/or a brand word of the target product, and then, a preset document template is called by a program instruction implemented by a document generation function corresponding to the service interface, where the document template includes an original text that is not replaceable and a replaceable tag, and a corresponding replaceable tag in the document template is replaced by the product word and/or the brand word of the target product, so as to obtain an advertisement document, and the advertisement document includes the original text and the product word and/or the brand word after replacement. It can be seen that, in the present embodiment, the advertisement document is in a text form.

In another embodiment, the service interface is configured to obtain given information input by the user, the given information may be a target commodity, or a product picture and/or advertisement text of a specified target product, the advertisement text may include advertisement words and product words and/or brand words of the target product, then, the program instruction realized by the file generation function corresponding to the service interface calls a preset file template, the file template comprises carrier resources such as original pictures/original animations/original videos and indication information indicating the insertion positions of the commodity pictures and/or the advertisement texts in the carrier resources, and inserting the commodity picture and/or the advertisement text of the target commodity into the corresponding position of the carrier resource according to the indication information so as to obtain the advertisement file. It can be easily understood that the advertisement scheme can be correspondingly expressed in any form of pictures, animations, videos and the like according to different original forms of carrier resources.

Therefore, the document generation function of the service interface provided by the application is essentially an intelligent advertisement creative function, the intelligent advertisement creative service is realized in a computer program form, the service interface is opened for a user, when the user calls the service interface, the corresponding advertisement document can be generated according to given information, and the user can use the generated advertisement document to place advertisements in an advertisement placement system of a e-commerce platform.

In one embodiment, the advertisement copy set may only include index identifiers corresponding to the advertisement copies, and the index identifiers may be used to call the advertisement database to obtain corresponding advertisement feature information; in another embodiment, the advertisement characteristic information may be collected and stored in the advertisement file set in advance so as to be directly called.

The advertisement documents in the advertisement document set may be advertisement documents already delivered to an advertisement delivery system in the e-commerce platform, or advertisement documents that have been configured in the advertisement delivery system but have not been delivered.

Step S2200, according to the advertisement characteristic information code of each advertisement copy to obtain its advertisement characteristic vector, presetting positive marking characteristic and negative marking characteristic in the advertisement characteristic vector, correspondingly generating the positive sample and negative sample of the advertisement copy;

the advertisement copy sets, each advertisement copy is from the advertisement delivery system of the e-commerce platform, so that the advertisement characteristic information corresponding to the advertisement copy is stored in the advertisement database of the advertisement delivery system in a correlated manner. As mentioned above, the advertisement feature information may also be pre-set in the advertisement document set after being called from the advertisement database to facilitate direct calling.

The advertisement characteristic information of the advertisement copy may include, for example, any of the following items of specific characteristic information: the system comprises merchant release characteristic information, commodity configuration characteristic information, audience configuration characteristic information, budget settlement characteristic information, bidding configuration characteristic information and release configuration characteristic information.

The merchant placement characteristic information, which is used to indicate the advertisement placement scale of the merchant user to which the corresponding advertisement copy belongs, may be composed of a plurality of preferable characteristics, for example: the merchant user who puts the corresponding advertisement file puts in the advertisement putting system for the first time, and the time length Last of the advertisement from the current time _user The number Sum of advertisements released by the merchant user in one or more preset historical time ranges _{user_ads} And the Total amount consumed by the merchant user for advertising in one or more preset historical time ranges _{user_pay} Etc., using any of the aboveOne or more items or other similar items are added to form the merchant release characteristic information. It can be seen that these features mainly serve to indicate the size of the advertisement placement of the merchant user, and provide reference information for the advertisement feature information in terms of the advertisement expense strength of the merchant user.

The product configuration feature information, which is used to indicate the product corresponding to the corresponding advertisement copy, may be composed of a plurality of preferred features, for example: semantic feature Product corresponding to commodity title, commodity type, commodity picture and the like of the commodity corresponding to the advertisement file _i The time length Last of the commodity from the current time to the shelf is released in the E-commerce platform for the first time _product Label feature Label corresponding to the portrait Label of the commodity _product And the commodity configuration characteristic information is formed by adopting any one or more items or adding other similar items. It can be seen that these features mainly serve to indicate the corresponding goods of the advertising copy, and provide reference information for the advertising feature information in terms of the configuration information of the goods themselves.

The audience configuration characteristic information, which is used to indicate the audience user population of the corresponding advertising copy, may be composed of a plurality of preferred characteristics, such as: the gender Sex of the targeted crowd is delivered to the corresponding advertisement file in the advertisement delivery system _people Age group Age _people Region, Region _people And the audience configuration characteristic information is formed by adopting any one or more items or adding other similar items. It can be seen that these features mainly serve to indicate the audience user population corresponding to the advertising copy, and provide reference information for the advertising feature information from the population-oriented aspect of the advertising copy.

The budget settlement feature information may be configured to indicate the consumable fund of the corresponding advertisement document, and may include a plurality of preferred features, for example, the budget amount Balance remaining in the delivered advertisement group to which the corresponding advertisement document belongs until the previous day in the advertisement delivery system _budget The amount of the single-day budget set by the merchant userBudget _oneday And the budget settlement characteristic information is formed by adopting any one or more items or adding other similar items. It can be seen that these features mainly serve to indicate whether the budget funds of the merchant user to which the advertisement copy belongs are sufficient, and provide reference information for the advertisement feature information in terms of the consumable funds of the advertisement copy.

The bidding configuration characteristic information is used for indicating bidding constraint information of the corresponding advertising copy. When a merchant user puts a corresponding advertisement into an advertisement putting system, the merchant user generally configures an advertisement Bidding amount Bidding representing the unit price of the current advertisement _per Advertisement bidding Strategy Strategy adopted by current advertisement _per And the bidding configuration characteristic information is formed by adopting any one or more items or adding other similar items. It can be seen that these features mainly serve to indicate the capital expenditure intention of the merchant user to which the advertisement document belongs to the corresponding advertisement, and provide reference information for the advertisement feature information in terms of the bidding constraint information of the advertisement document.

The delivery configuration characteristic information is used for indicating delivery constraint information of the corresponding advertisement file. When a merchant user launches a corresponding advertisement in each advertisement launching system, the merchant user generally configures launching time slot advertising of the corresponding advertisement _timezone Advertisement putting optimization mode advertisement _method And the budget settlement characteristic information is formed by adopting any one item or any plurality of items or adding other similar items. It can be seen that these features mainly serve to indicate the corresponding merchant user's control intention of the placement behavior of the advertisement document, and provide reference information for the advertisement feature information in terms of placement constraint information of the advertisement document.

According to the above disclosure regarding the composition of the advertisement feature vectors of the advertisement documents, in one embodiment, the encoding can be implemented by organizing the above-exemplified full amount of feature information in a preset order to form the initial feature vectors of each advertisement document, whereby each initial feature vector forms the initial feature vector of each advertisement document

Can be expressed as:

in order to use the initial feature vector of each advertisement case for generating the positive sample and the negative sample required by the contrast group, further, based on the initial feature vector, the positive Label feature Label can be added thereto respectively _positive And negative marker signature Label _negative Constructing two advertisement characteristic vectors corresponding to each advertisement file, and using the two advertisement characteristic vectors as forward samples

And negative samples

Accordingly, the expression of the advertisement feature vector of the forward sample is as follows:

the expression of the advertisement feature vector of the negative sample is as follows:

the forward tagging feature is used to assume for the corresponding advertising copy that it was generated by the service interface even though the advertising copy was not generated by the service interface, thereby forcing tagging of the corresponding advertising copy as belonging to the data generated by the service interface.

The negative marking feature is used to assume for the corresponding advertising copy that it was not generated by the service interface even though the advertising copy was generated by the service interface, thereby forcing the marking of the corresponding advertising copy as not belonging to the data generated by the service interface.

It can be seen that the positive-sign feature and the negative-sign feature are two signs mutually exclusive in a representation sense, such sign features are added to the initial feature vector, and essentially intervention variables are added to the initial feature vector, so that samples are generated through introduction of the intervention variables, and the prediction result of the model which is predicted based on the samples is influenced. For convenience of implementation, in an embodiment, the positive label features and the negative label features may be represented by binarizing with 1 and 0, respectively, but those skilled in the art may also flexibly select different values for feature representation according to the principle that the two label features are different from each other.

The positive marking characteristic and the negative marking characteristic are respectively preset in the advertisement characteristic vectors of the advertisement documents to form a positive sample and a negative sample, each advertisement document correspondingly obtains two advertisement characteristic vectors corresponding to different comparison groups, and the two advertisement characteristic vectors are subjected to the same operation to obtain different results.

Step S2300, adopting a preset advertisement effectiveness prediction model to respectively predict the advertisement effectiveness indexes of the positive sample and the negative sample, wherein the advertisement effectiveness prediction model is trained to a convergence state in advance;

in order to realize the prediction of the advertisement performance indexes based on the positive samples and the negative samples, any advertisement performance prediction model suitable for predicting the corresponding advertisement performance indexes according to the samples is prepared.

For example, the advertisement performance prediction model can be implemented based on a classical machine learning model, and can also be implemented by adopting a deep learning model. The classical machine learning model can be a model modeled by any algorithm such as Lasso Regression, XGboost, LightGBM and the like, and the deep learning model can be a feedforward neural network, a convolution neural network, deep FM and the like. The advertisement prediction model can be trained in advance to a convergence state by a person skilled in the art according to the principle disclosed in the present application by using a sufficient amount of training samples, so that the person learns to predict the corresponding advertisement performance index according to any positive sample or negative sample.

The advertisement performance index can be used for indicating click through rate, collection rate, purchase rate, conversion rate or input-output ratio corresponding to the advertisement delivery performance.

The Click Through Rate CTR (Click-Through-Rate) is a general term for internet advertisements, and refers to a Click arrival Rate of a web advertisement (picture advertisement/text advertisement/keyword advertisement/ranking advertisement/video advertisement, etc.), that is, an actual number of clicks of the advertisement (strictly speaking, the number of hits to a target page) is divided by a display amount of the advertisement (Show content).

The collection rate refers to the ratio of the total quantity of the collected commodity pages which are obtained by jumping after the network advertisements are clicked to the total quantity of the number of clicked users, namely the number of visitors.

The purchase adding rate is similar to the collection rate and refers to the ratio of the total amount of commodities added to the shopping cart in the commodity page which is jumped to after the network advertisement is clicked to the total amount of the number of clicked users, namely the number of visitors.

The conversion rate cvr (conversion rate) is a rate of conversion by a netizen who enters a promotion site by clicking a web advertisement, and generally reflects a direct profit of the advertisement. The evaluation criteria of the Chinese network marketing (advertisement) effect on the Chinese Internet association network marketing work committee member, which was originally called 6-18 th month in 2009, provides that the statistical period usually has hours, days, weeks, months and the like, and can be set as required, and the objects to be counted include various advertisement forms such as flash advertisements, picture advertisements, character chain advertisements, soft texts, mail advertisements, video advertisements, rich media advertisements and the like. CVR (conversion/click count) 100%.

The Return On Advertising Speed (ROAS) is a marketing index for measuring the effectiveness of network advertisement delivery. ROAS is total revenue/cost of ad placement.

It is understood that any one of the above advertisement performance indicators can be obtained by directly calculating according to the calculation principle of the advertisement performance indicator according to the corresponding data generated after the advertisement to which each advertisement file belongs in the advertisement delivery system is delivered. In the stage of training the advertisement effectiveness prediction model, corresponding training samples and the training samples are adopted to train the advertisement effectiveness prediction model, and the advertisement effectiveness index supervision model of the advertisement corresponding to the training samples is adopted to train, so that the advertisement effectiveness prediction model can acquire the capability of predicting the advertisement effectiveness index corresponding to the sample according to the corresponding sample.

Therefore, the positive samples and the negative samples of each advertisement in the advertisement file set can be input into the prepared advertisement effectiveness prediction model one by one for prediction, and the advertisement effectiveness indexes predicted by the model can be obtained.

Through the process, the samples corresponding to the advertising copy of the whole advertising copy set comprise two groups, wherein the first group is formed by forward samples, each forward sample is intervened by a preset forward mark characteristic, and the corresponding advertising copy is supposed to be generated by the service interface and belongs to a complete intervention group; the second group is composed of negative-going samples, wherein each negative-going sample is intervened by a preset negative-going flag feature, assuming that the corresponding advertising copy was not generated by the service interface and belongs to an uninhibited group. And in the complete intervention group and the non-intervention group, each sample can obtain the corresponding advertisement performance index through the advertisement performance prediction model.

And step S2400, comparing and calculating the advertisement performance indexes of the positive samples and the negative samples to obtain the quality indexes of the service interfaces.

And the advertisement performance indexes of the samples in the complete intervention group and the samples in the non-intervention group are utilized to realize a comparison experiment, and an evaluation parameter is determined through comparison implementation and is used as a quality index for measuring the quality of the service interface.

In one embodiment, the comparison experiment may be performed to determine the quality index by the following specific steps, including:

step S2410, respectively calculating the average values of the advertisement performance indexes of the positive samples and the negative samples corresponding to the advertisement file sets;

for the complete intervention group and the non-intervention group, the mean value of the advertisement performance indicators of all the samples in each group can be calculated respectively, that is, the mean value of the advertisement performance indicators of all the positive samples in the complete intervention group is obtained as a first mean value, and the mean value of the advertisement performance indicators of all the negative samples in the non-intervention group is obtained as a second mean value. It will be understood that each mean value correspondingly reflects the overall benefit that the advertising copy in the advertising copy set may obtain once delivered, both in the case of intervention of the service interface and in the case of no intervention.

Step S2420, subtracting the average value corresponding to the negative sample from the average value corresponding to the positive sample to obtain a difference value, and using the difference value as the quality index of the service interface.

In order to realize comparison, a first mean value corresponding to the positive sample is subtracted from a second mean value corresponding to the negative sample to obtain a difference value, the difference value can be used as a quality index for evaluating the quality of the service interface, and the quality index also has the function of measuring the quality of the intelligent advertisement creative function of the service interface by reflecting the overall benefit difference of the service interface under two conditions of intervention and non-intervention on the advertisement scheme.

In other embodiments, the calculation process may be modified as appropriate, for example, the positive and negative samples of each advertisement document are used to calculate the difference between the advertisement performance indicators, and then the differences corresponding to all advertisement documents are averaged, and the average is used as the quality indicator.

From the above embodiments, it can be seen that the present application has various advantages, including at least:

Secondly, the advertisement success prediction model is applied after being trained, and because the function of the service interface is evaluated without depending on the advertisement success indexes generated after the advertisement documents generated by the service interface are actually put in the application process, the comparison experiment can be realized only by depending on the advertisement documents generated by the service interface, so that the early intervention during the gray scale period before a large number of advertisement success indexes are not generated after the corresponding functions of the service interface are on line, the intervention is not performed after the corresponding advertisement documents generate the corresponding advertisement success data, the early evaluation on the corresponding functions can be realized in advance, and the corresponding functions can be ensured to be optimized and upgraded in time.

On the basis of any of the above embodiments, referring to fig. 2, before the step S2300 of predicting the advertisement performance indicators of the positive samples and the negative samples respectively by using a preset advertisement performance prediction model, the method includes the following steps:

step S1100, acquiring a training data set, wherein the training data set comprises a plurality of released advertising documents and advertising effectiveness indexes generated after the release of the advertising documents;

a training data set used for training the advertisement effectiveness prediction model of the application is prepared, the advertisement documents delivered in the advertisement delivery system are included in the training data set, and the advertisement effectiveness indexes actually generated after the advertisement documents are delivered in the advertisement delivery system are obtained, so that the mapping relation between each advertisement document and the corresponding advertisement effectiveness index is established in the training data set.

The advertising copy in the training data set, some of which are generated by the intelligent advertising creative service providing the service interface, and another of which may not be generated by the service interface, such as user-defined, may be added with a source tag in the training data set for each advertising copy to characterize whether the corresponding advertising copy is generated via the service interface, and the source tag may be represented as 1 or 0 in a binarized form, respectively, so that the source tag may be subsequently used directly as the positive tag feature or the negative tag feature.

Similarly, according to the setting of the advertisement performance index predicted by the advertisement performance prediction model, the advertisement performance index associated with the advertisement file in the training data set is consistent with the type of the advertisement performance index expected to be predicted by the model, and can be any one of click through rate, collection rate, purchase rate, conversion rate and input-output ratio.

In one embodiment, the advertisement copy in the training data set may be used as the advertisement copy in the advertisement copy set of the present application after the training of the advertisement performance prediction model is performed.

During the on-line gray scale of the intelligent advertising creative service, the number of the generated and delivered advertising documents is relatively limited, and the number of the advertisements which are generated by the advertising delivery system and have corresponding advertising performance indexes is also relatively limited, however, because the application applies the counterfactual inference principle to train the advertising performance prediction model, the dependence on the training samples required by model training is greatly reduced, and even if the training is carried out on the advertising performance prediction model by adopting the advertisement feature vectors corresponding to the limited advertising documents during the gray scale, the advertising performance prediction model is easier to train to a convergence state. Of course, the specific sample size can be flexibly grasped by those skilled in the art in the actual training process.

Step S1200, obtaining advertisement characteristic vectors according to the advertisement characteristic information codes of each advertisement case, and generating training samples according to whether the advertisement case is generated by a preset service interface or not and corresponding preset positive marking characteristics or negative marking characteristics in the advertisement characteristic vectors;

according to the principle disclosed in step S2200 in the foregoing of the present application, advertisement feature information of an advertisement corresponding to each advertisement copy in the training data set may be obtained from an advertisement delivery system, and encoded according to each feature data in the advertisement feature information to generate a corresponding initial feature vector, and then a positive-direction labeled feature or a negative-direction labeled feature is added on the basis of the initial feature vector to construct a corresponding advertisement feature vector, so as to obtain a corresponding positive-direction sample or negative-direction sample, which is used as a training sample in a training phase.

It should be noted that, for the advertisement feature vectors, the training phase of the advertisement performance prediction model is different from the foregoing reasoning phase, in that the labeled features preset in the two advertisement feature vectors of each advertisement document in the training phase, that is, the positive labeled features and the negative labeled features, do not need to apply a counter-fact assumption, but label the positive labeled features or the negative labeled features according to the fact whether the corresponding advertisement document is generated by the service interface of the present application. Thus, in the training phase, only one training sample is generated for each advertising copy in the training data set, and the training samples are represented as positive samples or negative samples according to the fact whether the advertising copy is generated by the service interface.

It is understood that, for the case that the source mark of each advertisement case has been marked in the training data set, the source mark of each advertisement case is directly adopted as the corresponding mark feature and is preset in the advertisement feature vector corresponding to the positive sample and the negative sample. That is, when the source tag of an advertisement document is characterized as "1", the corresponding positive sample can be obtained by presetting the positive tag feature "1" to the corresponding position of the initial feature vector, and when the source tag of an advertisement document is characterized as "0", the corresponding negative sample can be obtained by presetting the negative tag feature "0" to the corresponding position of the initial feature vector.

Through the above processes, training samples of the advertisement documents in the training data set can be obtained, and can be used for implementing iterative training of the advertisement outcome prediction model.

Step 1300, carrying out iterative training on the advertisement effectiveness prediction model to a convergence state by adopting the training samples corresponding to the advertisement documents in the training data set, wherein the advertisement effectiveness indexes generated after the training samples are put are adopted to supervise the prediction result of the model.

When the training samples corresponding to the advertisement documents in the training data set are used for training the advertisement effectiveness prediction, a supervised training mode is adopted for implementation, wherein each training sample, namely a positive sample or a negative sample corresponding to the advertisement documents, is used as the input of the advertisement effectiveness prediction model, the input content is a corresponding advertisement characteristic vector, and after semantic reasoning is carried out on the advertisement characteristic vector through the model, a corresponding advertisement effectiveness index is predicted to serve as a prediction result. And the advertisement effect index generated after the advertisement file is put is used as a supervision label of the advertisement effect prediction model to supervise the prediction result of the training sample.

In each iteration, the loss value of the model can be obtained by comparing the difference value between the supervision label and the prediction result output by the model prediction, then the loss value is used for carrying out back propagation on the whole advertisement effect prediction model, the weight parameters of each link of the model are corrected, the gradient update of the model is realized, and the model is enabled to approach convergence continuously. Then, under the condition that the model is not converged, continuously acquiring a training sample corresponding to the next advertisement file in the training data set, and circularly iterating and continuously training the model. When the loss value reaches or infinitely approaches a preset threshold value, such as "0", it is considered that the advertisement performance prediction model has been trained to a convergent state, and thus the training thereof may be terminated.

According to the above embodiments, it can be understood that, when the advertisement success prediction model is trained, the marker feature generated according to whether the advertisement official document is generated by the service interface of the application is associated, and is used as the feature corresponding to the fact information, the model is trained to the convergence state, and then when the model is used for reasoning, the counter-fact inference principle can be applied, and intervention is realized by presetting the positive marker feature and the negative marker feature in the advertisement feature vector, so that the model can obtain the advertisement success indexes under the assumption of two conditions for comparison test, and the quality index of the service interface is determined through the comparison test, thereby realizing the quality evaluation of the intelligent advertisement creative service or function corresponding to the service interface.

On the basis of any of the above embodiments, referring to fig. 3, in step S2200 and/or step S1200, the encoding of the advertisement feature information according to each advertisement copy to obtain the advertisement feature vector thereof includes the following steps:

step S3100, calling advertisement characteristic information of the advertisement copy from a data interface provided by an advertisement delivery system;

the advertisement delivery system can realize a data interface for external calling of the corresponding advertisement characteristic information of each advertisement in the advertisement delivery system in advance, when the data interface is called, the corresponding advertisement object is determined according to the given advertisement file, then the advertisement characteristic information of the advertisement object required by the application is called and returned to the calling party for use, and therefore the advertisement characteristic information corresponding to each advertisement file in the training data set and the advertisement file set of the application can be obtained through calling.

Step S3200, carrying out data cleaning on the advertisement characteristic information to obtain characteristic data, and converting the characteristic data into corresponding characteristic values;

according to the requirement of extracting the advertisement characteristic vector corresponding to the advertisement file, particularly according to the characteristic requirement in the initial characteristic vector of the advertisement characteristic vector, the data cleaning can be firstly carried out on each advertisement characteristic information, and each characteristic data in the advertisement characteristic information can be cleaned.

The feature data may be collectively expressed in a numerical form, and each specific feature constituting the initial feature vector may be expressed in the numerical form according to the principle disclosed in step S2200 in the present application. For the case that the original feature information is not in numerical value form, for example, for semantic feature Product in commodity configuration feature information _i Label characteristic Label _product And the gender of the population Sex in the audience configuration profile _people Age group Age _people Region, Region _people Etc. may also be converted to corresponding numerical characteristics. For example, for the semantic feature Product _i The text feature extraction can be carried out on the commodity title of the corresponding commodity by means of a preset text feature extraction model, and the corresponding vector can be obtained and used as the numerical feature of the corresponding commodity. As another example, for the Region _people The corresponding codes of each region can be used as the corresponding data characteristics, etc. And the advertisement information is subjected to data cleaning to obtain characteristic data expressed in a numerical form, and each specific information is expressed as a characteristic value.

And S3300, encoding the characteristic values corresponding to the specific configuration information into advertisement characteristic vectors according to a preset ordering rule.

After the data of the advertisement characteristic information is cleaned, according to a preset coding sorting rule, the coding sorting rule is consistent in a training stage and an inference stage of the advertisement effect prediction model, and according to the sorting rule, all characteristic values are organized in order, so that an advertisement characteristic vector corresponding to an advertisement file can be formed. Of course, the advertisement feature vector may preset corresponding marker features, i.e. the positive marker features or the negative marker features, in adaptation to the training phase or the inference phase. Thus, the encoding process of the advertisement feature vector of each advertisement file is realized.

According to the embodiment, the advertisement characteristic information corresponding to the advertisement file is subjected to data cleaning to convert the characteristic information into characteristic values, the characteristic values are organized into the advertisement characteristic vectors according to the preset encoding rules, the encoding process of the advertisement characteristic information of the advertisement file is realized, the advertisement characteristic vectors are simplified, the comprehensive representation of the information of the advertisement corresponding to the advertisement file is realized through the advertisement characteristic vectors, and the advertisement success prediction model can obtain effective samples for training or reasoning and output effective advertisement success indexes.

On the basis of any of the above embodiments, referring to fig. 4, after the step S2400 of obtaining the quality index of the service interface according to the advertisement performance index comparison calculation of the positive sample and the negative sample, the method includes the following steps:

and S2500, judging whether the quality index is lower than a preset threshold value, and sending an alarm message to a preset communication interface when the quality index is lower than the preset threshold value.

As mentioned above, the quality index corresponding to the online intelligent advertising creative service of the e-commerce platform is essentially an evaluation result of the quality of the advertising copy generated by the service interface provided by the intelligent advertising creative service, if the quality index is low, the revenue of the generated advertising copy is relatively poor, and if the quality index is high, the revenue of the generated advertising copy is relatively good. Accordingly, the e-commerce platform can adopt a preset threshold which can be an actual measurement threshold or an experience threshold, the preset threshold is flexibly set by a person skilled in the art, the preset threshold is used for being compared with the quality index, when the quality index is higher than the preset threshold, no processing is performed, and the normal operation of the intelligent advertisement creative service is maintained; otherwise, when the quality index is lower than the preset threshold value, sending alarm information to a management user of the E-commerce platform by calling a preset communication interface so as to prompt the management user to optimize and upgrade the creative service of the intelligent advertisement, and opening a corresponding service interface after the optimization and the upgrade are completed.

According to the embodiment, the quality index is used for controlling the open authority of the service interface of the intelligent advertisement creative service, the closed loop of the utilization of the quality index is realized, the service interface can be timely controlled according to the quality index, whether the service interface can be called or not can be realized, when the intelligent advertisement creative service is not effective, early intervention can be carried out, the stop loss of an e-commerce platform can be timely realized, the use experience of a user related to advertisement putting is avoided, and the high-quality service is provided for an advertisement user.

Referring to fig. 5, a service interface quality evaluation apparatus adapted to one of the objectives of the present application is provided, which embodies the functionality of the service interface quality evaluation method of the present application, and the apparatus includes a document acquisition module 2100, an encoding processing module 2200, an index prediction module 2300, and a comparison evaluation module 2400, wherein: the document acquisition module 2100 is configured to acquire an advertisement document set, where the advertisement document set includes a plurality of advertisement documents generated by preset service interfaces; the encoding processing module 2200 is configured to obtain an advertisement feature vector according to the advertisement feature information encoding of each advertisement case, preset a positive flag feature and a negative flag feature in the advertisement feature vector, and generate a positive sample and a negative sample of the advertisement case correspondingly; the index prediction module 2300 is configured to respectively predict the advertisement performance indexes of the positive direction sample and the negative direction sample by using a preset advertisement performance prediction model, where the advertisement performance prediction model is trained to a convergence state in advance; the comparison evaluation module 2400 is configured to compare and calculate advertisement performance indicators of the positive samples and the negative samples to obtain a quality indicator of the service interface.

On the basis of any of the above embodiments, prior to the index prediction module 2300, the method includes: the data set acquisition module is used for acquiring a training data set, wherein the training data set comprises a plurality of released advertisement documents and advertisement effect indexes generated after the release of the advertisement documents; the sample generating module is used for obtaining the advertisement characteristic vector of each advertisement case according to the advertisement characteristic information code of the advertisement case, and generating a training sample according to whether the advertisement case is generated by a preset service interface or not and corresponding preset positive mark characteristics or negative mark characteristics in the advertisement characteristic vector; and the training execution module is used for carrying out iterative training on the advertisement effectiveness prediction model to a convergence state by adopting the training samples corresponding to the advertisement documents in the training data set, wherein the advertisement effectiveness indexes generated after the training samples are put are adopted to supervise the prediction result of the model.

On the basis of any of the above embodiments, the encoding processing module 2200 or the sample generation module includes: the information calling unit is used for calling the advertisement characteristic information of the advertisement file from a data interface provided by an advertisement delivery system; the data cleaning unit is used for carrying out data cleaning on the advertisement characteristic information to obtain characteristic data and converting the characteristic data into corresponding characteristic values; and the characteristic coding unit is used for coding the characteristic value corresponding to each specific configuration information into an advertisement characteristic vector according to a preset sorting rule.

On the basis of any of the above embodiments, the advertisement characteristic information includes any of the following items: the merchant release characteristic information is used for indicating the advertisement release scale of the merchant user to which the corresponding advertisement file belongs; the commodity configuration characteristic information is used for indicating commodities corresponding to the corresponding advertising copy; audience configuration characteristic information for indicating audience user groups of the corresponding advertising copy; budget settlement feature information indicating consumable funds for the corresponding advertising copy; the bidding configuration characteristic information is used for indicating bidding constraint information of the corresponding advertisement case; and the delivery configuration characteristic information is used for indicating delivery constraint information of the corresponding advertising copy.

On the basis of any of the above embodiments, the set of advertising copy comprises advertising copy in the training data set; or the advertisement effectiveness index is used for indicating click through rate, collection rate, purchase rate, conversion rate or input-output ratio corresponding to the advertisement putting effectiveness; or the advertisement file comprises any one or more items of text, pictures, videos and animations; or the service interface is used for calling a preset document template to generate an advertisement document corresponding to given information, and the given information comprises product words and/or brand words of the target commodity.

On the basis of any of the above embodiments, the alignment evaluation module 2400 includes: the mean value statistical unit is used for respectively calculating the mean values of the advertisement performance indexes of the positive samples and the negative samples corresponding to the advertisement file sets; and the index comparison unit is used for subtracting the average value corresponding to the negative sample from the average value corresponding to the positive sample to obtain a difference value, and the difference value is used as the quality index of the service interface.

Based on any of the above embodiments, the alignment evaluation module 2400 includes: and the decision processing module is used for judging whether the quality index is lower than a preset threshold value or not, and sending an alarm message to a preset communication interface when the quality index is lower than the preset threshold value.

In order to solve the technical problem, an embodiment of the present application further provides a computer device. As shown in fig. 6, the internal structure of the computer device is schematically illustrated. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected by a system bus. The computer readable storage medium of the computer device stores an operating system, a database and computer readable instructions, the database can store control information sequences, and when the computer readable instructions are executed by a processor, the processor can realize a commodity search category identification method. The processor of the computer device is used for providing calculation and control capability and supporting the operation of the whole computer device. The memory of the computer device may have stored therein computer readable instructions that, when executed by the processor, may cause the processor to perform the service interface quality assessment method of the present application. The network interface of the computer device is used for connecting and communicating with the terminal. Those skilled in the art will appreciate that the architecture shown in fig. 6 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In this embodiment, the processor is configured to execute specific functions of each module and its sub-module in fig. 5, and the memory stores program codes and various data required for executing the modules or the sub-modules. The network interface is used for data transmission to and from a user terminal or a server. The memory in this embodiment stores program codes and data necessary for executing all modules/sub-modules in the service interface quality evaluation device of the present application, and the server can call the program codes and data of the server to execute the functions of all sub-modules.

The present application also provides a storage medium storing computer-readable instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of the service interface quality assessment method of any of the embodiments of the present application.

The present application also provides a computer program product comprising computer programs/instructions which, when executed by one or more processors, implement the steps of the method as described in any of the embodiments of the present application.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments of the present application can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when the computer program is executed, the processes of the embodiments of the methods can be included. The storage medium may be a computer-readable storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random Access Memory (RAM).

In summary, the quality index can be determined for the function corresponding to the service interface for generating the advertisement file in the e-commerce platform, so that the early intervention evaluation of the related function is realized, and the e-commerce platform can be conveniently optimized and upgraded in time.

Those of skill in the art will appreciate that the various operations, methods, steps in the processes, acts, or solutions discussed in this application can be interchanged, modified, combined, or eliminated. Further, other steps, measures, or schemes in various operations, methods, or flows that have been discussed in this application can be alternated, altered, rearranged, broken down, combined, or deleted. Further, steps, measures, schemes in the prior art having various operations, methods, procedures disclosed in the present application may also be alternated, modified, rearranged, decomposed, combined, or deleted.

The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.

Claims

1. A service interface quality evaluation method is characterized by comprising the following steps:

2. The method for evaluating the quality of the service interface according to claim 1, wherein the step of predicting the advertisement performance indicators of the positive samples and the negative samples respectively by using a preset advertisement performance prediction model is preceded by the steps of:

3. The service interface quality assessment method according to claim 1 or 2, wherein the advertisement feature vector is obtained according to the advertisement feature information coding of each advertisement file, comprising the steps of:

calling the advertisement characteristic information of the advertisement copy from a data interface provided by an advertisement delivery system;

carrying out data cleaning on the advertisement characteristic information to obtain characteristic data, and converting the characteristic data into corresponding characteristic values;

4. The service interface quality assessment method according to claim 1 or 2, wherein said advertisement characteristic information comprises any of:

bid configuration feature information indicating bid constraint information for a corresponding advertising copy;

5. The method of claim 2, wherein the step of evaluating the quality of the service interface comprises:

the set of advertising copy comprises advertising copy in the training data set; or the like, or, alternatively,

6. The method for evaluating the quality of the service interface according to claim 1 or 2, wherein the quality index of the service interface is obtained by comparing and calculating the advertisement performance indexes of the positive sample and the negative sample, and the method comprises the following steps:

7. The method for evaluating the quality of the service interface according to claim 1 or 2, wherein the step of obtaining the quality index of the service interface according to the advertisement performance index comparison calculation of the positive sample and the negative sample comprises the following steps:

8. An apparatus for evaluating quality of service interface, comprising:

the file acquisition module is used for acquiring an advertisement file set, wherein the advertisement file set comprises a plurality of advertisement files generated by preset service interfaces;

the coding processing module is used for coding according to the advertisement characteristic information of each advertisement case to obtain an advertisement characteristic vector of each advertisement case, presetting a positive marking characteristic and a negative marking characteristic in the advertisement characteristic vector respectively, and correspondingly generating a positive sample and a negative sample of the advertisement case;

the index prediction module is used for predicting the advertisement effectiveness indexes of the positive samples and the negative samples respectively by adopting a preset advertisement effectiveness prediction model, and the advertisement effectiveness prediction model is trained to be in a convergence state in advance;

and the comparison evaluation module is used for comparing and calculating the advertisement performance indexes of the positive samples and the negative samples to obtain the quality indexes of the service interfaces.

9. A computer device comprising a central processor and a memory, characterized in that the central processor is adapted to invoke execution of a computer program stored in the memory to perform the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that it stores, in the form of computer-readable instructions, a computer program implemented according to the method of any one of claims 1 to 7, which, when invoked by a computer, performs the steps comprised by the corresponding method.