CN115730046A

CN115730046A - Marketing auxiliary information generation method, marketing auxiliary information generation device, marketing auxiliary information generation medium and electronic equipment

Info

Publication number: CN115730046A
Application number: CN202110987853.9A
Authority: CN
Inventors: 张国成; 洪沛; 杨国锋; 徐虎; 李冠华; 戴胜林; 马亮; 张超; 费相如; 王璇
Original assignee: China Telecom Corp Ltd
Current assignee: China Telecom Corp Ltd
Priority date: 2021-08-26
Filing date: 2021-08-26
Publication date: 2023-03-03

Abstract

The application relates to the field of auxiliary marketing, and discloses a method, a device, a medium and electronic equipment for generating marketing auxiliary information. The method comprises the following steps: extracting structured data; converting the structured data into triple information to obtain shallow triple information; acquiring a data table; performing variable screening operation on the data table to obtain a final data table; establishing a basic classification prediction model by utilizing the final data table; determining the interpretation weight of each feature on the service data according to the service data associated with the user in the final data table by taking the variable in the final data table as the feature; for each service data, determining target characteristics according to the interpretation weight of each characteristic to the service data, and generating deep triple information according to variable values of the target characteristics; constructing a knowledge graph by utilizing the shallow triple information and the deep triple information; and querying the knowledge graph to obtain a retrieval result, and generating marketing auxiliary information according to the retrieval result. The method can improve marketing efficiency and success rate.

Description

Marketing auxiliary information generation method, marketing auxiliary information generation device, marketing auxiliary information generation medium and electronic equipment

Technical Field

The present application relates to the field of marketing assistance technologies, and in particular, to a method, an apparatus, a medium, and an electronic device for generating marketing assistance information.

Background

When a front-line marketer is used for marketing in business, the marketer generally queries a single customer information system and then carries out customer marketing according to the queried information. The defect of the mode is that the acquired information is single, and the acquired information is represented by data, so that the acquired information is shallow in hierarchy, and the assistance of marketers in the aspect of assisting marketing is not large.

Disclosure of Invention

In the technical field of auxiliary marketing, in order to solve the technical problems, an object of the present application is to provide a method, an apparatus, a medium, and an electronic device for generating marketing auxiliary information.

According to an aspect of the present application, there is provided a method of generating marketing assistance information, the method including:

extracting structured data from a big data platform, wherein the structured data comprises user related data and product related data, and the big data platform comprises a plurality of information management systems;

converting the structured data into triple information to obtain shallow triple information;

acquiring a data table from the big data platform, wherein the data table comprises a plurality of items of service data, the service data comprises variables and variable values corresponding to the variables, and each item of service data is associated with one user;

performing variable screening operation on the data table to remove at least one variable in the data table to obtain a final data table;

establishing a basic classification prediction model by utilizing the final data table;

determining the marginal contribution average value of each feature in the service data to the prediction result of the basic classification prediction model according to the service data associated with each user in the final data table by taking the variable in the final data table as the feature, wherein the marginal contribution average value is used as the interpretation weight of each feature to the service data;

for each service data, determining a target feature according to the interpretation weight of each feature to the service data, and generating deep triple information according to a variable value corresponding to the target feature, wherein the interpretation weight of the target feature is greater than that of other features;

constructing a knowledge graph by using the shallow triple information and the deep triple information;

and when a service problem retrieval request is received, obtaining a retrieval result by inquiring the knowledge graph, and generating marketing auxiliary information according to the retrieval result.

According to another aspect of the present application, there is provided a marketing assistance information generating apparatus, the apparatus including:

an extraction module configured to extract structured data from a big data platform, the structured data comprising user-related data and product-related data, the big data platform comprising a plurality of information management systems;

the conversion module is configured to convert the structured data into triple information to obtain shallow triple information;

the acquisition module is configured to acquire a data table from the big data platform, wherein the data table comprises a plurality of items of service data, the service data comprises variables and variable values corresponding to the variables, and each item of service data is associated with one user;

the removing module is configured to perform variable screening operation on the data table so as to remove at least one variable in the data table to obtain a final data table;

a building module configured to build a base classification prediction model using the final data table;

the determining module is configured to determine, by taking the variable in the final data table as a feature, an average marginal contribution value of each feature in the service data to a prediction result of the basic classification prediction model according to the service data associated with each user in the final data table, and the average marginal contribution value is used as an interpretation weight of each feature to the service data;

the first generation module is configured to determine target characteristics according to the interpretation weight of each characteristic on the business data and generate deep triple information according to variable values corresponding to the target characteristics, wherein the interpretation weight of the target characteristics is greater than that of other characteristics;

a construction module configured to construct a knowledge graph using the shallow triple information and the deep triple information;

and the second generation module is configured to obtain a retrieval result by querying the knowledge graph when a service problem retrieval request is received, and generate marketing auxiliary information according to the retrieval result.

According to another aspect of the present application, there is provided a computer readable program medium storing computer program instructions which, when executed by a computer, cause the computer to perform the method as previously described.

According to another aspect of the present application, there is provided an electronic device including:

a processor;

a memory having computer readable instructions stored thereon which, when executed by the processor, implement the method as previously described.

The technical scheme provided by the embodiment of the application can have the following beneficial effects:

according to the marketing auxiliary information generation method, the marketing auxiliary information generation device, the marketing auxiliary information generation medium and the electronic equipment, the shallow triple information and the deep triple information are respectively obtained, the knowledge map is constructed by utilizing the shallow triple information and the deep triple information, the deep triple information is generated according to the variable value of the target characteristic, and the target characteristic is generated according to the interpretation weight of the characteristic on the business data, so that more comprehensive and more profound business knowledge is formed by combining the shallow knowledge and the deep knowledge, after a marketing person inputs a business problem, a corresponding query result can be obtained by querying the knowledge map, the knowledge fed back to the marketing person is more accurate and visual, and is popular and easy to understand, and the marketing auxiliary information generation method can assist the first-line marketing person to improve the marketing efficiency and the success rate; meanwhile, the one-stop intelligent question-answering mode is adopted to feed back the question answers to the marketers, so that the marketers can acquire knowledge more conveniently and accurately.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

Fig. 1 is a system architecture diagram illustrating a method of generating marketing assistance information in accordance with an exemplary embodiment;

FIG. 2 is a flow diagram illustrating a method of generating marketing assistance information in accordance with an exemplary embodiment;

FIG. 3 is a diagram illustrating a data table upon which structured data is extracted, according to an example embodiment;

4A-4B are schematic diagrams illustrating shallow triplet information in accordance with an exemplary embodiment;

FIG. 5 is a diagram illustrating a partial data table obtained from a big data platform in accordance with an exemplary embodiment;

FIG. 6 is a schematic diagram of a final data table shown in accordance with an exemplary embodiment;

FIG. 7 is a diagram illustrating an interpretable model based on machine learning according to an exemplary embodiment;

FIG. 8 is a schematic illustration of a flowchart illustrating calculation of an average contribution margin value, in accordance with an exemplary embodiment;

fig. 9 is a diagram illustrating the impact of various features visually output based on a DKMM model on a prediction outcome according to an example embodiment;

FIG. 10 is a diagram illustrating the construction and use of a knowledge-graph corresponding to deep triplet information, according to an exemplary embodiment;

FIG. 11 is a diagram illustrating a knowledge-graph constructed based on deep triplet information, in accordance with an exemplary embodiment;

FIG. 12 is a schematic diagram illustrating a knowledge-graph constructed based on shallow triple information and deep triple information, according to an example embodiment;

FIG. 13 is a diagram illustrating the generation of answers to questions based on input questions, according to an exemplary embodiment;

fig. 14 is a diagram illustrating answer information corresponding to a query question according to a method of generating marketing assistance information, according to an exemplary embodiment;

fig. 15 is a block diagram illustrating a marketing assistance information generation apparatus according to an exemplary embodiment;

FIG. 16 is a block diagram illustrating an example of an electronic device implementing the method for generating marketing assistance information described above, according to an example embodiment;

fig. 17 is a program product for implementing the above-described marketing assistance information generation method according to an exemplary embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

Furthermore, the drawings are merely schematic illustrations of the present application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities.

In the related art, when a front-line marketing person carries out accurate marketing of 5G business, the traditional method is to mine potential high-probability users as target customers by a machine learning classification method and send orders to a front line for marketing. Because the machine learning classification model only gives a prediction result and cannot explain the reason of high probability, a front-line person can only acquire surface knowledge by inquiring a plurality of systems (a knowledge base, a CRM (customer relationship management), a product base, a label base and the like), the efficiency is low, the knowledge is group-specific, deep knowledge such as key factors which have the greatest influence on each user cannot be acquired, the multi-user differentiated conversational marketing cannot be carried out, and the marketing effect is not ideal.

Although more information can be retrieved by adopting the related technology, the obtained information is scattered, messy and high in dispersion, and accurate answers are difficult to query quickly; meanwhile, the obtained explanations are all group-oriented, and the auxiliary requirement of the individual for developing the operation of thousands of people cannot be met. The model can only output results, is not explanatory, and the process is a closed black box, so that the cause and effect relationship reflected by the output cannot be understood manually.

Therefore, the application firstly provides a method for generating marketing auxiliary information, which can overcome the defects, realize the efficient and accurate generation of the marketing auxiliary information and meet the individual requirement for carrying out the auxiliary operation of thousands of people.

The implementation terminal of the present application may be any device having computing, processing, and communication functions, which may be connected to an external device for receiving or sending data, and specifically may be a portable mobile device, such as a smart phone, a tablet computer, a notebook computer, a PDA (Personal Digital Assistant), or the like, or may be a fixed device, such as a computer device, a field terminal, a desktop computer, a server, a workstation, or the like, or may be a set of multiple devices, such as a physical infrastructure of cloud computing or a server cluster. Optionally, the implementation terminal of the present application may be a server or a physical infrastructure of cloud computing.

The scheme of the embodiment of the application can be applied to marketing scenes of various products or services, and the products can be solid products or virtual products. The entity products can be products such as smart phones and household appliances; the virtual product may be a package of the operator, an insurance of an insurance company, or the like.

Fig. 1 is a system architecture diagram illustrating a method for generating marketing assistance information according to an exemplary embodiment. As shown in fig. 1, the system architecture of the system includes three components, which are a system application model, a DKMM algorithm model and a data acquisition module, where the DKMM (deep knowledge mining model) is a model capable of executing the method for generating marketing assistance information provided in the embodiment of the present application. The system has 4 system roles, namely an electronic outbound person, a customer manager direct sales person, a customer service marketing person and a store marketing person, namely all the persons with the 4 system roles can use the system to obtain marketing auxiliary information. In fig. 1, the data acquisition module can acquire various types of data, which are CRM data, knowledge base data, tag data, and product data, and these data may be acquired from a plurality of systems, for example, may be acquired from a CRM system, a knowledge base system, a tag base system, a product base system, and the like. The data extracted by the data acquisition module is used to construct a model of the DKMM algorithm. The DKMM comprises a service width table, a graph database established based on the service width table, a plurality of functional modules and an ability layer, wherein the graph database can comprise a knowledge graph generated by the service width table; the functional modules in the DKMM algorithm model may include a business model, a classification model, an interpretation module, knowledge reasoning, and a knowledge retrieval engine. The capability layer in the DKMM algorithm model provides the capability of the DKMM algorithm model for a system application module, the system application module comprises a unified knowledge retrieval query entry, various system roles in the system can query required marketing auxiliary information in the DKMM algorithm model in a one-stop mode through the unified knowledge retrieval query entry, and the marketing auxiliary information can comprehensively, accurately and colloquially reflect marketing knowledge, for example, the reason that a marketing object needs a product is provided, so that marketing personnel are assisted to improve business marketing efficiency and success rate.

Fig. 2 is a flow chart illustrating a method of generating marketing assistance information, according to an exemplary embodiment. The marketing assistant information generation method provided by the embodiment can be executed by the server, and can be applied to marketing activities in which an operator upgrades a 4G package to a 5G package. As shown in fig. 2, the method comprises the following steps:

step 210, extracting structured data from the big data platform.

The structured data includes user-related data and product-related data, and the big data platform includes a plurality of information management systems.

Specifically, in the field of communications, the information management system may be any one of the following systems: CRM, knowledge base, label base, product base. CRM is an abbreviation for Customer information Management (Customer Relationship Management). Of course, in other fields, the information management system may take other types. Structured data is data that is stored in the form of a database. The database comprises a data table, and data corresponding to the fields are recorded in the data table, and the data are structured data.

FIG. 3 is a diagram illustrating a data table upon which structured data is extracted, according to an example embodiment. Referring to FIG. 3, a plurality of data tables are shown, so that data can be extracted from the plurality of data tables in a big data platform; each data table includes fields, sequence numbers corresponding to the fields, and field meanings, for example, a data table in the upper left corner of fig. 3 includes a field "NET _ NUM", where the field meaning corresponding to the field is: whether it is on the net. Each data table contains different types of fields so that data can be extracted from multiple data tables of a large data platform.

In particular, triplet information includes two entities and a relationship or attribute for establishing a connection between the two entities. In order to convert the structured data into the triple information, the extraction of the structured data needs to be performed from the perspective of the triple.

For example, in the 5G marketing field, data extraction can be performed based on a knowledge graph Schema framework, including 9 types of entities of users, packages, terminals, package areas, developers, package instances, clients, accounts, and tags, and including 8 relationships among user-package, user-terminal, package-package instances, user-developers, user-clients, user-accounts, and user-package instances; and acquiring 5G service information related data through a big data platform, wherein the source data mainly comprises user basic attributes, ordering relation data, personalized interest data and the like. 16 entities and relationship tables are generated by extracting data and 8 relationships among 8 types of entities of a user (163 fields), a package (10 fields), a terminal (156 fields), a package area (8 fields), a developer (3 fields), a package instance (34 fields), a client (5 fields) and an account (7 fields) through table data association. Wherein the label may indicate whether the user ordered the 5G package. Here, the customer refers to the owner of the telecommunication product under the same registration name. The client communication information comprises all product information under the unit name; the account refers to the attribution of accounting under the client, and one client can own one account or multiple accounts. A subscriber is a telecommunications product and an entity based on value added services on top of the product, such as a mobile phone or broadband service. The relationship of customer, account, user can be summarized as: there may be multiple accounts under the customer, with multiple users under the accounts.

Step 220, converting the structured data into triple information to obtain shallow triple information.

Specifically, fields in all entity tables are used as attributes of corresponding entities, and the relationship table is converted into triples in similar (user ID, order package, package ID), (package ID, attributes, presentation traffic) (tags, attributes, tag interpretation) forms.

Fig. 4A-4B are schematic diagrams illustrating shallow triplet information, according to an example embodiment. As shown in fig. 4A, the shallow triple information shown is (user ID, holding terminal, terminal model); in fig. 4B, not only this shallow triplet information (user ID, holding terminal, terminal model) but also other forms of shallow triplet information, for example, it shows this shallow triplet information (user ID, home package, package name). It is easy to understand that, since the triple information corresponds to the knowledge-graph, the shallow triple information shown in fig. 4A and 4B can also be understood as a knowledge-graph established based on the shallow triple information.

Step 230, obtaining a data table from the big data platform.

The data table comprises a plurality of items of service data, the service data comprises variables and variable values corresponding to the variables, and each item of service data is associated with one user.

The data table may be the same as or different from the data table from which the structured data was previously extracted. The variables here correspond to the aforementioned fields. One item of service data is a collection of data corresponding to each field.

The data table is obtained from a big data platform, and the data table comprises 101 fields, wherein the fields can comprise packages, package values, fusion or not, roaming or not, user preferences and the like. Fig. 5 is a schematic diagram illustrating a partial data table obtained from a big data platform according to an exemplary embodiment, and fig. 5 illustrates a plurality of fields and data corresponding to the fields, where data of each row may be a service data.

And 240, performing variable screening operation on the data table to remove at least one variable in the data table to obtain a final data table.

In one embodiment, before performing the variable filtering operation on the data table, the method further comprises: for each variable, determining the number proportion of the business data of which the variable value corresponding to the variable is a null value in all the business data; and eliminating the variable according to the condition that the data ratio is greater than a preset ratio threshold value.

The predetermined ratio threshold may be set arbitrarily empirically, and may be set to 80%, for example. In the embodiment of the application, the variable with higher null value ratio is removed, so that the validity of the data is ensured.

The data in the data table can be further processed to further improve the effectiveness of the data. For example, for discrete variables, a WOE (Weight of Evidence) method is used for carrying out feature transformation to form variables which can be identified by a program; for continuous variables, the z-score method was used for feature normalization and units of data were removed.

In an embodiment, the performing a variable screening operation on the data table to remove at least one variable in the data table to obtain a final data table includes: and sequentially performing variable screening operation on the data table by using a chi-square test method, a correlation coefficient calculation method and an information value evaluation method to remove at least one variable in the data table to obtain a final data table.

Specifically, the chi-square test method can be used for testing the difference of independent variables to qualitative dependent variables, and the variables with obvious difference are retained, for example, the variables with the difference larger than a preset threshold are retained, and other variables are removed; calculating the correlation coefficient among the variables by using a correlation coefficient method, verifying whether multiple collinearity exists among the variables, screening the variables, selecting the variables with the correlation coefficient of more than 0.5, and rejecting other variables; the prediction ability of the variables is evaluated by an IV (Information Value) method, and the variables are screened, thereby obtaining a final data table. FIG. 6 is a diagram illustrating a final data table according to an example embodiment. The final data table may be as shown in fig. 6, and includes corresponding field names, field types, chinese comments, and sample data, where the field name is a variable that is retained after the variable is removed, and the sample data is a variable value corresponding to the variable.

For example, fk _ cnt is a variable, where the text is annotated as the number of slave cards, and the corresponding variable value is 2.

In the embodiment of the application, by performing variable elimination, variables which are helpful for generating marketing auxiliary information are obtained.

And step 250, establishing a basic classification prediction model by using the final data table.

Each item of business data in the final data table can be used as a sample for building a basic classification prediction model, and the business data can comprise a label indicating whether a user upgrades a 5G package. Specifically, the final data table may be divided into a training set, a test set, and a verification set according to a predetermined ratio, then the training set is used to train the basic classification prediction model, the test set is used to perform model testing, and the verification set is used to perform model verification, where the predetermined ratio may be 8.

In one embodiment, the building a base classification prediction model using the final data table includes: respectively training by using the final data table to obtain a logistic regression model, a CART model and an Xgboost model; and establishing a basic classification prediction model according to the logistic regression model, the CART model and the Xgboost model, wherein the prediction result of the basic classification prediction model is a weighted calculation result obtained by carrying out weighted calculation on the output results of the logistic regression model, the CART model and the Xgboost model.

A Logistic Regression (LR) model is a model in which a Logistic function is applied based on linear Regression, and is a generalized linear Regression analysis model; the CART (Classification And Regression Trees) model is an algorithm established based on a decision tree that can be used for both Classification And Regression; the Xgboost model is an ensemble learning model built by integrating a plurality of base learners.

In other words, the basic classification prediction model may comprise a weight calculation module for outputting weight calculation results, which are respectively connected to the output terminals of the logistic regression model, the CART model and the Xgboost model.

In the embodiment of the application, the multiple models are integrated in the established basic classification prediction model, and the final model output result is obtained by weighting and calculating the output results of the multiple models, so that the accuracy of model prediction is further improved.

The basic classification prediction model may output a success rate of marketing. For example, the success rate of upgrading the 5G package by the user may be output.

And step 260, determining a marginal contribution average value of each feature in the service data to the prediction result of the basic classification prediction model according to the service data associated with each user in the final data table by taking the variable in the final data table as the feature, and using the marginal contribution average value as an interpretation weight of each feature to the service data.

The variable value corresponding to the feature corresponds to the feature value.

In one embodiment, the determining, by using the variables in the final data table as the features, an average value of marginal contributions of each feature in the service data to the prediction result of the basic classification prediction model according to the service data associated with each user in the final data table, as an interpretation weight of each feature to the service data, includes: dividing the service data in the final data table into a plurality of layers according to a preset rule, wherein each layer comprises a plurality of service data; selecting one service data from the final data table as selected service data, and iteratively executing a marginal contribution value determining step for each feature until a predetermined number of times is executed, wherein the marginal contribution determining step comprises: randomly generating a characteristic sequence, and respectively sequencing the selected service data and the service data in the final data table according to the characteristic sequence; selecting a layer from unselected layers, and randomly selecting a service data from the service data of the layer as a constructed service data corresponding to the selected service data; respectively constructing first instance service data and second instance service data according to the selected service data and the constructed service data, wherein the first instance service data comprises variable values corresponding to the features in the selected service data and the features before the features and variable values corresponding to the features after the features in the constructed service data, and the second instance service data comprises variable values corresponding to the features in the selected service data before the features and variable values corresponding to the features in the constructed service data and the features after the features; respectively inputting the selected service data and the constructed service data into the basic classification prediction model to respectively obtain a first prediction result corresponding to the selected service data and a second prediction result corresponding to the constructed service data; determining a marginal contribution value corresponding to the feature based on the first prediction result and the second prediction result; for each feature, determining a marginal contribution average value as an interpretation weight of the feature on the traffic data based on marginal contribution values obtained by performing the step of determining a marginal contribution value for the feature.

In the embodiment of the application, the business data are layered, the constructed business data are selected according to the layers, the two example business data and the second example business data are respectively constructed according to the constructed business data, the marginal contribution value is determined based on the first prediction result and the second prediction result, the accuracy of the determined marginal contribution value is ensured, the marginal contribution value can be equivalent to the weighted score of each feature on the model prediction result, and therefore the interpretability of the system is improved.

FIG. 7 is a diagram illustrating an interpretable model based on machine learning according to an exemplary embodiment. The interpretable model based on machine learning is a core part of the DKMM algorithm model, and is established by the following process as shown in fig. 7:

1. establishing machine learning classification algorithm by using data

Dividing data into a training set, a testing set and a verification set; and respectively establishing an LR model, a CART model and an XGBOOST model according to the requirements of the service scene, and adjusting the parameters of each model to be optimal.

2. Prediction of basic classification prediction model

And (5) training the results of the LR model, the CART model and the XGBOOST model again, and combining the results into a final prediction result in a weighted mode to serve as the prediction result of the basic classification prediction model integrating the LR model, the CART model and the XGBOOST model.

3. Building interpretable modules

The interpretable algorithm is constructed by randomly drawing instances x, z.

4. The output is interpreted.

And outputting the contribution value of each characteristic to each user. For each user analysis, a weighted score for each feature to the model prediction can be calculated.

FIG. 8 is a schematic diagram illustrating a flow of computing an average contribution margin value, according to an exemplary embodiment. The embodiment shown in fig. 8 may be applied to the field of telemarketing, such as marketing of 5G packages, and the process of determining the average value of the marginal contribution in the above embodiment may be specifically implemented by fig. 8. As shown in fig. 8, the calculation flow includes the following steps:

1. hierarchical sampling of data

Because the telecommunication user package amount has obvious hierarchical characteristics, the user is divided into a plurality of layers according to the consumption grade by utilizing the statistical hierarchical sampling technology. For example, the monthly expenditure amount can be divided into the layers of 0-50, 51-100, 101-200, 201-300 and more than 300.

2. A random order of features o is generated.

The step of randomly sequencing the features to be calculated can reduce the influence of the sequencing of the features in the original table on the result.

3. A random order instance is generated.

First generation random order instance x _o ＝(x ₁ ，..，x _j ，..，x _p ) Regenerating a random sequence instance z _o ＝(z ₁ ，..，z _j ，…，z _p ) Wherein x is _o Corresponding to the selected service data in the previous embodiment, z _o Corresponding to the construction of the service data in the previous embodiment.

For example, arbitrarily take any user x _o (e.g., features such as age 41, male, 59 package), any random user z is taken in a hierarchy in step 1 _o (20 years old, female, 99 set, etc.).

4. Building feature instances

Example x constructed with feature j _+j ＝(x ₁ ，...，x _j-1 ，x _j ，z _j+1 ，...，z _p ) Construct instance x without feature j _-j ＝(x ₁ ，...，x _j-1 ，z _j ，z _j+1 ，...，z _p ) All of which are based on x _o And z _o The construction is carried out. Wherein x is _+j Corresponding to the first instance service data, x, in the previous embodiment _-j Corresponding to the second example service data in the previous embodiment. For example, the extracted instances are reconstructed to form new feature users x _+j (41 years old, male, 99 yuan package, etc.), x _-j (41 years old, female, 99 yuan package, etc.)

5. Calculating marginal contribution

New feature user x _+j 、x _-j Substituting into the basic classification prediction model to calculate the marginal contribution of single feature (such as gender) to the prediction result. In particular, by

This formula calculates the contribution margin value, wherein,

the predictive model is classified on the basis.

6. Calculating a marginal contribution average

For example x _+j Randomly extracting different users z from different layers, calculating the marginal contribution value by using different users z each time, circularly iterating for M times, and finally utilizing

This formula calculates the mean as the marginal contribution of feature j, i.e., the mean marginal contribution, which is the j feature pair x _o Interpretation weights of sample predictors.

In one embodiment, the method further comprises: and outputting the interpretation weight of each characteristic to the service data in a visual mode according to the user request. Specifically, the interpretation weights may be output in a graphical manner.

Fig. 9 is a diagram illustrating the impact of various features visually output based on a DKMM model on a prediction result according to an example embodiment. The embodiment of fig. 9 is applied to the marketing process of 4G migration 5G. As shown in fig. 9, the influence of each feature in the telecommunication data set on the prediction result can be observed through the DKMM model, the decision process of the final prediction result is visually presented, and a certain customer migration reason is found in an assisted manner, as shown in fig. 8, wherein a node on the right side of f (x) is a feature playing a negative role, a node on the left side of f (x) is a feature playing a positive role, the initial score of the user is 23.4 points, the initial score of ofr _ cdma _ nums (the number of mobile phones under a package) feature is 8.17 points increased for the user, the average traffic (avg _ flux _3 m) feature user is reduced by 5.39 points, and after all features are traversed, the final score of the user is 64.03 points, that is, the migration probability of the user is 64.03%. When the method is formally applied, the forward factors are characterized as user labels, sales dialogs are matched, interpretable reasons of users are finally output, first-line marketing is assisted, and success rate is improved.

And 270, determining target characteristics according to the interpretation weight of each characteristic on the service data and generating deep triple information according to variable values corresponding to the target characteristics for each service data.

Wherein the interpretation weight of the target feature is greater than that of other features.

In one embodiment, the determining, for each service data, a target feature according to an interpretation weight of each feature on the service data includes: for each service data, sequencing the interpretation weight of each feature to the service data from large to small; and taking the first predetermined number of the characteristics as target characteristics.

In one embodiment, the determining, for each service data, a target feature according to an interpretation weight of each feature on the service data includes: and acquiring the characteristic that the absolute value of the interpretation weight of the service data is greater than a preset interpretation weight threshold value as a target characteristic for each service data.

The interpretation weight of the feature on the traffic data may be a positive value or a negative value.

In one embodiment, the method further comprises: and generating deep triple information according to the prediction result of the basic classification prediction model on the service data. In particular, for example, the deep triplet information may include whether a user would like to upgrade a 5G package. Thus, the marketing assistant information can be provided more accurately and colloquially.

And step 280, constructing a knowledge graph by using the shallow triple information and the deep triple information.

In practical application, the shallow triple information and the deep triple information can be imported into a Neo4j graph database, so that a 5G service knowledge graph is constructed.

FIG. 11 is a diagram illustrating a knowledge-graph constructed based on deep triplet information, according to an example embodiment. As shown in fig. 11, the knowledge graph constructed based on the deep triple information includes various knowledge such as the overflow times of the process in the last 3 months corresponding to the unique ID of the user.

FIG. 12 is a schematic diagram illustrating a knowledge-graph constructed based on shallow triple information and deep triple information, according to an example embodiment. As shown in fig. 12, the constructed knowledge graph includes 3 kinds of information, which are shallow knowledge, interpretable factors, and basic attribute factors, for example, age 36 corresponding to the user ID is a basic attribute factor, flow saturation higher corresponding to the user ID is an interpretable factor, donation flow 40G corresponding to 5G189 for enjoying is shallow knowledge, where the shallow knowledge is shallow triplet information, and the interpretable factor is triplet information.

And 290, when a service problem retrieval request is received, obtaining a retrieval result by querying the knowledge graph, and generating marketing auxiliary information according to the retrieval result.

In one embodiment, after generating marketing assistance information according to the retrieval result, the method further comprises: and returning the marketing auxiliary information to the sending end of the business problem retrieval request.

Specifically, a unified knowledge retrieval query entry may be provided, which may be set in a text entry box manner; the user can submit the service question searching request by inputting the service question in the knowledge searching and inquiring entrance.

Fig. 10 is a diagram illustrating the construction and use of a knowledge-graph corresponding to deep triplet information, according to an example embodiment. As shown in fig. 10, first, in the model prediction stage, the possibility of upgrading 4G to 5G of the user is predicted, and the result is high probability; then, in the feature weighting stage, model interpretation is carried out by using the interpretable module in the embodiment to obtain interpretation weight corresponding to each feature; then, in the map construction stage, constructing a knowledge map based on the interpretation result; next, in a knowledge translation stage, performing knowledge translation and retrieval; finally, in the knowledge output stage, the knowledge output is realized, so that not only the probability condition that the user 4G rises to 5G can be output, but also the reason behind the result can be output.

In one embodiment, the constructing a knowledge graph using the shallow triple information and the deep triple information includes: importing the shallow triple information and the deep triple information into a graph database to form a knowledge graph in the graph database; when a service problem retrieval request is received, a retrieval result is obtained by inquiring the knowledge graph, and marketing auxiliary information is generated according to the retrieval result, wherein the method comprises the following steps: when a service problem retrieval request is received, acquiring a service problem in the service problem retrieval request; determining a result template matched with the service problem; inquiring the knowledge graph in the graph database to obtain a retrieval result corresponding to the service problem; and filling the retrieval result into the result template to obtain marketing auxiliary information.

In one embodiment, the determining the result template matching the business issue includes: determining the category of the business problem through a preset classification model; determining a problem template matched with the business problem from the problem templates corresponding to the categories through a first syntactic analysis model, and using the problem template as a target problem template; acquiring a result template corresponding to the target problem template as a result template matched with the service problem; the querying the knowledge graph in the graph database to obtain a retrieval result corresponding to the business problem includes: extracting a query object in the business problem through a second syntactic analysis model; and inquiring the knowledge graph in the graph database according to the inquiry object to obtain a retrieval result corresponding to the service problem.

The query object can be information of entities, attributes, relations and the like to be queried in the business problem. A plurality of question templates and a result template corresponding to each question template may be set in advance.

Specifically, the preset classification model may be a naive bayes model, the second syntactic analysis model and the first syntactic analysis model may be the same model, and the second syntactic analysis model and the first syntactic analysis model may be implemented by a Bert model.

FIG. 13 is a diagram illustrating generation of answers to questions based on input questions, according to an exemplary embodiment. Fig. 13 shows the following process: firstly, extracting data from a 5G information database, and preprocessing the extracted data; then, importing the preprocessed data into a DKMM model; after inputting the questions, as a query answer, inputting the questions into a DKMM model, performing Chinese word segmentation on the questions to obtain word segmentation results, classifying the input questions through a Bayesian classifier, performing question matching through a standard question bank, and determining a matched Cypher template file; then, carrying out syntactic analysis on the segmentation result to realize entity identification to obtain an entity; then, searching by utilizing the triple search library to obtain a corresponding retrieval result; then, cypher construction is carried out based on the retrieval result, the Cypher construction is combined with the determined Cypher template file, and the combination result is used as an answer to be provided for the Neoj service; the Neoj service returns the answer, generating an answer.

Fig. 14 is a diagram illustrating answer information corresponding to a query question according to a method of generating marketing assistance information, according to an exemplary embodiment. As shown in fig. 14, the method for generating marketing assistance information according to the embodiment of the present application can generate a matching and accurate answer regardless of any desired questions input by a marketer. For example, when a question of "1811966XXXX users are why they are 5G high-probability users" is input, an answer of "1811966XXXX users are not enough in traffic, like playing games, young group" can be automatically generated. Therefore, key factors for upgrading 5G users with high probability can be obtained.

The application also provides a device for generating the marketing auxiliary information, and the following embodiment of the device is provided.

Fig. 15 is a block diagram illustrating a marketing assistance information generation apparatus according to an exemplary embodiment. As shown in fig. 15, the apparatus 1500 includes: an extraction module 1510 configured to extract structured data from a big data platform, the structured data comprising user-related data and product-related data, the big data platform comprising a plurality of information management systems; a conversion module 1520, configured to convert the structured data into triple information, resulting in shallow triple information; an obtaining module 1530 configured to obtain a data table from the big data platform, where the data table includes multiple items of service data, the service data includes variables and variable values corresponding to the variables, and each item of service data is associated with one user; a removing module 1540 configured to perform variable screening operation on the data table to remove at least one variable in the data table to obtain a final data table; a building module 1550 configured to build a base classification prediction model using the final data table; a determining module 1560, configured to determine, according to the service data associated with each user in the final data table, an average value of marginal contribution of each feature in the service data to the prediction result of the basic classification prediction model as an interpretation weight of each feature to the service data, by taking the variable in the final data table as a feature; a first generating module 1570 configured to, for each service data, determine a target feature according to an interpretation weight of each feature on the service data, and generate deep triple information according to a variable value corresponding to the target feature, where the interpretation weight of the target feature is greater than that of other features; a construction module 1580 configured to construct a knowledge graph using the shallow triplet information and the deep triplet information; the second generation module 1590 is configured to, when a service problem retrieval request is received, obtain a retrieval result by querying the knowledge graph, and generate marketing assistance information according to the retrieval result.

According to a third aspect of the present application, there is also provided an electronic device capable of implementing the above method. As will be appreciated by one skilled in the art, aspects of the present application may be embodied as a system, method or program product. Accordingly, various aspects of the present application may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system. An electronic device 1600 according to this embodiment of the application is described below with reference to fig. 16. The electronic device 1600 shown in fig. 16 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application. As shown in fig. 16, electronic device 1600 is in the form of a general purpose computing device. Components of electronic device 1600 may include, but are not limited to: the at least one processing unit 1610, the at least one memory unit 1620, and a bus 1630 that couples various system components including the memory unit 1620 and the processing unit 1610. Wherein the storage unit stores program code that can be executed by the processing unit 1610, so that the processing unit 1610 performs the steps according to various exemplary embodiments of the present application described in the section "method of embodiment" above in this specification. The memory unit 1620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM) 1621 and/or a cache memory unit 1622, and may further include a read only memory unit (ROM) 1623. The storage unit 1620 may also include a program/utility 1624 having a set (at least one) of program modules 1625, such program modules 1625 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment. Bus 1630 may be any of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures. The electronic device 1600 can also communicate with one or more external devices 1800 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 1600, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 1600 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 1650, such as with a display unit 1640. Also, the electronic device 1600 can communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the internet) via the network adapter 1660. As shown, the network adapter 1660 communicates with the other modules of the electronic device 1600 via the bus 1630. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with electronic device 1600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the embodiments of the present application.

According to a fourth aspect of the present application, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, various aspects of the present application may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the present application described in the "exemplary methods" section above of this specification, when the program product is run on the terminal device.

Referring to fig. 17, a program product 1700 for implementing the above method according to an embodiment of the present application is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing. Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider). Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the present application, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A method for generating marketing assistance information, the method comprising:

2. The method of claim 1, wherein the constructing a knowledge graph using the shallow triple information and the deep triple information comprises:

importing the shallow triple information and the deep triple information into a graph database to form a knowledge graph in the graph database;

when a service problem retrieval request is received, a retrieval result is obtained by inquiring the knowledge graph, and marketing auxiliary information is generated according to the retrieval result, wherein the method comprises the following steps:

when a service problem retrieval request is received, acquiring a service problem in the service problem retrieval request;

determining a result template matched with the service problem;

inquiring the knowledge graph in the graph database to obtain a retrieval result corresponding to the service problem;

and filling the retrieval result into the result template to obtain marketing auxiliary information.

3. The method of claim 2, wherein determining the result template matching the business problem comprises:

determining the category of the business problem through a preset classification model;

determining a problem template matched with the business problem from the problem templates corresponding to the categories through a first syntactic analysis model, and using the problem template as a target problem template;

acquiring a result template corresponding to the target problem template as a result template matched with the service problem;

the querying the knowledge graph in the graph database to obtain a retrieval result corresponding to the business problem comprises:

extracting a query object in the business problem through a second syntactic analysis model;

and inquiring the knowledge graph in the graph database according to the inquiry object to obtain a retrieval result corresponding to the service problem.

4. The method of claim 1, further comprising: and according to the user request, the interpretation weight of each characteristic to the service data is output in a visual mode.

5. The method of claim 1, wherein said using said final data table to build a base classification prediction model comprises:

respectively training by utilizing the final data table to obtain a logistic regression model, a CART model and an Xgboost model;

and establishing a basic classification prediction model according to the logistic regression model, the CART model and the Xgboost model, wherein the prediction result of the basic classification prediction model is a weighted calculation result obtained by carrying out weighted calculation on the output results of the logistic regression model, the CART model and the Xgboost model.

6. The method according to claim 1, wherein performing a variable filtering operation on the data table to remove at least one variable in the data table to obtain a final data table includes:

and carrying out variable screening operation on the data table by sequentially utilizing a chi-square test method, a correlation coefficient calculation method and an information value evaluation method so as to remove at least one variable in the data table to obtain a final data table.

7. The method of claim 1, wherein the determining an average value of marginal contribution of each feature in the service data to the prediction result of the basic classification prediction model according to the service data associated with each user in the final data table by using the variable in the final data table as the feature to explain the service data comprises:

dividing the service data in the final data table into a plurality of layers according to a preset rule, wherein each layer comprises a plurality of service data;

selecting one service data from the final data table as selected service data,

for each feature, iteratively executing the step of determining the marginal contribution value until a predetermined number of times, wherein the step of determining the marginal contribution comprises: randomly generating a characteristic sequence, and respectively sequencing the selected service data and the service data in the final data table according to the characteristic sequence; selecting a layer from the unselected layers, and randomly selecting a service data from the service data of the layer as a constructed service data corresponding to the selected service data; respectively constructing first instance service data and second instance service data according to the selected service data and the constructed service data, wherein the first instance service data comprises variable values corresponding to the features in the selected service data and the features before the features and variable values corresponding to the features after the features in the constructed service data, and the second instance service data comprises variable values corresponding to the features in the selected service data before the features and variable values corresponding to the features in the constructed service data and the features after the features; respectively inputting the selected service data and the constructed service data into the basic classification prediction model to respectively obtain a first prediction result corresponding to the selected service data and a second prediction result corresponding to the constructed service data; determining a marginal contribution value corresponding to the feature based on the first prediction result and the second prediction result;

for each feature, determining an average marginal contribution value as an interpretation weight of the feature on the traffic data based on marginal contribution values obtained by performing the step of determining the marginal contribution value for the feature.

8. An apparatus for generating marketing assistance information, the apparatus comprising:

9. A computer-readable program medium, characterized in that it stores computer program instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that the electronic device comprises:

a processor;

a memory having stored thereon computer readable instructions which, when executed by the processor, implement the method of any of claims 1 to 7.