CN113064904B - Sketch construction method based on data self-learning - Google Patents
Sketch construction method based on data self-learning Download PDFInfo
- Publication number
- CN113064904B CN113064904B CN202110476312.XA CN202110476312A CN113064904B CN 113064904 B CN113064904 B CN 113064904B CN 202110476312 A CN202110476312 A CN 202110476312A CN 113064904 B CN113064904 B CN 113064904B
- Authority
- CN
- China
- Prior art keywords
- algorithm
- entity
- label
- data
- tag
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
Abstract
The invention discloses a portrait construction method based on data self-learning, which is characterized in that an algorithm is defined, corresponding entity algorithm authority is issued and authorized, a label is defined for an entity, and the corresponding relation between the label and the algorithm is bound; grouping a plurality of labels under an entity, and appointing a label list combination under each group; binding an entity with a data set, and specifying association conditions among the data sets; and constructing an entity portrait task. The method for constructing the portrait can more intuitively express the relationship between the entity and the portrait, more finely control the generation process of the label and the construction process of the portrait, and more flexibly adjust the realization process of the algorithm through the dynamic adjustment of the threshold parameter and the input parameter, thereby achieving the multiplexing capability of the algorithm. In addition, the accuracy of the label can be fed back dynamically through secondary correlation analysis of the grouping and the label, so that a basis is provided for adjustment of algorithm parameters.
Description
Technical Field
The invention relates to the technical field of data portrayal, and particularly provides a portrayal construction method based on data self-learning.
Background
Data mining is a process of processing and model training data by using a tool and an algorithm, so as to discover the association relationship between data and information hidden between data.
The data image is a process of abstracting data by using a tool, extracting typical characteristics contained in the data, giving a label name to the data, labeling data contents, and forming an image prototype based on certain description such as statistical elements, scenes and the like.
With the development of science and technology, people enter an intelligent era of everything interconnection, people-to-people communication is more frequent, data value application is prominent, and intelligent application of data is realized to become a tool for all walks of life. Enterprises, products, businesses, people and the like can understand the business conditions, the product use conditions, the business conditions, the health conditions of people and the like of the enterprises more deeply by means of data images, and the enterprises, the products, the businesses, the people and the like depend on data decision and reference provided by data, so that the enterprises, the products, the businesses, the people and the like are more scientific and more intelligent.
With the generation of mass data and the continuous development of big data technology, the barriers between data are broken, thereby realizing the interconnection and intercommunication of data and enabling the incidence relation between data to be explored and utilized. The construction of the portrait aims to more vividly and visually display the relationship between data and discover the value of the data, thereby playing a positive guiding role in the decision and behavior in reality. Most often, the data of the individual's behavior on the consuming internet is used to construct a representation of the user to achieve accurate marketing.
Disclosure of Invention
A common portrait construction system is used for marking data through preset rules and then performing label grouping so as to visually display portraits according to conditions, and the portrait construction method cannot meet the requirements of a portrait dynamic generation process, self-learning perfect management service and data release service of portraits and forwarding service of data interfaces, so that the diversity of different application system requirements is met. The invention realizes the dynamic learning of the portrait data and the automatic perfection of the portrait parameters on the basis of the existing portrait construction mode, namely, the portrait construction method based on the data self-learning.
In order to achieve the purpose, the invention provides the following technical scheme:
a portrait construction method based on data self-learning is disclosed, wherein the portrait construction method issues and authorizes corresponding entity algorithm authority by defining an algorithm, defines a label for an entity and binds the corresponding relation between the label and the algorithm;
grouping a plurality of labels under an entity, and appointing a label list combination under each group;
binding an entity with a data set, and specifying association conditions among the data sets;
and constructing an entity portrait task.
The method comprises the steps of performing secondary association analysis by taking a tag list of an entity portrait task as a data source, analyzing the relation between an entity and each tag, grading the portrait accuracy, giving an abnormal tag value according to the grading, and performing parameter optimization and logic optimization of a tag binding algorithm
The binding process of the entity and the data set comprises the following contents:
the data set corresponding to the entity comprises a plurality of data tables with different dimensions, each data table has a certain incidence relation, and the incidence relation among the related data tables is appointed during binding.
The process of constructing the entity portrait task comprises the following contents:
after an entity is selected, a detailed field list under the data set is displayed;
selecting fields in batches as required, binding input relations between the fields and the labels, designating execution sequences of the labels and statistical sequences of grouping, and constructing entity label tasks to obtain label results.
The specific implementation process is as follows:
a) selecting an entity in a canvas, selecting an available field in a popped up data set list
b) Selecting labels, setting threshold parameters and binding input parameter columns
c) Specifying tag result output paths
And the label result is synchronized into a target database for the visual presentation of the front-end portrait system, and a basis can be provided for the strategy formulation of accurate marketing of market analysts.
The method detects the accuracy of each label value by taking the label result as a self-learning data source, predicting and analyzing the result between each label value and the entity data set, and performs parameter optimization and logic adjustment on label values with overlarge deviation so as to achieve accurate presentation of the image.
The algorithm is visually constructed by selecting a logical combination of existing algorithms.
The algorithm is constructed by uploading a third party algorithm SDK;
and the uploaded third-party algorithm SDK inherits the API interface specified by the platform.
The authorization of the entity algorithm authority comprises the following contents:
a) authorized entities can view the algorithm in the algorithm list and show that the algorithm has the right of use;
b) an unauthorized entity may see the algorithm in the algorithm list, but not have access, and the entity may submit a request for use, waiting for review by the algorithm issuer.
The implementation process of defining the label for the entity and binding the corresponding relation between the label and the algorithm in the method comprises the following contents:
a) defining a label name;
b) parameters specifying a label, threshold parameters and input parameters, wherein:
threshold parameters: the parameter values which play a role of judging nodes in the budgeting process;
inputting parameters: data content to be tagged;
c) a tag output value is defined.
Compared with the prior art, the portrait construction method based on data self-learning has the following outstanding beneficial effects:
the method ensures that the portrait is closer to the entity in the real world by constructing more accurate data portrait, thereby achieving the purpose of guiding commercial behaviors. The method for constructing the portrait can more intuitively express the relationship between the entity and the portrait, more finely control the generation process of the label and the construction process of the portrait, and more flexibly adjust the realization process of the algorithm through the dynamic adjustment of the threshold parameter and the input parameter, thereby achieving the multiplexing capability of the algorithm. In addition, the accuracy of the label can be fed back dynamically through grouping and secondary correlation analysis of the label, so that a basis is provided for adjustment of algorithm parameters, the accuracy of the portrait is higher and higher in the continuous data self-learning process, and more accurate reference is provided for commercial application.
Drawings
FIG. 1 is a flow chart of a method implementation of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples.
As shown in FIG. 1, a sketch construction method based on data self-learning is realized by the following content schemes:
(1) the method comprises the steps of constructing an algorithm in a platform, uploading a three-party algorithm SDK package or self-defining through a visual panel, when the three-party algorithm SDK is uploaded, specifying the name of the algorithm, specifying input parameters and threshold parameters, describing and limiting the parameters, issuing after setting is completed, if the algorithm passes rule verification, issuing is successful, otherwise, corresponding error information is prompted, and the three-party algorithm SDK must inherit an algorithm interface well defined by the platform, otherwise, the algorithm cannot be identified. When the algorithm is customized through the visual panel, the existing algorithm can be logically combined, and the code can also be written through java or shell, corresponding input parameters and threshold parameters still need to be specified, and the parameters are described and limited.
(2) After the algorithm is constructed, the algorithm is required to be issued, the authority of an entity to the algorithm is required to be appointed when the algorithm is issued, and the authority configuration information is as follows:
and (3) entity selection:
| properties | Description of the invention |
| All | All entities |
| Portion | Partial entity, entity list requiring multiple selection |
| Single | Single entity, need to singleton entity list |
And (3) selecting the authority:
| properties | Description of the invention |
| All | All rights, visible, available, editable |
| Editable | Visible, editable, unusable |
| Viewable | Visible, unavailable, uneditable |
| Usable | Visible, available, uneditable |
(3) In the platform, entities are defined with tags, tag names are assigned and algorithms are selected from an authorized list of algorithms, and threshold parameters are assigned. After the label definition of the entity is completed, the labels are combined to construct an entity group, and the labels in the group are noticed to be non-conflicted, namely the logic meanings of the two labels cannot be conflicted, so that the grouping is ensured not to have entity data which does not conform to the group due to the logic conflict between the labels.
(4) The data set binding is performed on the entities, one entity includes multiple features, so that the data set corresponding to one entity generally includes multiple data tables with different dimensions, the data tables have a certain association relationship, and the association relationship of the related data tables needs to be specified when the entities are bound, as follows:
suppose that the data set of entity "person" has three tables basc _ message, income _ message, and content _ message, i.e. basic information, income information, and consumption information, and the field information is as follows:
a) and specifying the association relationship between the basic information table and the income information table:
basc_message.basc_id＝income_message.income_id
b) specifying the association relationship between the basic information table and the consumption information table:
basc_message.basc_id＝consume_message.consume_id
c) and specifying the association relationship between the income information table and the consumption information table:
income_message.income_id＝consume_message.consume_id
(5) after the data set is bound, entity tasks need to be constructed, a detailed field list under the data set can be automatically displayed after an entity is selected, fields are selected in batches as required, then the input relation between the fields and the labels is bound, and the execution sequence of the labels and the grouping statistical sequence are specified.
a) The execution process of the label A needs to use the algorithm a1 and the algorithm a2
b) The execution process of the label B requires the algorithm a1 and the algorithm a3
c) The execution process of the label C requires the algorithm a2 and the algorithms a3 and a4
d) Assuming that the algorithms a1, a2, a3 and a4 are involved in the entity task, the time consumption of each algorithm is t1, t2, t3 and t4
e) The most common tag execution process takes time as: time consuming for tag a + time consuming for tag B + time consuming for tag C t1+ t2+ t1+ t3+ t2+ t3+ t4 ═ 2t1+2t2+2t3+ t4
f) The execution process after optimization through the data self-learning algorithm is that the execution result of the a1 algorithm is used by the label a and the label B, the execution result of the a2 algorithm is used by the label a and the label C, and the execution result of the a3 algorithm is used by the label B and the label C, so the whole execution process takes time: t1+ t2+ t3+ t4
g) Assuming that t1, t2, t3 and t4 respectively mean that the time consumption before optimization is 2t1+2t2+2t3+ t4 and 7t1 respectively, and the time consumption after optimization is t1+ t2+ t3+ t4 and 4t1 respectively, the performance improvement is more obvious as the recombination rate of the algorithm used by the label is higher.
(6) After the entity tag task is established, the tag result needs to be synchronized into a target database for visual presentation of a front-end portrait system, and a basis is provided for accurate marketing strategy formulation of market analysts.
(7) Another application of the label results is as a self-learning data source, the accuracy of each label value is detected through result prediction and correlation analysis between each label value and the entity data set, and parameter optimization and logic adjustment are performed on label values with excessive deviation so as to achieve accurate representation of the image.
a) Assuming A, B, C, D, E five labels, theoretically, a relatively complete image of the data can be displayed by the five labels.
b) And performing correlation analysis on the tag result A, B, C, D and the entity data to predict the result conformity of the E.
c) When the conformity of E is not expected, the threshold parameter of E is adjusted.
d) The above analysis and adjustment processes are continuously performed on A, B, C, D, E five labels in sequence, so that the accuracy of the final digital portrait is ensured to be higher and higher.
The above-described embodiments are merely preferred embodiments of the present invention, and general changes and substitutions by those skilled in the art within the technical scope of the present invention are included in the protection scope of the present invention.
Claims (8)
1. A portrait construction method based on data self-learning is characterized in that the method issues and authorizes corresponding entity algorithm authority through defining an algorithm, defines a label for an entity and binds the corresponding relation between the label and the algorithm;
grouping a plurality of labels under an entity, and appointing a label list combination under each group;
binding an entity with a data set, and specifying association conditions among the data sets; the binding process of the entity and the data set comprises the following contents: the data set corresponding to the entity comprises a plurality of data tables with different dimensions, the data tables have an incidence relation, and the incidence relation between the related data tables is specified during binding, as follows:
the data set comprises three data tables basc _ message, income _ message and content _ message, namely basic information, income information and consumption information, wherein basc _ id in the basic information tables represents an identity card number; income _ id in the income information table represents an identity card number; the consumejd in the consumption information table represents the identity card number;
a) and specifying the association relationship between the basic information table and the income information table:
basc_message.basc_id＝income_message.income_id
b) specifying the association relationship between the basic information table and the consumption information table:
basc_message.basc_id＝consume_message.consume_id
c) and specifying the association relationship between the income information table and the consumption information table:
income_message.income_id＝consume_message.consume_id
constructing an entity portrait task; after the binding of the data set is completed, an entity task is required to be constructed, a detailed field list under the data set can be automatically displayed after an entity is selected, fields are selected in batches according to requirements, then the input relation between the fields and the tags is bound, the execution sequence of the tags and the grouping statistical sequence are designated, and an entity tag task is constructed to obtain a tag result;
a) the execution process of the tag A needs to use the algorithm a1 and the algorithm a 2;
b) the execution process of the label B requires the algorithm a1 and the algorithm a 3;
c) the execution process of the label C requires the algorithm a2 and the algorithms a3 and a 4;
d) in the entity task, the time consumption of each algorithm is t1, t2, t3 and t4 respectively as the inclusion of algorithms a1, a2, a3 and a4 is the same;
e) the most common tag execution process takes time as: time spent on tag a + time spent on tag B + time spent on tag C t1+ t2+ t1+ t3+ t2+ t3+ t4 ═ 2t1+2t2+2t3+ t 4;
f) the execution process after optimization through the data self-learning algorithm is that the execution result of the a1 algorithm is used by the label a and the label B, the execution result of the a2 algorithm is used by the label a and the label C, and the execution result of the a3 algorithm is used by the label B and the label C, so the whole execution process takes time: t1+ t2+ t3+ t 4;
g) when t1 is t2 is t3 is t4, the time consumption before optimization is 2t1+2t2+2t3+ t4 is 7t1, and the time consumption after optimization is t1+ t2+ t3+ t4 is 4t 1.
2. The sketch construction method based on data self-learning of claim 1, wherein the method comprises performing secondary association analysis on a tag list of an entity sketch task as a data source, analyzing the relationship between an entity and each tag, scoring the sketch accuracy, and performing parameter optimization and logic optimization of a tag binding algorithm according to the scores to give abnormal tag values.
3. The portrait construction method based on data self-learning of claim 1, wherein the tag result is synchronized to a target database for visual presentation by a front-end portrait system, and provides a basis for accurate marketing strategy development by market analysts.
4. The method for constructing a portrait based on data self-learning of claim 1, wherein the method detects the accuracy of each tag value by using the tag result as a self-learning data source, performing result prediction and correlation analysis between each tag value and the entity data set, and performing parameter optimization and logic adjustment for tag values with excessive deviation so as to achieve accurate representation of the portrait.
5. An image construction method based on data self-learning as claimed in claim 1, wherein the algorithm is constructed visually by selecting a logical combination of existing algorithms.
6. An image construction method based on data self-learning as claimed in claim 1, wherein the algorithm is constructed by uploading a third party algorithm SDK;
and the uploaded third-party algorithm SDK inherits the API interface specified by the platform.
7. A sketch construction method based on data self-learning as claimed in claim 1, wherein the authorization of the entity algorithm authority includes the following contents:
a) the authorized entity looks up the algorithm in the algorithm list and shows that the authorized entity has the right of use;
b) an unauthorized entity sees the algorithm in the algorithm list, but has no access rights, and submits an application for use, waiting for review by the algorithm issuer.
8. A sketch construction method based on data self-learning as claimed in claim 1, wherein the implementation process of defining a label for an entity and binding the corresponding relationship between the label and the algorithm comprises the following steps:
a) defining a label name;
b) parameters specifying a label, threshold parameters and input parameters, wherein: threshold parameters: the parameter values which play a role of judging nodes in the budgeting process; inputting parameters: data content to be tagged;
c) a tag output value is defined.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110476312.XA CN113064904B (en) | 2021-04-29 | 2021-04-29 | Sketch construction method based on data self-learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110476312.XA CN113064904B (en) | 2021-04-29 | 2021-04-29 | Sketch construction method based on data self-learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113064904A CN113064904A (en) | 2021-07-02 |
| CN113064904B true CN113064904B (en) | 2022-04-08 |
Family
ID=76567955
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110476312.XA Active CN113064904B (en) | 2021-04-29 | 2021-04-29 | Sketch construction method based on data self-learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113064904B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109711874A (en) * | 2018-12-17 | 2019-05-03 | 平安科技（深圳）有限公司 | User's portrait generation method, device, computer equipment and storage medium |
| CN110442761A (en) * | 2019-06-21 | 2019-11-12 | 深圳中琛源科技股份有限公司 | A kind of user draws a portrait construction method, electronic equipment and storage medium |
| CN111553729A (en) * | 2020-04-27 | 2020-08-18 | 广州探途网络技术有限公司 | Method and device for generating portrait data of e-commerce user and computing equipment |
| CN111915366A (en) * | 2020-07-20 | 2020-11-10 | 上海燕汐软件信息科技有限公司 | User portrait construction method and device, computer equipment and storage medium |
| WO2020248131A1 (en) * | 2019-06-11 | 2020-12-17 | 深圳市欢太科技有限公司 | Method for creating user persona, and related product |
| CN112100256A (en) * | 2020-08-06 | 2020-12-18 | 北京航空航天大学 | Data-driven urban accurate depth image system and method |
| WO2020252639A1 (en) * | 2019-06-17 | 2020-12-24 | 深圳市欢太科技有限公司 | Content pushing method and related product |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106980623B (en) * | 2016-01-18 | 2020-02-21 | 华为技术有限公司 | Data model determination method and device |
| CN112232909A (en) * | 2020-10-13 | 2021-01-15 | 汉唐信通(北京)科技有限公司 | Business opportunity mining method based on enterprise portrait |
-
2021
- 2021-04-29 CN CN202110476312.XA patent/CN113064904B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109711874A (en) * | 2018-12-17 | 2019-05-03 | 平安科技（深圳）有限公司 | User's portrait generation method, device, computer equipment and storage medium |
| WO2020248131A1 (en) * | 2019-06-11 | 2020-12-17 | 深圳市欢太科技有限公司 | Method for creating user persona, and related product |
| WO2020252639A1 (en) * | 2019-06-17 | 2020-12-24 | 深圳市欢太科技有限公司 | Content pushing method and related product |
| CN110442761A (en) * | 2019-06-21 | 2019-11-12 | 深圳中琛源科技股份有限公司 | A kind of user draws a portrait construction method, electronic equipment and storage medium |
| CN111553729A (en) * | 2020-04-27 | 2020-08-18 | 广州探途网络技术有限公司 | Method and device for generating portrait data of e-commerce user and computing equipment |
| CN111915366A (en) * | 2020-07-20 | 2020-11-10 | 上海燕汐软件信息科技有限公司 | User portrait construction method and device, computer equipment and storage medium |
| CN112100256A (en) * | 2020-08-06 | 2020-12-18 | 北京航空航天大学 | Data-driven urban accurate depth image system and method |
Non-Patent Citations (2)
| Title |
|---|
| 基于大数据的电子商务用户画像构建研究;李佳慧等;《电子商务》;20190115(第01期);46-49 * |
| 基于用户画像的读者周边好书推荐服务研究;解娜;《情报探索》;20200815(第08期);109-113 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113064904A (en) | 2021-07-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021179720A1 (en) | Federated-learning-based user data classification method and apparatus, and device and medium | |
| US7367011B2 (en) | Method, system and program product for developing a data model in a data mining system | |
| Vahidi et al. | Describing the necessity of multi-methodological approach for viable system model: case study of viable system model and system dynamics multi-methodology | |
| Delen et al. | An integrated and intelligent DSS for manufacturing systems | |
| CN110852881B (en) | Risk account identification method and device, electronic equipment and medium | |
| Dasarathy | Image data compression: block truncation coding | |
| CN111159241A (en) | Click conversion estimation method and device | |
| CN113064904B (en) | Sketch construction method based on data self-learning | |
| Kiviat | Simulation, technology, and the decision process | |
| Seidmann et al. | Microcomputer selection process for organizational information management | |
| Tolstykh et al. | The Control of Continuing Education Based on the Digital Economy | |
| CN110765110A (en) | Generalization capability processing method, device, equipment and storage medium | |
| Kaboudan | A dynamic-server queuing simulation | |
| CN111882415A (en) | Training method and related device of quality detection model | |
| CN111179076A (en) | IT system intelligent management method, IT system intelligent management device and computer readable storage medium | |
| Loumos et al. | Change management and quality of service through business process modeling: The n-vis, a public sector project | |
| Hassannezhad et al. | A cybernetic participatory approach for policy system of systems mapping: Case study of inclusive economies | |
| Doszhan et al. | QuALITATIVE ASSESSMENT OF THE DEVELOPMENT OF CREATIVE INDuSTRIES IN EMERGING COuNTRIES: THE CASE OF KAzAKHSTAN | |
| WO2022114364A1 (en) | Method and apparatus for applying user intent in unsupervised learning | |
| Hassannezhad et al. | A Cybernetic Participatory Approach for Whole-Systems Modelling and Analysis, with Application to Inclusive Economies | |
| CN114943406A (en) | Test paper image labeling method and device, storage medium and electronic equipment | |
| Weitzel et al. | Towards an interdisciplinary theory of networks. | |
| Patria et al. | Industrial base sustainment through risk modeling with model-based systems engineering applications | |
| Setiawati | The Conceptual Framework of Corporate Social Responsibility for Indonesia: Learning Study of CSR in Vietnam | |
| CN117539857A (en) | Table splicing method, apparatus, storage medium and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |