CN113761390B - Method and system for analyzing attribute intimacy - Google Patents
Method and system for analyzing attribute intimacy Download PDFInfo
- Publication number
- CN113761390B CN113761390B CN202110955230.3A CN202110955230A CN113761390B CN 113761390 B CN113761390 B CN 113761390B CN 202110955230 A CN202110955230 A CN 202110955230A CN 113761390 B CN113761390 B CN 113761390B
- Authority
- CN
- China
- Prior art keywords
- attribute
- analysis
- index
- rule
- affinity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9536—Search customisation based on social or collaborative filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides an analysis method and system for attribute intimacy, which comprises the steps of extracting all associated attribute information in an original data packet, storing the attribute information in a relational database, and storing an analysis index of an attribute association record in an index database, wherein the analysis index comprises a starting attribute, a terminating attribute and an associated edge; traversing the index database, determining whether a record exists in the index database for analyzing the index, if not, executing an unassociated attribute intimacy degree analysis engine, and importing an analysis result into an association edge; if yes, executing the associated attribute intimacy degree adjusting engine, and updating the attribute intimacy degree analysis result to the associated edge; and acquiring all correlation attribute relations of the current analysis index, and updating the attribute affinity analysis result to the correlation edge of the current analysis index if the common correlation attribute exists between the initial attribute and the termination attribute of the current analysis index according to the attribute affinity adjustment rule. The method and the system can quickly analyze the intimacy degree between different attributes and are automatic and accurate.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to an analysis method and system for attribute intimacy.
Background
The original data accessed by the big data system is generated by depending on different business requirements and tools, the industry span is large, the unified business data model standard and description specification are lacked, the data has the characteristics of isomerism, ambiguity, large noise and the like, and the data information content and the value density are diluted by mass noise data. The data is full of a large amount of noise, a large amount of isolated and useless entity and attribute relations are generated, the intimacy between different attributes is analyzed by a traditional method and is usually calculated according to a fixed rule, the analysis result of the traditional method is low in accuracy and more in redundant data, the difficulty in business study and judgment is increased, and the main factor for restricting the business study and judgment of a large data system service is formed.
Because the original data accessed by a big data system has the characteristics of isomerism, ambiguity, large noise and the like, the existing attribute affinity analysis method in the market at present carries out calculation according to a fixed rule, and the technologies have the following defects:
1. the attribute intimacy degree analysis range is small, the intimacy degree of the attribute relation can only be calculated and analyzed according to a single condition due to the fact that the conventional method only calculates according to a fixed rule, calculation cannot be automatically carried out according to the service characteristics, the optimal result of the attribute intimacy degree is further obtained, and therefore a plurality of potential valuable clues cannot be automatically mined.
2. The accuracy of attribute intimacy analysis is low, the traditional method can calculate the value of attribute intimacy only after data meeting established rules come in, but in many practical scenes, various original data in new formats need to be accessed, so that the analysis result of attribute intimacy cannot be adjusted in time, the analysis result is repeated or wrong, and the value of researching and judging big data service business is influenced.
Disclosure of Invention
In order to solve a series of technical problems of small attribute intimacy degree analysis range, low accuracy and the like in the prior art, the invention provides an attribute intimacy degree analysis method and system to solve the technical problems.
According to a first aspect of the present invention, there is provided an analysis method for attribute intimacy degree, comprising:
s1: extracting all associated attribute information in an original data packet, storing the attribute information in a relational database, and storing an analysis index of an attribute association record in an index database, wherein the analysis index comprises a starting attribute, a terminating attribute and an associated edge;
s2: traversing the index library, determining whether a record exists in the index library for analyzing the index, if not, executing an unassociated attribute intimacy degree analysis engine, and importing an analysis result into an association edge; if yes, executing the associated attribute intimacy degree adjusting engine, and updating the attribute intimacy degree analysis result to the associated edge; and
s3: traversing the index library, acquiring all the association attribute relations of the current analysis index, and updating the attribute affinity analysis result to the association edge of the current analysis index according to the attribute affinity adjustment rule if the common association attribute exists between the initial attribute and the termination attribute of the current analysis index.
In some specific embodiments, in response to that each attribute association relationship is stored in the relationship database, a new attribute affinity is added to the scheduling task table to calculate the to-be-processed task, and the to-be-processed task is allocated according to the idle condition of the calculation resource. By means of the setting, the influence on the warehousing efficiency of mass data can be avoided.
In some specific embodiments, the starting attribute and the ending attribute in the analysis index are used as arguments, and all attribute affinity analysis rules of the attribute are retrieved from the attribute affinity analysis rule library to form a rule analysis set Pn, where elements of the rule analysis set Pn include a data source, an operation rule, an operation result threshold, a weight, a competition factor, and an interference factor, and the rule analysis set Pn is arranged in a reverse order of the weight.
In some particular embodiments, the unassociated attribute affinity analysis engine includes: traversing the rule analysis set Pn, if the data source of the rule analysis set Pn is the same as the data source in the analysis index, obtaining a result set Mn according to the operation rule, and if the operation result of the result set Mn is greater than the operation result threshold of the rule analysis set Pn, the attribute affinity of the analysis index is the product of the operation result of the result set Mn and the weight. The affinity relationship between the attributes can be quickly analyzed and obtained by utilizing an unassociated attribute affinity analysis engine.
In some specific embodiments, the associated attribute affinity adjustment engine adjusts the associated attribute affinity according to the competition factor of the rule analysis set Pn by performing the associated attribute affinity competition adjustment. Adjusting the associated attribute affinities by the competition factor update may improve the accuracy of the attribute affinities.
In some specific embodiments, the competition factor includes an account opening date, and if the account opening date of the newly added association is later than the old association in the index library, the attribute affinity analysis result is updated to the association edge.
According to a second aspect of the present invention, there is provided a computer readable storage medium having stored thereon one or more computer programs which, when executed by a computer processor, implement the method of any of the above.
According to a third aspect of the present invention, there is provided an analysis system for attribute intimacy, the system comprising:
a database and index database construction unit: the method comprises the steps that configuration is used for extracting all associated attribute information in an original data packet and storing the attribute information in a relational database, and storing an analysis index of an attribute association record in an index database, wherein the analysis index comprises a starting attribute, a terminating attribute and an associated edge;
an intimacy degree analysis adjusting unit: the method comprises the steps that the index library is configured to be traversed, whether records exist in the index library in the analysis indexes or not is determined, if the records do not exist in the index library, an unassociated attribute intimacy degree analysis engine is executed, and an analysis result is led into an association edge; if yes, executing the associated attribute intimacy degree adjusting engine, and updating the attribute intimacy degree analysis result to the associated edge;
an association attribute affinity promotion adjustment unit: and updating the attribute affinity analysis result to the association edge of the current analysis index if the common association attribute exists between the initial attribute and the termination attribute of the current analysis index according to the attribute affinity adjustment rule.
In some specific embodiments, the method further includes: and the configuration is used for responding to the fact that each attribute incidence relation is stored in the relation database, newly adding an attribute intimacy degree in the scheduling task table to calculate the task to be processed, and distributing the task to be processed according to the idle condition of the calculation resources. The arrangement of the scheduling unit can avoid influencing the warehousing efficiency of mass data.
In some specific embodiments, the starting attribute and the ending attribute in the analysis index are used as arguments, and all attribute affinity analysis rules of the attribute are retrieved from the attribute affinity analysis rule library to form a rule analysis set Pn, where elements of the rule analysis set Pn include a data source, an operation rule, an operation result threshold, a weight, a competition factor, and an interference factor, and the rule analysis set Pn is arranged in a reverse order of the weight.
In some specific embodiments, the unassociated attribute affinity analysis engine comprises: traversing the rule analysis set Pn, if the data source of the rule analysis set Pn is the same as the data source in the analysis index, obtaining a result set Mn according to the operation rule, and if the operation result of the result set Mn is greater than the operation result threshold of the rule analysis set Pn, the attribute affinity of the analysis index is the product of the operation result of the result set Mn and the weight. The affinity relationship between the attributes can be quickly analyzed and obtained by utilizing an unassociated attribute affinity analysis engine.
In some specific embodiments, the associated attribute affinity adjustment engine adjusts the associated attribute affinity according to the competition factor of the rule analysis set Pn by performing the associated attribute affinity competition adjustment. Adjusting the associated attribute affinities by the competition factor update may improve the accuracy of the attribute affinities.
In some specific embodiments, the competition factor includes an account opening date, and if the account opening date of the newly added association is later than the old association in the index library, the attribute affinity analysis result is updated to the association edge.
The invention provides an analysis method and a system for attribute intimacy, which mainly utilize two algorithms of unassociated attribute intimacy analysis and associated attribute intimacy adjustment, take pairwise attribute combination as a reference, combine factors such as data elements, sources, operation types, rules, timeliness and the like, quickly analyze intimacy of each group of attribute relationship, find associated attribute nodes and automatically and accurately adjust intimacy between associated attributes. By using two algorithms of correlation attribute intimacy degree competitive adjustment and correlation attribute intimacy degree promoting adjustment, intimacy degree adjustment with the same attribute type and different values is met, and intimacy degree adjustment of correlation attribute relations is met when intimacy degree of a certain pair of attribute relations changes, analysis timeliness of the attribute intimacy degree is greatly improved, and accuracy of business application is guaranteed. The invention accurately adjusts the intimacy of the associated node, each analysis link is asynchronously executed, and the analysis result is backfilled into the relational database, thereby not only utilizing the computing resource to the maximum extent, but also automatically adjusting the attribute intimacy more quickly.
Drawings
The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and together with the description serve to explain the principles of the invention. Other embodiments and many of the intended advantages of embodiments will be readily appreciated as they become better understood by reference to the following detailed description. Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is a flow chart of an analysis method for attribute intimacy density according to one embodiment of the present application;
FIG. 2 is a schematic diagram of the main steps of an analysis method for attribute intimacy in accordance with a specific embodiment of the present application;
FIG. 3 is a diagram illustrating an attribute association relationship according to a specific embodiment of the present application;
FIG. 4 is a flow chart of an analysis method for attribute intimacy degree according to a specific embodiment of the present application;
FIG. 5 is a block diagram of an analysis system for attribute intimacy degree according to an embodiment of the present application;
FIG. 6 is a schematic block diagram of a computer system suitable for use to implement the electronic device of an embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 shows a flowchart of an analysis method for attribute intimacy degree according to an embodiment of the present application. As shown in fig. 1, the method includes:
s101: extracting all associated attribute information in the original data packet, storing the attribute information in a relational database, and storing an analysis index of the attribute association record in an index database, wherein the analysis index comprises a starting attribute, a terminating attribute and an associated edge.
In a specific embodiment, in response to that each attribute incidence relation is stored in the relational database, an attribute affinity degree calculation task to be processed is newly added in the scheduling task table, and the task to be processed is distributed according to the idle condition of the calculation resources, so that the storage efficiency of mass data is prevented from being influenced.
S102: traversing the index database, determining whether a record exists in the index database for analyzing the index, if not, executing an unassociated attribute intimacy degree analysis engine, and importing an analysis result into an association edge; if yes, executing the associated attribute intimacy degree adjusting engine, and updating the attribute intimacy degree analysis result to the associated edge.
In a specific embodiment, the initial attribute and the end attribute in the analysis index are used as parameters, all attribute affinity analysis rules of the attribute are retrieved from an attribute affinity analysis rule library to form a rule analysis set Pn, elements of the rule analysis set Pn include a data source, an operation rule, an operation result threshold, a weight, a competition factor and an interference factor, and the rule analysis set Pn is arranged in a reverse order of the weight. The unassociated attribute affinity analysis engine comprises: traversing the rule analysis set Pn, if the data source of the rule analysis set Pn is the same as the data source in the analysis index, obtaining a result set Mn according to the operation rule, and if the operation result of the result set Mn is greater than the operation result threshold of the rule analysis set Pn, the attribute affinity of the analysis index is the product of the operation result of the result set Mn and the weight. The associated attribute intimacy degree adjusting engine adjusts the associated attribute intimacy degree according to the competitive factor of the rule analysis set Pn through the associated attribute intimacy degree competitive adjustment, in a specific embodiment, the competitive factor may be an account opening date, if the account opening date of the newly added associated relationship is later than the old associated relationship in the index database, the attribute intimacy degree analysis result is updated to the associated edge, and the competitive factor may also be other parameters such as a time limit and the like, and may be used as the intimacy degree basis for judging the newly added associated relationship.
S103: traversing the index library, acquiring all the association attribute relations of the current analysis index, and updating the attribute affinity analysis result to the association edge of the current analysis index according to the attribute affinity adjustment rule if the common association attribute exists between the initial attribute and the termination attribute of the current analysis index. And finding out the attribute relation associated with the attribute, calling a corresponding attribute intimacy degree adjusting engine according to the attribute intimacy degree adjusting rule, deleting the old attribute intimacy degree analysis result on the edges of the two attribute nodes, and adding a new attribute intimacy degree analysis result.
In a specific embodiment, the method aims to construct an attribute relationship and an affinity calculation set of the associated attribute relationship through an analysis program, automatically analyze the affinity of attribute association, adjust the affinity of the associated attribute in time, backfill an analysis result into a relationship database, improve the working efficiency and accuracy of attribute affinity analysis, and support a user to quickly extract a most useful clue of the highest affinity. The attribute intimacy degree analysis process is mainly based on the following two core libraries: attribute affinity analysis rule base: the rule of calculating the intimacy degree associated with different attributes is obtained and is defined as table 1:
TABLE 1 Attribute intimacy analysis rule base
An attribute intimacy adjustment rule base: the adjustment rule for obtaining the affinity of each associated attribute is defined as table 2:
TABLE 2 Attribute intimacy adjustment rule Table
| Attribute name | Attribute description | Remarks to note |
| Id | Record Id | |
| RuleID | Rule ID | |
| CXLS | Starting type | 1-person, 2-mobile phone, 3-bank card, 4-QQ … |
| ZXLS | Type of termination | 1-person, 2-mobile phone, 3-bank card, 4-QQ … |
| BCL | Adjusting rules | |
| zt | State of state | 0-not available, 1-available |
In addition, an attribute affinity level table is also defined to represent the definition of different levels, which is specifically shown in table 3:
TABLE 3 Attribute intimacy level Table
Fig. 2 is a schematic diagram illustrating the main steps of an analysis method for attribute intimacy degree according to a specific embodiment of the present invention, as shown in fig. 2, which mainly includes:
step 201: and analyzing and warehousing the data. The method comprises the steps of accessing an original data packet of a big data system, extracting the data packet through an analysis and storage engine, storing all related attribute information contained in the data packet into a relational database, wherein each attribute related record comprises two attribute nodes and a relational edge, the attribute nodes are stored in a Key-Value mode, all keys are coded by adopting Base64, in order to improve retrieval efficiency, the relational edges contain two special attributes of _ from and _ to, and then the two nodes and the edges are used as unique analysis indexes T (a starting attribute type, a terminating attribute type and a related edge) to be stored into an index Base. Example (c): one interface vehicle passing information is to extract attribute information of association relations such as person-vehicle, person-mobile phone, person-ETC, mobile phone-QQ card, etc., and after data analysis and storage, the attribute information is stored in a relational database, as shown in the attribute association relation in FIG. 3.
Step 202: and (4) scheduling tasks. After each attribute incidence relation is stored in the relational database, a task to be processed with attribute affinity calculation is newly added in the scheduling task table, and the operation of the scheduling task is distributed by the platform according to the idle condition of the calculation resources, so that the storage efficiency of mass data is prevented from being influenced.
Step 203: and (6) judging rules. Traversing the index library, and judging whether the T (initial attribute, termination attribute and associated edge) has repeated records in the index library, if the two attributes of the current processing task have no association relationship, the intimacy is low, and if the two attributes have no association relationship, the intimacy indicates that the intimacy of the associated attribute has been stored before.
Step 204: and performing affinity analysis on the unassociated attributes. If the attribute is not associated, executing an unassociated attribute intimacy degree analysis engine, and marking the analysis result on the edges of the two nodes to obtain an intimacy degree analysis result of the two attributes.
Step 205: the associated attribute affinity is adjusted. If the record is old record, analyzing the change factors of the association relation of the current attribute, executing the associated attribute intimacy degree adjusting engine, deleting the old attribute intimacy degree analysis result on the edges of the two attribute nodes, and adding a new attribute intimacy degree analysis result.
Step 206: and adjusting the chain attribute intimacy. Traversing the index library, finding out the attribute relation associated with the attribute, calling a corresponding attribute affinity adjustment engine according to the attribute affinity adjustment rule, deleting the old attribute affinity analysis result on the edges of the two attribute nodes, and adding a new attribute affinity analysis result.
With continued reference to fig. 4, fig. 4 is a flow chart illustrating an analysis method for attribute intimacy degree according to a specific embodiment of the present invention, as shown in fig. 4, the method including:
s401: analyzing and warehousing original data, and storing the original data into a relational database in a Key-Value mode;
s402: task scheduling, wherein after an attribute association relation is stored in a relational database, a task to be processed with attribute affinity calculation is newly added in a scheduling task table, and the operation of the scheduling task is distributed by a platform according to the idle condition of computing resources, so that the storage efficiency of mass data is prevented from being influenced;
s403: rule judgment, namely traversing the index database, and judging whether the T (initial attribute, termination attribute and associated edge) has repeated records in the index database, namely whether the T is a new relation;
s404: judging whether the relationship is a new relationship, if so, performing unassociated attribute affinity analysis in the step S405, and storing the result in a relationship database in the step S401, and if not, performing associated attribute affinity analysis in the step S406, and storing the result in the relationship database in the step S401;
s407: and adjusting the chain attribute intimacy degree, and saving the result to a relational database in S401.
Through 2 algorithms of unassociated attribute intimacy degree analysis and associated attribute intimacy degree adjustment, pairwise attribute combination is taken as a reference, factors such as data elements, sources, operation types, rules and timeliness are combined, intimacy degree of each group of attribute relations is rapidly analyzed, associated attribute nodes are found out, intimacy degree between associated attributes is automatically and accurately adjusted, each analysis link is asynchronously executed, an analysis result is backfilled into a relation database, a user is supported, clues with low values are rapidly filtered, a troubleshooting range is reduced, and troubleshooting work efficiency is improved.
In a specific embodiment, taking attribute affinity analysis of an identity card-mobile phone number as an example, core content of the attribute affinity analysis is described as follows:
the first step is as follows: taking T (initial attribute type and termination attribute type) as a parameter, retrieving all attribute affinity analysis rules of the attribute from an attribute affinity analysis rule library to form a rule analysis set Pn, wherein the elements are as follows: SJLR data source, WSGZ operation rule, WSZGHZ operation result threshold, QZ weight, ZCRZ competition factor and GLRZ interference factor; the analysis sets Pn are arranged in reverse QZ order.
The second step: the unassociated attribute affinity analysis, traverse Pn,
if ([ Pn ]. SJLR ═ t. data source) { obtain operation rule WSGZ, obtain result Mn from WSGZ operation, traverse Mn,
if ([ Mn ]. operation result > [ Pn ]. WSZGHZ)
{ t. attribute affinity ═ Mn. weight ═ Mn. operation result },
}；
for example: the analysis rule base of the attribute affinity of the identity card and the mobile phone number has a rule of operation rule (mobile phone association QQ → QQ disassociates the identity card), the threshold value of an operation result is 5, and the weight is 1; the corresponding algorithm is: 1) acquiring last used imsi of a mobile phone, inquiring virtual identity table relationships, inquiring related QQ according to imsi, 2) sorting according to the last time, returning latest ten pieces of data, and acquiring QQ numbers with the number of login times being more than 5, 3) respectively acquiring QQ remarks from friend lists of QQ meeting conditions, and returning remark names and remark numbers of each QQ number, 4) removing father relation real names { "mother", "old woman", "son", "nephel" }; 5) and finally obtaining an attribute affinity result T of the association relationship between the mobile phone number and the name.
The third step: adjusting affinity competition of the associated attributes, namely adjusting the value of the affinity of the associated attributes according to [ Pn ] competition factors, wherein the affinity adjustment of the association relationships with the same attribute type but different values is mainly satisfied, for example: 130 x 5678 originally is the ID card A in use, the intimacy of 130 x 5678-A is high, but a new correlation relationship of 130 x 5678-B is judged according to [ Pn ] competition factors (opening date), the result of the intimacy analysis of 130 x 5678-B is found to be high, and the intimacy of 130 x 5678-A is changed from high to low.
The fourth step: the method mainly satisfies the condition that when the affinity of a certain pair of attribute relations changes, the affinity of the associated attribute relations correspondingly adjusts, obtains the associated attributes influenced by T from an attribute affinity adjusting rule base, and adjusts according to the adjusting rules, for example: 130 × 5678 cannot determine whether the id a is in use, and the attribute affinity of that 130 × 5678-a is low; when the intimacy degree of 130 x 5678-QQ (1 x 4) -remark name (A) is high, the intimacy degree analysis result of 130 x 5678-QQ (1 x 4) is set to be high, and the intimacy degree of 130 x 5678-A is changed from low to high.
By means of an analysis program, aiming at a scene of how to quickly and accurately analyze the attribute intimacy, an intimacy calculation set of the attribute relationship and the associated attribute relationship is constructed, the intimacy of the attribute association relationship is automatically analyzed, the intimacy of the associated attribute is adjusted in time, and the problem of how to quickly and accurately find the most useful clues for early warning according to the intimacy of the attribute relationship is solved. The method provides 2 algorithms of unassociated attribute intimacy degree analysis and associated attribute intimacy degree adjustment, takes pairwise attribute combination as a reference, combines factors such as data elements, sources, operation types, rules and timeliness, rapidly analyzes the intimacy degree of each group of attribute relations, automatically finds out associated attribute nodes, accurately adjusts the intimacy degree of the associated attributes, asynchronously executes each analysis link, backfills an analysis result into a relational database, supports a user to rapidly filter out clues with low values, reduces a troubleshooting range and improves troubleshooting working efficiency. The method is based on the scene of massive original data, the intimacy degree of the attribute relation generated by the massive original data is automatically analyzed, the analysis speed is high, the analysis result is accurate, the intimacy degree of the attribute is automatically analyzed and early warning is timely carried out, the work of manual analysis and judgment is greatly reduced, and the analysis efficiency is improved by more than 5 times compared with the manual analysis.
With continued reference to fig. 5, fig. 5 illustrates a block diagram of an analysis system for attribute intimacy, according to an embodiment of the present invention. The system specifically includes a database, an index base construction unit 501, an affinity analysis adjustment unit 502, and an associated attribute affinity promotion adjustment unit 503.
In a specific embodiment, the database and index base constructing unit 501 is configured to extract all associated attribute information in an original data packet and store the attribute information in a relational database, and store an analysis index of an attribute association record in an index base, where the analysis index includes a start attribute, a termination attribute, and an association edge; the affinity analysis adjusting unit 502 is configured to traverse the index library, determine whether there is a record in the index library for the analysis index, if not, execute the unassociated affinity analysis engine, and import the analysis result into the associated edge; if yes, executing the associated attribute intimacy degree adjusting engine, and updating the attribute intimacy degree analysis result to the associated edge; the correlation attribute affinity promoting and adjusting unit 503 is configured to traverse the index library, obtain all correlation attribute relationships of the current analysis index, and update the attribute affinity analysis result to the correlation edge of the current analysis index if a common correlation attribute exists between the start attribute and the end attribute of the current analysis index according to the attribute affinity adjusting rule.
Referring now to FIG. 6, shown is a block diagram of a computer system 600 suitable for use in implementing the electronic device of an embodiment of the present application. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the use range of the embodiment of the present application.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Liquid Crystal Display (LCD) and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. A driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that the computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609 and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the method of the present application when executed by a Central Processing Unit (CPU) 601. It should be noted that the computer readable storage medium of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable storage medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, or the like, as well as conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present application may be implemented by software or hardware.
As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable storage medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: extracting all associated attribute information in the original data packet, storing the attribute information in a relational database, and storing an analysis index of the attribute association record in an index database, wherein the analysis index comprises a starting attribute, a terminating attribute and an associated edge; traversing the index library, determining whether a record exists in the index library for analyzing the index, if not, executing an unassociated attribute intimacy degree analysis engine, and importing an analysis result into an association edge; if yes, executing the associated attribute intimacy degree adjusting engine, and updating the attribute intimacy degree analysis result to the associated edge; traversing the index library, acquiring all the association attribute relations of the current analysis index, and updating the attribute affinity analysis result to the association edge of the current analysis index according to the attribute affinity adjustment rule if the common association attribute exists between the initial attribute and the termination attribute of the current analysis index.
The foregoing description is only exemplary of the preferred embodiments of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (7)
1. An analytical method for attribute intimacy, comprising:
s1: extracting all associated attribute information in an original data packet, storing the attribute information in a relational database, and storing an analysis index of an attribute association record in an index database, wherein the analysis index comprises a starting attribute, a terminating attribute and an associated edge;
s2: traversing the index library, determining whether the analysis index has a record in the index library, if not, executing an unassociated attribute intimacy degree analysis engine, and importing an analysis result into the association edge; if yes, executing the associated attribute intimacy degree adjusting engine, and updating the attribute intimacy degree analysis result to the associated edge; and
s3: traversing the index library, acquiring all association attribute relations of the current analysis index, and updating an attribute affinity analysis result to the association edge of the current analysis index according to an attribute affinity adjustment rule if a common association attribute exists between the initial attribute and the termination attribute of the current analysis index;
taking the initial attribute and the termination attribute in the analysis index as reference, retrieving all attribute affinity analysis rules of the initial attribute and the termination attribute from an attribute affinity analysis rule library to form a rule analysis set Pn, wherein the elements of the rule analysis set Pn comprise a data source, an operation rule, an operation result threshold, a weight, a competitive factor and an interference factor, and the rule analysis set Pn is arranged in a weight reverse order;
the executing unassociated attribute affinity analysis engine comprises: traversing the rule analysis set Pn, if the data source of the rule analysis set Pn is the same as the data source in the analysis index, obtaining a result set Mn according to an operation rule, and if the operation result of the result set Mn is greater than the operation result threshold of the rule analysis set Pn, the attribute affinity of the analysis index is the product of the operation result of the result set Mn and the weight;
and the associated attribute intimacy degree adjusting engine adjusts the associated attribute intimacy degree according to the competitive factor of the rule analysis set Pn through the associated attribute intimacy degree competitive adjustment.
2. The method of claim 1, wherein in response to each attribute association being stored in the relational database, a new attribute affinity in the scheduling task table is added to calculate a task to be processed, and the task to be processed is allocated according to a computing resource vacancy condition.
3. The method as claimed in claim 1, wherein the competition factor includes an account opening date, and if the account opening date of the new association is later than the old association in the index database, the attribute affinity analysis result is updated to the association edge.
4. A computer-readable storage medium having one or more computer programs stored thereon, which when executed by a computer processor perform the method of any one of claims 1 to 3.
5. An analysis system for attribute intimacy, the system comprising:
a database and index database construction unit: the method comprises the steps that configuration is carried out, all associated attribute information in an original data packet is extracted and stored in a relational database, and an analysis index of attribute association records is stored in an index database, wherein the analysis index comprises a starting attribute, a terminating attribute and an associated edge;
an intimacy degree analysis adjusting unit: the system is configured to traverse the index library, determine whether the analysis index has a record in the index library, if not, execute an unassociated attribute intimacy degree analysis engine, and import an analysis result into the association edge; if yes, executing the associated attribute intimacy degree adjusting engine, and updating the attribute intimacy degree analysis result to the associated edge;
an association attribute affinity promotion adjustment unit: the index database is configured to be traversed to obtain all correlation attribute relations of the current analysis index, and according to an attribute affinity adjustment rule, if a common correlation attribute exists between the initial attribute and the termination attribute of the current analysis index, an attribute affinity analysis result is updated to the correlation edge of the current analysis index;
taking the initial attribute and the termination attribute in the analysis index as reference, retrieving all attribute affinity analysis rules of the initial attribute and the termination attribute from an attribute affinity analysis rule library to form a rule analysis set Pn, wherein the elements of the rule analysis set Pn comprise a data source, an operation rule, an operation result threshold, a weight, a competitive factor and an interference factor, and the rule analysis set Pn is arranged in a weight reverse order;
the executing unassociated attribute affinity analysis engine comprises: traversing the rule analysis set Pn, if the data source of the rule analysis set Pn is the same as the data source in the analysis index, obtaining a result set Mn according to an operation rule, and if the operation result of the result set Mn is greater than the operation result threshold of the rule analysis set Pn, the attribute affinity of the analysis index is the product of the operation result of the result set Mn and the weight;
and the associated attribute intimacy degree adjusting engine adjusts the associated attribute intimacy degree according to the competitive factor of the rule analysis set Pn through the associated attribute intimacy degree competitive adjustment.
6. The analysis system for attribute intimacy degree according to claim 5, further comprising a task scheduling unit: and the configuration is used for responding to the condition that each attribute incidence relation is stored in the relation database, newly adding an attribute affinity in a scheduling task table to calculate the task to be processed, and distributing the task to be processed according to the idle condition of the calculation resources.
7. The system of claim 5, wherein the competition factor comprises an account opening date, and if the account opening date of the new association is later than the old association in the index database, the attribute affinity analysis result is updated to the association edge.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110955230.3A CN113761390B (en) | 2021-08-19 | 2021-08-19 | Method and system for analyzing attribute intimacy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110955230.3A CN113761390B (en) | 2021-08-19 | 2021-08-19 | Method and system for analyzing attribute intimacy |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113761390A CN113761390A (en) | 2021-12-07 |
| CN113761390B true CN113761390B (en) | 2022-07-22 |
Family
ID=78790488
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110955230.3A Active CN113761390B (en) | 2021-08-19 | 2021-08-19 | Method and system for analyzing attribute intimacy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113761390B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115865712B (en) * | 2022-11-15 | 2024-08-02 | 北京众鼎景和科技有限公司 | Edge data storage method based on aging intimacy model |
| CN116578652B (en) * | 2023-07-13 | 2024-01-16 | 中国人民解放军国防科技大学 | Multi-table associated data set backfilling system and method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108897780A (en) * | 2018-06-06 | 2018-11-27 | 山东合天智汇信息技术有限公司 | A kind of method and system of analytical calculation personnel cohesion |
| CN109428928A (en) * | 2017-08-31 | 2019-03-05 | 腾讯科技（深圳）有限公司 | Selection method, device and the equipment of information push object |
| CN111917574A (en) * | 2020-07-21 | 2020-11-10 | 上海阿尔卡特网络支援系统有限公司 | Social network topology model and construction method thereof, user confidence degree and intimacy degree calculation method and telecommunication fraud intelligent interception system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170301028A1 (en) * | 2016-04-13 | 2017-10-19 | Gregory David Strabel | Processing system to generate attribute analysis scores for electronic records |
-
2021
- 2021-08-19 CN CN202110955230.3A patent/CN113761390B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109428928A (en) * | 2017-08-31 | 2019-03-05 | 腾讯科技（深圳）有限公司 | Selection method, device and the equipment of information push object |
| CN108897780A (en) * | 2018-06-06 | 2018-11-27 | 山东合天智汇信息技术有限公司 | A kind of method and system of analytical calculation personnel cohesion |
| CN111917574A (en) * | 2020-07-21 | 2020-11-10 | 上海阿尔卡特网络支援系统有限公司 | Social network topology model and construction method thereof, user confidence degree and intimacy degree calculation method and telecommunication fraud intelligent interception system |
Non-Patent Citations (1)
| Title |
|---|
| "群智感知中采用节点社会属性的亲密度量化方法";张文东等;《西安交通大学学报》;20170328;第51卷(第06期);第73-78页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113761390A (en) | 2021-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110019218B (en) | Data storage and query method and equipment | |
| CN113761390B (en) | Method and system for analyzing attribute intimacy | |
| CN111563103B (en) | Method and system for detecting data blood relationship | |
| CN107688626B (en) | Slow query log processing method and device and electronic equipment | |
| CN111382320A (en) | Large-scale data increment processing method for knowledge graph | |
| CN116126901A (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| CN110737727B (en) | Data processing method and system | |
| CN111125199A (en) | Database access method and device and electronic equipment | |
| CN109189343B (en) | Metadata disk-dropping method, device, equipment and computer-readable storage medium | |
| CN117807091A (en) | Data association method and device | |
| CN113094444B (en) | Data processing method, data processing device, computer equipment and medium | |
| Colosi et al. | Time series data management optimized for smart city policy decision | |
| CN113641705B (en) | Marketing disposal rule engine method based on calculation engine | |
| CN116010345A (en) | Method, device and equipment for realizing table service scheme of flow batch integrated data lake | |
| CN114036160A (en) | Real-time data acquisition method and device, electronic equipment and storage medium | |
| CN117472873A (en) | Data migration method, device, computing device cluster and storage medium | |
| CN114328606A (en) | Method, device and storage medium for improving SQL execution efficiency | |
| CN114048219A (en) | Graph database updating method and device | |
| CN114116908A (en) | Data management method and device and electronic equipment | |
| CN117421297B (en) | Statistical information collection method for dynamic partition table of distributed database | |
| CN114691749B (en) | Method for parallel incremental mining of frequent item sets based on sliding window | |
| CN111291040B (en) | Data processing method, device, equipment and medium | |
| CN118154139B (en) | Intelligent business process monitoring method and system based on AI | |
| CN118427186B (en) | Data blood edge tracing method, device, equipment and medium | |
| CN118535584A (en) | Data processing method, device, medium and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |